Veryfi Accuracy Reports Tool | Veryfi, Inc. Help Center

What is the Veryfi Accuracy Reports?

The Veryfi Accuracy Tool is a regression testing and performance monitoring system designed to track model performance across different releases. It helps you monitor improvements for specific support issues, vendors, or fields that aren't extracting correctly.

What APIs do Veryfi Accuracy Reports support?

Accuracy Reports support:

Receipts & Invoices ML Model

Bank Checks ML Model

W-9 Forms ML Model

Bank Statements ML Model

What can I use the Accuracy Tool for?

The tool is primarily used for:

Regression testing: Tracking performance changes across different model releases as well as dev and production releases
Performance monitoring: Measuring extraction accuracy over time
Issue tracking: Monitoring improvements for specific vendors, document types, or fields
Quality assurance: Validating that model updates improve (rather than degrade) extraction quality

Getting Started

How do I access the Accuracy Tool?

The Accuracy Tool is accessed through the Analytics > Accuracy Reports tab in your Veryfi dashboard.

Creating a Baseline

What is a Baseline?

A Baseline is your "ground truth" - a control set of documents that you have manually verified and corrected to ensure all relevant data is 100% accurate. This set will serve as the reference point for measuring data extraction accuracy over time, model over model. The Baseline provides the standard against which all future extractions are measured. Without an accurate Baseline, you can't reliably measure model performance or track improvements/regressions.

How do I Prepare Documents for the Baseline?

Step 0: Decide on the approach

Include documents that are:

Representative of your typical use case or problem, vendor, or field you want to track
Are of a good quality ( no blur, crumpleness, bleed-through)

Step 1: Tag your documents

Once decided on a dataset, you can either tag existing documents or re-process those one more time so they are at the top of your inbox
Tag documents with an identifier like "Ground truth" or "Baseline" or "Ground Truth for X issue"
Use clear, consistent tags for your ground truth documents
Example tags: "verified_vendor_name", "verified_line_items", "2024_Q1_review"

This makes them easy to locate when creating your baseline or modifying it

You can do it using Bulk Actions on the Document List view

Or Inside Document Details

Use Advanced Filters inside the Inbox to Filter data by Tags to locate your baseline

Step 2: Review and correct each document. Before adding documents to your Accuracy Report, manually verify and correct the ground truth data. Every field you plan to track must be perfectly accurate in your Baseline, as these become your "source of truth".

How to correct ground truth?

Navigate to the Inbox, filter by tag documents you want to add to your Baseine.

Press View and the system will load your Document Details. Use the Document details view to update extracted values.

IF a field is missing in your view, see if that field is toggled on for your view & save.

How to create an Accuracy Report?

Step 0: Go to Accuracy Reports

1. Press CREATE REPORT

2. Select Document Type

Step 1: Give your Report a Name

Give your reports clear names
- Use descriptive names that indicate purpose and scope
- Example: "Q1-2024_Vendor-Name-Accuracy" or "Healthcare-Invoice-Line-Items"
Add detailed descriptions
- Include the scope of what's being measured
- Note any special conditions or filters applied
- Document why this report was created

Step 2: Select Fields

Tick the fields you want to add to your report

Step 3: Select Documents

Here is where your previously tagged Baseline helps you to filter documents.

Step 4: Review & Confirm

Step 5: Run Trial

Pressing Run Trial will kick off a cycle of preprocessing your Baselines against the Production ML Model version

How do I compare model performance across releases?

After creating your initial Baseline, you can re-run the same Baseline against every new ML model versions to compare:

Overall accuracy changes
Improvements in specific fields
Vendor-specific performance
Line item extraction accuracy

How often should I run accuracy tests?

Run tests:

After each major model update or release
When investigating extraction issues

How is Matching Done?

Matching Algorithms are Explained in this Article

Interpreting Results

How do I view detailed results?

Click the "Show Details" button to investigate specific discrepancies and see detailed breakdowns of matches and mismatches.

What information does the results view show?

The results interface displays:

Matches: Fields correctly extracted compared to Baseline
Mismatches: Fields incorrectly extracted compared to Baseline

How can I investigate specific extraction errors?

View field discrepancies:

Hover over incorrect fields to see:
- Extracted value: What the OCR model extracted
- Baseline value: What the correct value should be (from your ground truth)

View the original document:

Click the hyperlink to open and view the actual document
Compare the visual document with extraction results

What should I look for in the results?

Focus on:

Recurring patterns: Do certain vendors or document types have lower accuracy?
Field-specific issues: Are certain fields (dates, line items, taxes) consistently extracted?
Improvement trends: Is accuracy improving or degrading over time?

How do I use results to improve extraction?

Identify problem areas: Use results to pinpoint specific vendors, fields, or document types with low accuracy
Document issues: Share specific examples with Veryfi support
Request improvements: Work with Veryfi to address recurring issues
Validate fixes: Re-run your Baseline after model updates to confirm improvements
Track progress: Maintain historical results to monitor trends over time