Scanned bank statements: how LedgerMap can read them

What I tested on

There were no scanned uploads in production yet (12 statements in total, all text PDFs). So I built three test sets. Two of them have exact answers.

A row counts as right only if its date, debit, credit and balance all match exactly. Narration is scored separately. Because the real-scan answers came partly from model agreement, they favour models that agree with Gemini and Claude Sonnet. The two other sets have no such bias.

Results

* Cloud times include the start-up of each command-line tool (10–20 s). A direct API call should take about 5–15 s per page, and pages of one statement can run at the same time. Costs are my estimate from the measured output size (about 820 output tokens per page) and the vendors' October 2026 prices. "Production" was run only with engines on this PC.

The mistakes the models made

All errors on the real scans fall into five kinds. These are the cases our checks must handle.

• A digit misread. GPT-6 Astra read a 6 as a 0 in an amount. The balance check flags it, and PP-OCR read the digit correctly.
• A guess where the page hides the value. A punched hole hid the first digit of a date. Gemini wrote a full date; Sonnet and Opus wrote "?". Where the dates on both sides fix the day (a date between 08/05 and 13/05 that ends in 3 must be 13/05), the guess was right. In one case it was not knowable. No automatic check can catch a plausible guess, so the prompt must ask for "?" and the app must show such rows.
• Paise invented on a cut-off page. The scan cut the balance column after the rupees. Sonnet wrote ".00", Gemini worked out the paise by arithmetic, and only GPT wrote "?". A check that PP-OCR also saw the paise digits would catch this.
• Rows shifted by one. On a page covered with handwritten notes, Gemini moved the amount column down one row. Sonnet did the same with the date column on another page. Shifted amounts break the balance on every row, which is easy to catch. Shifted dates do not touch the balance. A row with a date but no amount is the clue.
• A whole page missing. Gemini on "high" thinking returned nothing for a 46-row page. The row count and the balance across the page break show this.

What our checks catch

For each page I ran four checks that need no right answer:

1. The running balance reconciles on every row (in page order or newest-first).
2. Every amount the model read also appears in the PP-OCR reading of the same page.
3. The dates never go backwards.
4. Every row has a balance and exactly one of debit or credit, and no "?".

On whole statements the balance check is even stronger. Gemini 3.8 Flash read all 124 pages of the 31 real scans (2,155 rows). 27 of 31 statements reconciled from the first row to the last, across page breaks. The other 4 showed 45 breaks, all on pages with heavy handwriting or ticks. Those statements would open as "to check", with the exact rows marked, as the app already does for needs_review.

The PP-OCR check is strict: on real scans it questions about 5% of correct amounts, because PP-OCR itself misreads some cells. So it should mark single cells to check, not reject the page.

Where to run it

At today's volume an API is far cheaper than any GPU. 10,000 scanned pages a month cost about $40 with Gemini Flash plus about $6 for a Haiku second reading. A GPU that is always on costs $400–700 a month.

Proposed design

The design keeps the rule that already protects the app: a statement is validated only when its numbers prove themselves, and otherwise it is needs_review with the rows marked.

Steps to build it

1. Ask the user (the accountant) for consent to send page images to the chosen provider. Today they consent only to sending extracted text to OpenAI.
2. Add a scan path on the server: render pages, call Gemini 3.8 Flash per page with a fixed JSON schema and amounts as strings, at temperature 0.
3. Run PP-OCR (RapidOCR, ONNX) on the server for the cross-check and the word boxes.
4. Reuse our validation: balance chain, totals, needs_review reasons, plus new reasons such as ocr_disagrees and unreadable_value.
5. Retry failed rows once with Claude Opus 5.5 (or Haiku as a cheap second reader), then show the rest to the user.
6. Make long statements asynchronous (a job with progress), because 20+ pages can pass Cloudflare's 100 s limit.
7. Track every scan outcome in tracking_events, so we can measure real accuracy.

External services

From web research on 17/10/2026. I tested only the general VLMs above; the rest are from the vendors' pages.

Open-source models

The newest public ranking (OmniDocBench v1.6, September 2026) is led by small open models. None of the benchmarks uses Indian bank statements, so I ran the ones that fit an 8 GB GPU on our sets.

Privacy

• Production statements never left this PC. Only local engines read them.
• A scan carries more than its text: names, full account numbers, signatures. Sending page images to Google or Anthropic needs its own consent from the accountant.
• To keep processing in India: Claude on Amazon Bedrock's India profile, Gemini on Vertex AI in Mumbai (check that 3.8 Flash is listed), or open models on our own GPU.
• The DPDP Rules on security and transfers abroad take effect around May 2027. Transfers abroad are allowed unless the government blocks a country. This is not legal advice.

Limits of this study

• Only 31 real scanned statements, mostly from Gujarat banks. The answers for real scans came partly from model agreement.
• Cloud models were called through their command-line tools, so the times include tool start-up. Costs are estimates.
• I did not test Mistral OCR, Textract, Azure or Document AI (no keys), or Chandra 2 (license, and too big for 8 GB).
• Local GPU tests ran on an RTX 5060 with 8 GB. A rented 24 GB GPU runs larger models and runs faster.