Scanned bank statements: how LedgerMap can read them

What I tested on

There were no scanned uploads in production yet (12 statements in total, all text PDFs). So I built three test sets. Two of them have exact answers.

A row counts as right only if its date, debit, credit and balance all match exactly. Narration is scored separately. Because the real-scan answers came partly from model agreement, they favour models that agree with Gemini and Claude Sonnet. The two other sets have no such bias.

Results

* Times through a command-line tool include its start-up and file-reading step (10–20 s); the direct API rows do not. Costs are API list prices (October 2026) times tokens per page. The direct API rows use the tokens the API reported: about 5,200 input tokens per page image and 1,300 output tokens. The CLI rows for Gemini, GPT and Opus count only the visible rows and assume fewer input tokens, so treat them as minimums. Through the CLI, Claude thinking cannot be turned off (I tried both settings); the API accepts thinking: {type: "disabled"} for Haiku 5.5 and {type: "between_tools"} for Sonnet 5.5, at effort high or below. "Production" was run only with engines on this PC.

Speed against a 10–12 s budget

The budget is for a whole statement, from upload to rows. Most scanned statements in the samples have 1–4 pages; the longest has 23.

Measured with the direct API: Haiku 5.5 with thinking off writes about 250–300 tokens a second, about twice the rate seen through the CLI. A typical page takes 4–7 s and a dense 46-row page about 11–12 s. The time of a statement is the time of its slowest page, not the number of pages: the 23-page statement took 5.6 s. Gemini was not measured through its API (no Gemini key).

Two more ways to make the wait feel shorter: show rows as they arrive (pages finish one by one), and start reading as soon as the upload ends, before the bank is confirmed.

The mistakes the models made

All errors on the real scans fall into five kinds. These are the cases our checks must handle.

• A digit misread. GPT-6 Astra read a 6 as a 0 in an amount. The balance check flags it, and PP-OCR read the digit correctly.
• A guess where the page hides the value. A punched hole hid the first digit of a date. Gemini wrote a full date; Sonnet and Opus wrote "?". Where the dates on both sides fix the day (a date between 08/05 and 13/05 that ends in 3 must be 13/05), the guess was right. In one case it was not knowable. No automatic check can catch a plausible guess, so the prompt must ask for "?" and the app must show such rows.
• Paise invented on a cut-off page. The scan cut the balance column after the rupees. Sonnet wrote ".00", Gemini worked out the paise by arithmetic, and only GPT wrote "?". A check that PP-OCR also saw the paise digits would catch this.
• Rows shifted by one. On a page covered with handwritten notes, Gemini moved the amount column down one row. Sonnet did the same with the date column on another page. Shifted amounts break the balance on every row, which is easy to catch. Shifted dates do not touch the balance. A row with a date but no amount is the clue.
• A date digit guessed with thinking off. Under the same punched hole ("?5/06/2019", between 04/06 and 28/06), Haiku with thinking off wrote 25/06; with thinking on it wrote "?". Sonnet with thinking off wrote 11/05 for a hidden 13/05. Only dates were guessed this way, never amounts.
• An opening balance read as a transaction. Haiku with thinking off kept a "B/F" row as a credit of 7,231.97. It reconciles, so the balance check cannot see it.
• A whole page missing. Gemini on "high" thinking returned nothing for a 46-row page. The row count and the balance across the page break show this.

What our checks catch

For each page I ran four checks that need no right answer:

1. The running balance reconciles on every row (in page order or newest-first).
2. Every amount the model read also appears in the PP-OCR reading of the same page.
3. The dates never go backwards.
4. Every row has a balance and exactly one of debit or credit, and no "?".

Two more checks, added after the API tests: (5) every date must also appear in the PP-OCR reading of the page, and (6) no row may be named B/F, brought forward, opening, carried forward or closing balance. With them, no wrong row passed for any Claude run, including the guessed dates and the B/F row above. The cost is more pages to look at: on the 31 banks, 21 instead of 29 pages pass by themselves, because PP-OCR misreads some dates. The date check should mark the one row, not the page.

On whole statements the balance check is even stronger. Gemini 3.8 Flash read all 124 pages of the 31 real scans (2,155 rows). 27 of 31 statements reconciled from the first row to the last, across page breaks. The other 4 showed 41 breaks, all on pages with heavy handwriting or ticks. Those statements would open as "to check", with the exact rows marked, as the app already does for needs_review. Through the direct API with thinking off: Haiku reconciled 22 of 31 statements (64 breaks), Sonnet 28 of 31 (9 breaks), and Haiku with Sonnet's page where Haiku's fails 28 of 31 (7 breaks).

What the last 7 breaks are (checked by eye on the page images):

• 4 breaks, one statement: rows missing from the scan. The scanner cut off the last row or two of pages 1–4. The model read every visible row right; the missing amounts (388, 620, 50 and 396) are not in the PDF at all. No reader can recover them. The user must add them or get a better scan.
• 1 break, one statement: a second printout stapled on. Page 11 is a separate printout from 21/03/2020 that repeats the last two rows of page 10. The model also misread one of them (49,995.00 circled in pen, read as 49,866.00). The app must spot repeated rows after a new "Brought forward".
• 2 breaks, one statement: a real reading error. An HDFC page prints each balance half a line above its row, and a ₹1.00 charge and its ₹1.00 refund are both in the Withdrawals column ("1.00" and "-1.00"). The model swapped the two ₹1 rows.

So of 2,147 rows, the only wrong values were two ₹1 rows and one amount on a repeated row. All three were flagged.

The PP-OCR check is strict: on real scans it questions about 5% of correct amounts, because PP-OCR itself misreads some cells. So it should mark single cells to check, not reject the page.

Confidence per row and the Sonnet fallback

Measured on 18/10/2026 through the direct API, on the 62 pages with exact answers (1,135 rows).

The score threshold (80) was chosen on the same pages it was tested on, and there are only 22 wrong rows. Treat it as a starting point. Measure it again on real scanned uploads and track it in tracking_events.

Where to run it

At today's volume an API is far cheaper than any GPU. 10,000 scanned pages a month cost about $12 with Haiku alone, or about $250 with Haiku and Sonnet together (measured tokens). A GPU that is always on costs $400–700 a month.

Proposed design

The design keeps the rule that already protects the app: a statement is validated only when its numbers prove themselves, and otherwise it is needs_review with the rows marked.

Steps to build it

1. Ask the user (the accountant) for consent to send page images to the chosen provider. Today they consent only to sending extracted text to OpenAI.
2. Add a scan path on the server: render pages, call Claude Haiku 5.5 and Sonnet 5.5 per page with thinking off, a fixed JSON schema and amounts as strings.
3. Run PP-OCR (RapidOCR, ONNX) on the server for the cross-check and the word boxes.
4. Reuse our validation: balance chain, totals, needs_review reasons, plus new reasons such as ocr_disagrees and unreadable_value.
5. Use Sonnet's page where Haiku's page fails the checks, then show the rest to the user with the rows marked.
6. Make long statements asynchronous (a job with progress), because 20+ pages can pass Cloudflare's 100 s limit.
7. Track every scan outcome in tracking_events, so we can measure real accuracy.

External services

From web research on 17/10/2026. I tested only the general VLMs above; the rest are from the vendors' pages.

Open-source models

The newest public ranking (OmniDocBench v1.6, September 2026) is led by small open models. None of the benchmarks uses Indian bank statements. I ran PaddleOCR-VL-1.6 and GLM-OCR on our real scans (74% and 77% of rows exact). The others are downloaded and ready for a GPU test.

Privacy

• Production statements never left this PC. Only local engines read them.
• A scan carries more than its text: names, full account numbers, signatures. Sending page images to Google or Anthropic needs its own consent from the accountant.
• To keep processing in India: Claude on Amazon Bedrock's India profile, Gemini on Vertex AI in Mumbai (check that 3.8 Flash is listed), or open models on our own GPU.
• The DPDP Rules on security and transfers abroad take effect around May 2027. Transfers abroad are allowed unless the government blocks a country. This is not legal advice.

Limits of this study

• Only 31 real scanned statements, mostly from Gujarat banks. The answers for real scans came partly from model agreement.
• Cloud models were called through their command-line tools, so the times include tool start-up. Costs are estimates.
• I did not test Mistral OCR, Textract, Azure or Document AI (no keys), or Chandra 2 (license, and too big for 8 GB).
• The local GPU tests did not run: the RTX 5060 was in use the whole time. The open models ran on CPU with the same weights. Their accuracy carries over; their GPU speed does not.
• My table-to-rows conversion for the open document models is simple. Part of their lower score may come from it, not from the reading.