PARAS MADAN
All writing

Latest in AI

Open OCR Cost per Page: What LightOnOCR's $95 Benchmark Leaves Out

GPU time is getting cheap enough to compare OCR models on your own pages before choosing one for a full archive.

Daniel van Strien's November 18 OCR benchmark puts LightOnOCR-3-0.8B at about $95 in GPU time per million mixed pages, or $65 to $130 for different page types. For an actual archive, the useful question is how many pages you can accept without sending them back through the pipeline.

What the OCR benchmark measured

The benchmark used 850 pages, eight models and one RTX PRO 6000 ($2.75/hour). At a shared 2,048-pixel input size, RolmOCR cost $0.51 per 1,000 pages versus $0.095 for LightOnOCR-3-0.8B. At lower resolution RolmOCR narrowed the gap to about threefold, with some quality loss. Those prices exclude startup, and several models ran outside their normal pipelines.

I would use those figures to pick candidates. If your current setup uses a separate layout model to separate columns, keep that step in your pilot and charge its time to your baseline. Otherwise you're comparing a full pipeline against one stage of the replacement.

Check how you're feeding the GPU

On dense print, GLM-OCR fell from $1.57 to $0.40 per 1,000 pages without switching models. The experiment changed the GPU, vLLM version, input size and how it queued pages together, so the result doesn't isolate a single cause.

If your GPU is spending time waiting for the next batch, measure that before changing the model. Keep the model and image size fixed for your first concurrency comparison. Log pages accepted per hour and wall-clock time, including startup, rather than multiplying peak inference speed by the length of the job.

Run a pilot on the pages that can break your workflow

For a scanned archive, take a sample of ordinary text, dense columns and exactly the pages someone would otherwise clean up by hand: forms, faint scans and pages with nothing on them. Keep the same page IDs in both runs. Start with the downloadable LightOnOCR-3-0.8B and compare it with the tool you already use. For plain text, use LightOn's default transcription mode; if your application needs coordinates or chart data, test its separate grounding mode and budget from that run.

A character accuracy score is useful for clean text. For an invoice or a catalogue table, count whether the amount or label ended up on the correct row. The benchmark found blank-page hallucinations and invented text where lines were obscured. Its human-checked transcriptions covered 282 book pages, and an automated judge used for comparisons sometimes reversed its choice when answer order changed. Check a small set yourself before trusting the aggregate score.

Budget for a clean output, not a model call

Use total run cost divided by pages that pass your checks. For example, if inference across a million pages costs $95 and just 1% need 20 seconds of manual review, that is about 56 hours of review. At an illustrative $20 an hour, review alone adds around $1,110. Those are planning assumptions, not benchmark results. Include storage, preprocessing, reruns and the time to fix bad outputs if they apply.

I'd start by routing easy pages through the cheaper model and keeping a separate queue for pages that fail checks. At this price, the best use of another GPU dollar may be a second reading on those pages, not a bigger model on the whole archive.

I’m Paras. I work with AI founders on technical writing.

I can help explain what your product does, where it works well and what developers need to know before they use it.

Book a 30-minute call