OCR for genealogy: limits, risks and how to use it better

OCR (Optical Character Recognition) converts images of text into machine-readable characters. It works by analyzing pixel patterns and matching them against known letterforms. Modern OCR engines like Tesseract use neural networks trained primarily on printed text — books, newspapers, typed documents.

The key assumption of conventional OCR is that characters are consistent: the same letter always looks (roughly) the same. This assumption holds well for printed text but breaks down completely with handwriting, where the same person may write the same letter differently within a single sentence.

Genealogical documents present a combination of challenges that conventional OCR was never designed to handle:

• Handwriting variation — every priest, registrar and port agent had a unique hand. Character shapes are inconsistent even within a single record.
• Connected cursive — letters flow into each other without clear boundaries, making it impossible for character-level OCR to segment individual letters.
• Kurrent script — German cursive used from the 16th to mid-20th century. Its letterforms are radically different from modern Latin script. Even trained human readers struggle with it.
• Scan quality — faded ink, water damage, bleed-through from the opposite page, tight bindings that distort the text near the spine — all common in archival scans.
• Rare accents and diacritics — names with umlauts, tildes, cedillas and other marks are frequently garbled or dropped by OCR engines not trained on that character set.

Conventional OCR and AI-powered extraction are fundamentally different approaches to reading documents:

• Tesseract (conventional OCR) — character-level recognition optimized for printed text. Fast, free and effective for typed documents. Poor results on manuscripts.
• AI vision models (e.g. Gemini) — analyze the entire page as an image, understanding context, layout and even partial words. Significantly better at reading handwriting, but slower and more expensive.
• Structured extraction — beyond raw text, AI extraction can identify fields (name, date, event type) and return structured data rather than a plain text dump.

KinProof uses both approaches with automatic routing: printed documents are processed with Tesseract for speed, while handwritten documents (especially Kurrent) are routed to Gemini Vision. If Tesseract output quality is below threshold (based on character count and alphabetic ratio), the system automatically falls back to the AI vision model.

Despite its limitations on manuscripts, conventional OCR remains the best choice for certain types of genealogical documents:

• Printed civil certificates — 20th-century birth, marriage and death certificates with typed or pre-printed fields.
• Typed immigration lists — many port authorities used typewriters for passenger manifests, especially from the early 1900s onward.
• Naturalization forms — government-issued forms with typed fields and standardized layout.
• High-resolution scans of printed text — scans at 200+ DPI of printed books, indexes and finding aids.
• Historical newspapers — obituaries, marriage announcements and community notices in printed newspapers.

The greatest danger of OCR in genealogical research is not that it fails — it is that it fails silently, producing plausible but wrong output:

• Garbled names — "ExampleSurname" may be read as "ExampleSurnameVariant1", "ExampleSurnameVariant2m" or "Bespermann". A researcher searching for the correct spelling will miss these records entirely.
• Wrong dates — a "3" misread as "8" changes a birth year from 1833 to 1838 — enough to confuse two generations.
• Swapped fields — in densely written parish records, OCR may attribute the father's name to the child or the godparent's name to the mother.
• Misplaced trust — researchers who rely solely on OCR output without cross-checking against the original image risk building family trees on incorrect data.