Undergraduate ResearchIS4990 Final Year Project · Batch 21University of Moratuwa

Neuro-symbolic OCR & NLP for Classical Sinhala Ayurvedic Palm-Leaf Manuscripts

A three-module pipeline that turns photographs of centuries-old, faded palm-leaf manuscripts into text that is both readable and safe to act on — correcting OCR errors, detecting sentence boundaries, and screening for toxic ingredient guidance before a human ever sees it.

My module: Module 3 — Correction, SBD & Safety
Team: Team Vision (3)
Supervisors: Dr. Ganegoda G.U | Dr. R.L.C. Shyama

Abstract

Overview

Archaic Sinhala Ayurvedic palm-leaf manuscripts encode centuries of traditional medical knowledge, including formulations that can be toxic if a purification step is missed or misread. Team Vision built a three-module, neuro-symbolic pipeline to digitize them safely: image preprocessing and character recognition (Module 1), word segmentation (Module 2), and, as my contribution, post-OCR error correction, sentence boundary detection, and a knowledge-graph safety guardrail with human-in-the-loop (HITL) escalation (Module 3). The modules are connected only by a JSON interface contract, so each can be developed, evaluated, and swapped independently. My module is built around JointCS, a joint correction-and-segmentation model sharing a domain-adapted XLM-R encoder, wrapped in an orchestrator that fails closed: whenever the system is unsure, a human expert reviews before the reading is trusted.

The source material

What we're actually digitizing

This is one board from a throat-medicine ola manuscript in the project's corpus — a real specimen, not a rendered mockup. Cursive Sinhala, written stylus-first into dried palm leaf, then rubbed with soot to make the incisions legible. On roughly half the ~95 leaf-sides surveyed, the ink has worn down to bare stylus scratches.

That's not a footnote — it's the reason Module 3 exists. Faded ink means OCR mistakes; a missed or misread purification step (shodhana) in a recipe for a toxic ingredient is a patient-safety issue, not just a transcription error. Correction, sentence-boundary detection, and the safety guardrail all exist to answer one question before a reading reaches an expert: how much of this can we trust, and where do we stop and ask?

Photograph of a stacked palm-leaf (ola) manuscript board from the project corpus, showing six leaf-sides of cursive Sinhala script threaded on string, with visible wear and ink fading. — Throat-medicine manuscript, cover-marked 1204 / 690-D.25 / “B” — six leaf-sides on this board, part of a ~95-leaf-side bundle. This is the raw input Module 1 photographs and Module 3's output is ultimately judged against.

Attribution

The team, and my role

This is a three-person group project. Everything below is scoped to Module 3, which I designed, built, and evaluated. Modules 1 and 2 belong to my co-researchers and are described here only for context.

Module 1

Read the leaf

Image preprocessing + CRNN/CTC character recognition baseline.

Ansar M.M.M

Module 2

Split the words

Hybrid word segmentation, BiLSTM/BIES top-3 candidates.

Hamas A.F.A

Module 3My work

Check the meaning

OCR correction, sentence boundary detection, KG safety guardrail, HITL orchestration.

Jayashanka W.W.M (me)

System design

Architecture

Three independent modules connected only by a JSON contract, run end-to-end with a single command, and designed to fail closed: when the system is not confident, a human expert makes the call instead of the model.

Method

JointCS & the safety guardrail

JointCS is a joint OCR error-correction and sentence-boundary-detection model: a shared, domain-adapted XLM-R encoder feeds three heads — a detection BiGRU, soft-masking in the style of Soft-Masked BERT, and a correction MLM head paired with an SBD head. Domain adaptation pre-training (DAPT) on Sinhala Ayurvedic text is what separates it from an off-the-shelf multilingual model.

To evaluate under realistic conditions, I built a seeded Sinhala glyph-confusion noise channel (substitutions, deletions, insertions, and split/merge errors) with four calibrated severity presets — measured character error rates of 4.8%, 9.6%, 14.7%, and 19.4% on gold text — so the model is never evaluated only on clean, synthetic input.

The riskiest failure mode isn't a wrong word — it's an unpurified toxic ingredient slipping through undetected. A 2,100-ingredient knowledge graph flags high/medium-high/medium toxicity entries, and a nearest-toxin binding rule ties each purification (shodhana) keyword to the specific toxin it modifies, rather than to any toxin in a nearby window — closing the exact failure mode that caused the two guardrail misses in the interim submission.

Bar chart showing post-correction CER at 10% input noise: 22.4% with argmax decoding, 12.5% with constrained decoding at margin 0, and 9.2% with the final constrained decoding at margin 3.0 — below the 9.6% input CER. — Decoding strategy matters as much as the model: naive argmax decoding makes noisy text worse (22.4% CER vs. 9.6% input); constrained decoding with a calibrated margin is what makes correction net-positive.

Live demo

Inside the orchestrator

These are real captures from the Streamlit viva demo running the live pipeline — not mockups. Each shows the orchestrator reasoning over an actual manuscript sentence, in Sinhala, end to end.

Screenshot of the JointCS correction diff view showing two words struck through in red (the OCR originals) replaced by green corrections, several tokens flagged but left unchanged, and a sentence-boundary-detection chart with P(STOP) bars and a 0.5 threshold line. — Correction stays conservative on purpose. 2 words replaced, 13 flagged but deliberately left alone — precision ≈0.96, over-correction rate 0.16%. Below it, the SBD head's P(STOP) score per token, cut where it crosses the threshold. Aggressive decoding would have made the noisy input worse, so the model only acts when confident.

Screenshot showing a knowledge-graph entity-detection table matching a toxic ingredient with its required shodhana purification, followed by an APPROVE verdict card reading 'safe to proceed', reliability 0.975, 0 issues. — Approve — the purification step was present. The KG matched a High-toxicity ingredient and confirmed its required shodhana step was present in the sentence — reliability 0.975, evidence chain shown for audit.

Screenshot showing a REJECT verdict card reading 'unpurified toxin', reliability 0.979, 1 issue, with an evidence table flagging a High-toxicity ingredient with reason 'missing_shodhana'. — Reject — an unpurified toxin. Same mechanism, opposite outcome: a High-toxicity ingredient appears with no purification keyword nearby — the guardrail hard-rejects rather than guessing.

Screenshot of a HITL (refer to human review) verdict card, reliability 0.992, flagging a possible corrupted toxin match between two Sinhala tokens at edit distance 1, with a stats row showing 0 OCR errors repaired, 3 sentence boundaries added, 0 KG safety issues found. — HITL — catching an OCR-corrupted toxin name. This is the fuzzy near-miss guard in action: a toxin's surface form was corrupted by OCR noise just enough to miss an exact knowledge-graph match (edit distance 1) — instead of silently passing, the pipeline escalates to a human rather than trusting an uncertain match.

Evaluation

Headline results

All numbers below are from the hardened evaluation protocol (holdout benchmark, multi-seed, reproduced byte-identically across three independent runs) — not the earlier, in-corpus-inflated numbers.

0.798–0.804

SBD F1 (EXPERT, out-of-corpus)

At 5–15% OCR noise; matches the old clean-text specialist (0.782) within CI

+7.2%

CER reduction

At 20% noise, calibration split, no leakage (margin = 3.0)

0 / 28

Toxic false negatives, clean

Observed 0; 95% CI bounds FNR ≤ 12.1% at this sample size

< 0.01

Expected calibration error

Max 0.0094 at 20% noise — confidence scores are well-calibrated

Bar chart: JointCS SBD EXPERT F1 stays at 0.80 across 5-20% noise, matching a clean-text specialist baseline of 0.782, while removing domain-adapted pretraining (no DAPT) drops F1 to roughly 0.75. — Sentence-boundary detection holds up under noise, and domain-adapted pretraining (DAPT) is the clear, consistent driver of the gain over a non-adapted encoder.

Line chart of worst-seed false-negative rate and human-review (HITL) rate against the escalation threshold tau, with the recommended tau=0.50 marked. — Tuning the HITL escalation threshold (τ) trades safety risk against reviewer workload; τ = 0.50 was chosen as the point where worst-seed false negatives stop dropping.

Bar chart of CER reduction at 10% noise by difficulty bucket: +8.6% easy, +6.3% medium, +6.3% hard, -4.2% expert (out-of-corpus). — Reported honestly, not selectively: correction helps on in-corpus text but costs 4.2% on the EXPERT (out-of-corpus) bucket — the templated training corpus is a genuine limitation, not a hidden one.

Pre-registered

Hypothesis scoreboard

Four hypotheses were pre-registered in the methodology before the final evaluation ran. All four are reported below, including the ones that only partially held.

H1Partial → Strong

Noisy-input SBD (F1 0.798–0.804) is statistically equivalent to a clean-text specialist (0.782) — equivalence, not superiority. The cleaner, significant effect is the DAPT ablation: removing domain-adapted pretraining costs 2.8–5.7 F1 points, consistently, at every noise level.

H2Supported

Correction gives a net CER reduction at the calibrated decision margin (3.0), confirmed on a held-out calibration split with no leakage — with the EXPERT-bucket exception reported above.

H3Partial

Zero toxic approvals observed on clean text (bounded, not exact, at n=28). The system stays a fail-closed HITL triage tool rather than a certified filter: end-to-end verdict accuracy drops below the 0.90 target at 15%+ noise, where residual misses trace to OCR-corrupted toxin surface forms, not knowledge-graph coverage gaps.

H4Supported

Expected calibration error stays under 0.01 across all noise levels (max 0.0094) — the model's confidence scores can be trusted to drive the HITL escalation decision.

Honesty

Limitations

The 70K-sentence training corpus has only 205 unique word types across 782K tokens — it's templated, not naturally varied. In-corpus evaluation numbers are inflated; the out-of-corpus EXPERT bucket is treated as the only real signal throughout.
The OCR noise channel is synthetic. Severity presets are calibrated to match measured CERs on gold text, but a separate run against three manuscript-attested confusion pairs (real scribal error patterns) was used to sanity-check robustness — headline metrics held, correction gains were slightly smaller.
Main results come from a single training seed, disclosed wherever cited. A parallel run completed the formal joint-vs-SBD-only ablation with one seed; multi-seed confirmation is future work.
Inter-annotator agreement and domain-expert validation studies were descoped as infeasible within the project timeline — reported here as an explicit limitation, not silently dropped.
The safety guardrail is a bounded, fail-closed triage aid, not a certified toxicity filter — framed that way throughout rather than oversold.

Where it stands

Status & deliverables

Final evaluation

Physical exhibition mode following Semester 8 exams — exhibit, submitted materials, and a viva-style discussion of individual contribution.

Deliverables in progress

Group final report, a 15-minute presentation video, an exhibition poster, and slide deck alongside the Streamlit viva demo.

Publication

Results are being written up as an IEEE conference paper, with LREC 2026 as the leading target venue.

Happy to talk through the methodology, the guardrail design, or the evaluation protocol in more depth.

Get in touch