Skip to content
AI

Khata2Sheet AI: Offline Bengali OCR & Grid Alignment

Engineering notes on processing handwritten Bengali shopkeeper ledgers, bounding-box grid detection with OpenCV, and offline digit parsing.

Remix into Workplace Studio:

What Was I Trying to Do?

Traditional retail shops across Bangladesh record daily sales, customer credit (বাকি), and supplier cash deposits in handwritten Bengali red ledger books (লাল খাতা). Most commercial cloud OCR APIs fail completely on cursive Bengali script and faint ballpoint ink on recycled paper.

My goal was to build a 100% local, offline-capable pipeline that accepts an ordinary smartphone photo of a khata page, corrects perspective distortions, detects table grids, and parses numerals into clean spreadsheet rows.

Architecture & Pipeline

  1. Adaptive Preprocessing (OpenCV):
    • Converted RGB input to grayscale and applied bilateral filtering to preserve ink edges while suppressing paper texture noise.
    • Evaluated Otsu thresholding versus adaptive Gaussian thresholding (cv2.adaptiveThreshold with block size 15).
  2. Morphological Grid Line Isolation:
    • Isolated horizontal and vertical ledger lines using rectangular morphological kernels (kernel_h = cv2.getStructuringElement(cv2.MORPH_RECT, (40, 1))).
    • Combined horizontal and vertical masks with bitwise AND to identify exact cell intersection coordinates.
  3. Cell Extraction & Local Inference:
    • Cropped isolated cells with 4px padding.
    • Passed numerals into an offline ONNX model fine-tuned on Bengali handwritten digits (০, ১, ২, ৩, ৪, ৫, ৬, ৭, ৮, ৯).

Key Highlights & Performance

  • Inference Speed: Average processing time was 380ms per full ledger page on standard laptop hardware without requiring a discrete GPU.
  • Perspective Tolerant: Automated 4-corner contour detection handles photos taken at angles up to 35 degrees.
  • Digit Accuracy: Achieved 94.2% recognition accuracy on clean ledger pages and 88.7% on heavily smudged ink.

What I Learned

The hardest part was not digit recognition, but handling broken handwritten lines where shopkeepers drew column dividers freehand without a ruler. Dilating broken line segments before Hough transform was essential to prevent row collapse. True client-side privacy is mandatory for shopkeeper financial data.

OCRBengali OCROffline AIKhataOpenCV
Md Sabbirul Islam Khan

Md Sabbirul Islam Khan

Written by THE SABBiR