Tesseract OCR: VobSub binarization thickens glyphs (max channel ≥ 90 counts the anti-aliasing ring as ink)

Aperta
#16,663 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
csharp

Direzione di ricerca

Read OcrViewModel.RunTesseractOcr, OcrSubtitleItem.GetSkBitmap, TesseractOcr.PrepareImage, NikseBitmap.MakeBlackAndWhiteForOcr, and VobSubColorIsolation.BinarizeForOcr to trace the current thresholds and pixel handling. Compare the default path with the existing “To one color” option, then confirm the reported cases with the attached subtitle images and Tesseract. Done means selecting a default binarization that preserves colored text while avoiding the reported thick glyphs, with results verified on real Subtitle Edit runs.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Hi,

While fine-tuning Tesseract models for subtitle OCR, I replicated Subtitle Edit 5.3.0's default image preparation for Tesseract and measured it on DVD subtitles. The text sent to Tesseract comes out much bolder
than the original glyphs, sometimes with letters merged together.

Code path (v5.3.0)

OcrViewModel.RunTesseractOcr → OcrSubtitleItem.GetSkBitmap() (pre-processing options all off by default
in SeOcr) → TesseractOcr.PrepareImage: AddMargin(10), then NikseBitmap.MakeBlackAndWhiteForOcr(90, 100)
→ tesseract --psm 6. For VobSub, the bitmap comes from SubPicture.GetBitmap (IDX palette, alpha = contrast × 17,
cropped), and FastBitmap writes raw BGRA bytes, so the test reads non-premultiplied values.

What happens

A pixel becomes text when alpha >= 100 and max(B, G, R) >= 90. I understand the max of the channels is
deliberate, so that yellow or red text is kept, and the comment on VobSubColorIsolation.BinarizeForOcr says
the dark outline and "the darker half of the antialias ramp" should become white. With a fixed threshold of 90,
that second part does not happen on common DVD palettes:

  • White fill (236,236,236), grey anti-aliasing (100,100,100), dark outline (28,28,28): the grey sits in the
    dark half (the midpoint is about 132) but passes the test (max = 100). Every stroke ends up about 1.8× wider
    than the real glyph. On small fonts, i dots merge into the stem and read as l (ikke → lkke).
  • Coloured palettes are worse. On one Finnish disc the opaque colours are light grey (228,228,228), magenta
    (228,0,215) and blue (0,0,226): each passes on a single strong channel, so fill, anti-aliasing and outline
    become one solid shape (Ei kai. is unreadable).

Existing option

The "To one color" OCR pre-processing (PreProcessingSettings.ConvertToOneColor, default threshold 128) keys on
luminance (0.299 R + 0.587 G + 0.114 B). Enabled, it fixes these cases. It is off by default
(PreProcessingToOneColor = false), so out of the box Tesseract gets the max-channel image. Like any fixed
luminance threshold, it would drop a pure red fill (luminance 76).

Suggestion

Make a luminance test the default for Tesseract, with a per-image threshold so coloured text is still kept:

lum  = 0.299 R + 0.587 G + 0.114 B
mid  = (min lum + max lum) / 2   over pixels with alpha >= 100
text = alpha >= 100 && lum > mid

The grey anti-aliasing and the blue outline become background; a red fill on a black outline stays text
(76 > 38).

Measurements (stock Tesseract 5.3.4 models, 1,777 DVD subtitle images in 12 languages, references reviewed
against the image, character error rate after quote/ellipsis normalization; my Python replica of the v5.3.0 rules
above: crop, raw values, 10 px margin, psm 6):

Image given to Tesseract CER Exact lines
5.3.0 default (max channel >= 90) 2.51 % 73.2 %
"To one color" option, threshold 128 0.26 % 92.3 %
Adaptive luminance (suggested default) 0.26 % 92.3 %

Default vs suggested, by language: fi 11.1 % → 0.05 %, de 6.0 % → 0.20 %, is 2.5 % → 0.37 %,
no 2.1 % → 0.19 %, sv 2.0 % → 0.11 %, da 1.5 % → 0.12 %, pt 1.5 % → 0.13 %, it 1.4 % → 0.14 %,
fr 0.44 % → 0.28 %, nl 0.36 % → 0.15 %; en and es unchanged (their palettes are not affected).

Attached: the same 12 subtitles (one per language) as decoded, with the 5.3.0 default, and with the suggested
binarization.

Caveats

  • These numbers come from a Python replica of the code above, not from running Subtitle Edit. I compared the
    replica with Subtitle Edit (its bundled Tesseract 5.5.3) on 50 French images only (48/50 identical readings),
    so please confirm on your side.
  • I did not test the resized retry passes (3×/2× and 4×/2×).
  • The same max-channel rule is in VobSubColorIsolation.BinarizeForOcr (seconv, PGS/DVB); I only measured DVD.

Thanks for Subtitle Edit!

Image Image Image
Lingua principale
C#
Stelle
15k
Fork
1.4k
Merge medio
1h 43m
PR unite (30g)
555

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di SubtitleEdit/subtitleedit

Tutte le issue di SubtitleEdit/subtitleedit

Issue simili

Altre issue su C#

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.