"I" (vowel) symbol recognition

Đang mở
  • Có gì không ổn?
#1,989 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
18/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Đình trệ
Công nghệ
fastapi, python, pytorch
Lĩnh vực
ai, machine-learning

Hướng nghiên cứu

The report concerns recognition on vertical layouts with preserve_aspect_ratio set to True, so start with the pre-processing path that resizes and pads crops before recognition, and with how the predictor exposes preserve_aspect_ratio and symmetric_pad to the caller. Reproduce with the attached FastAPI script on the two JSON outputs and count the "I" characters. Done would mean a documented or automatic choice that recovers the missing "I" characters without degrading horizontal text, and the issue has been open over a year with one comment and no maintainer decision, so it needs a design call first.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

type: enhancement
🚀 The feature

"I" (vowel) symbol is systematically missing from recognition model output when using a vertical image layout with preserve _aspect_ratio option set True.

Motivation, pitch

I have discovered that the model consistently struggles with recognizing the vowel "I" in various positions within a sentence—particularly at the start and end, when the preserve_aspect_ratio parameter in the Pre-Processor is set to True.

Setting preserve_aspect_ratio=False helps mitigate the issue, but only for vertically oriented text.

However, when using horizontally oriented text setting preserve_aspect_ratio=True results in better recognition of "I" symbols occurrences.

I have also tried to change interpolation method in image resize preprocessing stage. While it improves the overall quality of recognition, it does not affect the "I"s.

Alternatives

I have three suggestions of how it is possible to fix that issue:

  1. To use the conditional check to define the value of preserve_aspect_ratio parameter based on the ratio of image sides. Where the detection of horizontal text will lead to using True value and vertical - to False value.

  2. Allow to choose the preserve_aspect_ratio value when calling the model.

  3. To add more horizontal text samples into the datasets and retrain/finetune the detection and recognition models.

Additional context

I have tested the issue with the image composed from different sentences containing I's on the latest version of doctr library (torch).

The picture represents a comparison between two runs of ocr on the same image using different values of preserve_aspect_ratio parameter.

Colors of boxes meaning:

  • the blue color is assigned to preserve_aspect_ratio set to False outlier results;
  • the red color is assigned to preserve_aspect_ratio set to True outlier results;
  • gray color signifies there is no change between runs;
  • other colors represent partial difference.

Two json files attached show the doctr ouput on the same image. In the doctr_ocr_par_False where are 10 "I" occurrences, while in doctr_ocr_par_True where are 5 "I" occurrences.

Image Image

doctr_ocr_par_False.json

doctr_ocr_par_True.json

The basic script used for tests:

from fastapi import FastAPI, UploadFile, File
import numpy as np
from doctr.models import ocr_predictor
from PIL import Image
import io
import torch
import uvicorn

app = FastAPI()

DETECTION_MODEL = "db_resnet50"
RECOGNITION_MODEL = "crnn_mobilenet_v3_large"
PRESERVE_ASPECT_RATIO = False

DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")

model = ocr_predictor(
    pretrained=True,
    det_arch=DETECTION_MODEL,
    reco_arch=RECOGNITION_MODEL,
    assume_straight_pages=True,
    preserve_aspect_ratio=PRESERVE_ASPECT_RATIO,
    symmetric_pad=True,
).to(device=DEVICE)

@app.post("/ocr")
async def ocr(file: UploadFile = File(...)):
    image = await file.read()
    await file.close()

    doc = []
    image_pil = Image.open(io.BytesIO(image)).convert("RGB")
    doc.append(np.asarray(image_pil))

    result = model(doc)
    return result.export()


if __name__ == "__main__":
    uvicorn.run("main:app", host="0.0.0.0", port=44556, reload=False
Ngôn ngữ chính
Python
Star
6.4k
Fork
684
Merge trung bình
1 ngày 5 giờ
Pull request đã merge (30 ngày)
28

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của mindee/doctr

Tất cả issue của mindee/doctr

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.