ggml-org/llama.cpp

Request: Nougat OCR Integration

開放

#3,294 建立於 2023年9月21日

 (8 則留言) (13 個反應) (0 位負責人)C++ (21,737 個分叉)batch import
help wantedmodel

倉庫指標

星標
 (124,043 顆星)
PR 合併指標
 (平均合併 6天 8小時) (30 天內合併 389 個 PR)

描述

Request: Nougat OCR Integration

I suggest adding Nougat OCR into llama.cpp to enable the processing of scientific PDF documents. This can act as a first step towards adding multimodal models to this project!

Implementation: It seems that Nougat is based on standard transformer architecture (like Bart and Swin Transformer) and most of the work would be on figuring out how to add the image processing.

Let me know what you think! P.S.: Love this repo! I hope to add my own retrieval-pretrained transformer at some point to this repo.

貢獻者指南