ggml-org/llama.cpp

Request: Nougat OCR Integration

开放

#3,294 创建于 2023年9月21日

 (8 条评论) (13 个反应) (0 位负责人)C++ (21,764 个派生)batch import
help wantedmodel

仓库指标

星标
 (124,134 个星标)
PR 合并指标
 (平均合并 6天 8小时) (30 天内合并 389 个 PR)

描述

Request: Nougat OCR Integration

I suggest adding Nougat OCR into llama.cpp to enable the processing of scientific PDF documents. This can act as a first step towards adding multimodal models to this project!

Implementation: It seems that Nougat is based on standard transformer architecture (like Bart and Swin Transformer) and most of the work would be on figuring out how to add the image processing.

Let me know what you think! P.S.: Love this repo! I hope to add my own retrieval-pretrained transformer at some point to this repo.

贡献者指南