Enhance pending documents API
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
This is a design-level API change: infer current pending-document handling from the existing documents list/filter by pending status, then introduce a processing-jobs model (start/end, progress, origin, unread/dismissed) plus date/origin filters and a streaming list endpoint. Start by locating the current pending-documents API and any related polling clients; done means jobs persist after processing, clients can filter and stream without polling, and bulk reprocess no longer floods the web indicator. No files or tests are named in the issue.
由索引模型根据 Issue 内容生成。
描述
Problems
- We have to poll for changes to processing state. If the API provided a streaming response, we could avoid hitting the endpoint multiple times.
- Once the document is done processing, it’s removed from the results. Because we’re polling for data, once the document is done, it disappears from our processing list.
- When dispatching a bulk reprocess from the API, all those documents will appear in the web indicator
- https://github.com/MuckRock/documentcloud/issues/230
- https://github.com/MuckRock/documentcloud/issues/211
Proposed Solutions
- Document processing jobs are inferred by filtering documents for a “pending” status. We should consider creating a new table for processing jobs, that track details like:
- start time and end time
- a value to track job progress
- relation to the document, or documents, involved in the job
- track the client origin of a processing job: did it come from the web client, an API client, someplace else
- optionally, some kind of human-readable summary or description of the job: “Applying OCR to 15 documents”
- a boolean “unread” or “dismissed” status, so that clients know which completed jobs to display to users
- Provide date and origin filters for processing jobs, so that the client can make a request like “send me all pending or non-dismissed completed jobs created from the frontend client in the last week”
- Provide a streaming endpoint so that the client only has to make one request for the list of processing documents, instead of continuously polling the endpoint.
- 主要语言
- Python
- 星标
- 55
- 派生
- 10
- 平均合并
- 1 天 13 小时
- 30 天内合并 PR
- 22
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
MuckRock/documentcloud 的其他 Issue
-
难度 1/5 1 小时以内 新手友好度 62/100
MuckRock/documentcloud#537 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 3/5 1-2 天 新手友好度 50/100
MuckRock/documentcloud#539 ·
维护者通常 1 天内回复
-
bug
难度 3/5 1-2 天 新手友好度 67/100
MuckRock/documentcloud#534 ·
维护者通常 1 天内回复
-
难度 3/5 1-2 天 新手友好度 66/100
MuckRock/documentcloud#531 ·
维护者通常 1 天内回复
-
难度 4/5 3-5 天 新手友好度 55/100
MuckRock/documentcloud#521 ·
维护者通常 1 天内回复
查看 MuckRock/documentcloud 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 83/100
维护者通常 1 天内回复
-
comp/desktop P3 type/feature
难度 2/5 1-3 小时 新手友好度 78/100
NousResearch/hermes-agent#154285 ·
维护者通常 1 天内回复
-
docs: qualify copied and custom kubectl debug policy notes可能已有人在做 @nym01 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
kubescape/cel-admission-library#166 · 1 条评论 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 68/100
SACGF/variantgrid#2804 · 1 条评论 ·
维护者通常 1 天内回复
-
[BUG] HTTP lifecycle dispatch treats a NO_PROXY-only environment as corporate egress and skips the resolve-and-pinned session可能已有人在做 @bodapatisaikrishna 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 1 天内回复