submitqueue: port internal queue contracts to proto (submitqueue/core/messagequeue)
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Anfängerfreundlichkeit
- 32/100
- Issue-Typ
- Refactoring
- Klarheit
- Größtenteils klar
- Aktivitätsstatus
- Ruhig
- Tech-Stack
- go
- Bereich
- distributed-systems
Rechercherichtung
Beginne mit doc/rfc/messagequeue-contract.md und vergleiche api/runway/messagequeue/ mit stovepipe/core/messagequeue/. Ordne die zwölf Themen gateway Land/Cancel, den orchestrator stage controllers, core/request.PublishLog sowie den DLQ-Controllern und dem Republish-Tooling zu. Als erledigt gilt, wenn submitqueue/core/messagequeue/ die Proto-Verträge, das generierte protopb, Glue, Bindings und Vertragstests enthält, alle Payload-Nutzer migriert sind und die aufgeführten Entity-Serializer entfernt wurden.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Problem
The repo's queue-contract convention (doc/rfc/messagequeue-contract.md) is proto3 payloads serialized as protojson, with a contract package owning both the messages and their topic_keys bindings — api/runway/messagequeue/ is the reference example and stovepipe/core/messagequeue/ follows it. The submitqueue domain has not been ported: there is no submitqueue/core/messagequeue/ package, and 11 of its 12 topics still carry hand-rolled JSON via ToBytes/FromBytes methods on entity types.
Current state per topic:
| Topic key | Payload | Serialization |
|---|---|---|
start |
RequestID |
entity JSON |
cancel |
CancelRequest |
entity JSON |
validate |
RequestID |
entity JSON |
batch |
RequestID |
entity JSON |
score |
BatchID |
entity JSON |
speculate |
BatchID |
entity JSON |
prioritize |
QueueID |
entity JSON |
build |
BatchID |
entity JSON |
buildsignal |
BuildID |
entity JSON |
conclude |
BatchID |
entity JSON |
log |
RequestLog |
entity JSON |
merge |
MergeRequest |
✅ protojson (api/runway/messagequeue — correctly borrowed; runway owns that queue's contract) |
Consequences of the split: no additive-evolution guarantees (protojson's unknown-field discard, UPPER_SNAKE enums, int64-as-string are only enforced on merge), no topic_keys binding or contract test for internal topics, and queue-payload serialization concerns living on domain entities (which are supposed to be pure data).
Work
- Create
submitqueue/core/messagequeue/mirroring the reference layout:proto/sources, committedprotopb/, and the generic protojson glue (Marshal/Unmarshal[T]/TopicKeys) — same asstovepipe/core/messagequeue/. - Define proto messages for each payload shape (request-ID carrier, batch-ID carrier, build-ID carrier, queue-ID carrier, cancel request, request log) and bind each topic key to its message via the
topic_keysoption fromapi/base/messagequeue. - Add the contract test: protojson round-trip per message + every topic key bound to exactly one message.
- Migrate producers and consumers topic by topic (gateway
Land/Cancelpublishes, all orchestrator stage controllers,core/request.PublishLog, and the DLQ controllers/republish tooling — anything touchingmsg.Payload). - Once no queue payload uses them, remove
ToBytes/FromBytesfrom the submitqueue entity types (RequestID,BatchID,BuildID,QueueID,CancelRequest,RequestLog, and the full-entity variants) so entities go back to being pure data. - Bazel
visibilitystays domain-scoped per the internal-contract rule.
Caveats
- Wire-format flip: protojson output (snake_case fields, int64-as-string) differs from the current
encoding/jsonentity output, so in-flight messages from before a cutover won't parse after it. Port topic-by-topic with drained queues (fine at the current pre-prod stage), or give consumers a transitional dual-format read if needed. - The
logtopic crosses gateway ↔ orchestrator but both are submitqueue-domain services, so the internal contract location (submitqueue/core/messagequeue/) is correct — it must not go underapi/. - The message-ID convention is a separate concern tracked in #352 (intent-scoped message IDs); porting payloads to proto neither depends on nor resolves it, but call sites will be touched by both — coordinate to avoid churn.
Related
- #352 — intent-scoped message IDs (same call sites)
- #357 — stovepipe counterpart (retire leftover JSON serializers, contracts for upcoming stages)
- Vorherrschende Sprache
- Go
- Sterne
- 225
- Forks
- 11
- Ø Merge
- 3 T. 8 Std.
- Gemergte PRs (30 T.)
- 61
Beitragsleitfaden
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus uber/submitqueue
-
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 25/100
uber/submitqueue#209 ·
Alle Issues in uber/submitqueue
Ähnliche Issues
-
kind/bug needs-triage
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 72/100
matrixorigin/matrixone#29223 ·
-
needs-acceptance wg/data-plane-networking
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
vllm-project/semantic-router#4024 · 1 Kommentar ·
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
-
bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 86/100
alexgorbatchev/dotfiles#107 ·
-
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 84/100