submitqueue: port internal queue contracts to proto (submitqueue/core/messagequeue)
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 32/100
- Tipo di issue
- Refactoring
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Tranquilla
- Stack tecnologico
- go
- Ambito
- distributed-systems
Direzione di ricerca
Inizia da doc/rfc/messagequeue-contract.md e confronta api/runway/messagequeue/ con stovepipe/core/messagequeue/. Mappa i dodici argomenti tra gateway Land/Cancel, gli stage controllers dell’orchestrator, core/request.PublishLog e i controller DLQ e gli strumenti di republish. Il lavoro è completo quando submitqueue/core/messagequeue/ contiene i contratti proto, il protopb generato, glue, bindings e contract test, tutti gli utenti del payload sono stati migrati e i serializer delle entità elencati sono stati rimossi.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Problem
The repo's queue-contract convention (doc/rfc/messagequeue-contract.md) is proto3 payloads serialized as protojson, with a contract package owning both the messages and their topic_keys bindings — api/runway/messagequeue/ is the reference example and stovepipe/core/messagequeue/ follows it. The submitqueue domain has not been ported: there is no submitqueue/core/messagequeue/ package, and 11 of its 12 topics still carry hand-rolled JSON via ToBytes/FromBytes methods on entity types.
Current state per topic:
| Topic key | Payload | Serialization |
|---|---|---|
start |
RequestID |
entity JSON |
cancel |
CancelRequest |
entity JSON |
validate |
RequestID |
entity JSON |
batch |
RequestID |
entity JSON |
score |
BatchID |
entity JSON |
speculate |
BatchID |
entity JSON |
prioritize |
QueueID |
entity JSON |
build |
BatchID |
entity JSON |
buildsignal |
BuildID |
entity JSON |
conclude |
BatchID |
entity JSON |
log |
RequestLog |
entity JSON |
merge |
MergeRequest |
✅ protojson (api/runway/messagequeue — correctly borrowed; runway owns that queue's contract) |
Consequences of the split: no additive-evolution guarantees (protojson's unknown-field discard, UPPER_SNAKE enums, int64-as-string are only enforced on merge), no topic_keys binding or contract test for internal topics, and queue-payload serialization concerns living on domain entities (which are supposed to be pure data).
Work
- Create
submitqueue/core/messagequeue/mirroring the reference layout:proto/sources, committedprotopb/, and the generic protojson glue (Marshal/Unmarshal[T]/TopicKeys) — same asstovepipe/core/messagequeue/. - Define proto messages for each payload shape (request-ID carrier, batch-ID carrier, build-ID carrier, queue-ID carrier, cancel request, request log) and bind each topic key to its message via the
topic_keysoption fromapi/base/messagequeue. - Add the contract test: protojson round-trip per message + every topic key bound to exactly one message.
- Migrate producers and consumers topic by topic (gateway
Land/Cancelpublishes, all orchestrator stage controllers,core/request.PublishLog, and the DLQ controllers/republish tooling — anything touchingmsg.Payload). - Once no queue payload uses them, remove
ToBytes/FromBytesfrom the submitqueue entity types (RequestID,BatchID,BuildID,QueueID,CancelRequest,RequestLog, and the full-entity variants) so entities go back to being pure data. - Bazel
visibilitystays domain-scoped per the internal-contract rule.
Caveats
- Wire-format flip: protojson output (snake_case fields, int64-as-string) differs from the current
encoding/jsonentity output, so in-flight messages from before a cutover won't parse after it. Port topic-by-topic with drained queues (fine at the current pre-prod stage), or give consumers a transitional dual-format read if needed. - The
logtopic crosses gateway ↔ orchestrator but both are submitqueue-domain services, so the internal contract location (submitqueue/core/messagequeue/) is correct — it must not go underapi/. - The message-ID convention is a separate concern tracked in #352 (intent-scoped message IDs); porting payloads to proto neither depends on nor resolves it, but call sites will be touched by both — coordinate to avoid churn.
Related
- #352 — intent-scoped message IDs (same call sites)
- #357 — stovepipe counterpart (retire leftover JSON serializers, contracts for upcoming stages)
- Lingua principale
- Go
- Stelle
- 225
- Fork
- 11
- Merge medio
- 3g 8h
- PR unite (30g)
- 61
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di uber/submitqueue
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
uber/submitqueue#209 ·
Tutte le issue di uber/submitqueue
Issue simili
-
kind/bug needs-triage
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 72/100
matrixorigin/matrixone#29223 ·
-
needs-acceptance wg/data-plane-networking
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
vllm-project/semantic-router#4024 · 1 commento ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 86/100
alexgorbatchev/dotfiles#107 ·
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 84/100