DLQ marks a request terminal error after its batch already handed off to land

Open
#818 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
54/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
go

Research direction

Start by tracing the speculate controller's batch outcome handoff and the DLQ reconciliation paths for land and landsignal; the payload does not name specific files or tests. Check how reconciliation determines whether a batch has handed its outcome to the land. Done means a request in that state is not marked terminal error by DLQ reconciliation, including the reachable land and landsignal cases.

Written by the indexing model from the issue text.

Description

bug

Problem

A request can be reported as error after its change has already landed on the target branch.

Cause

The speculate controller decided a batch's outcome and published the land before its delivery finished. When something later in the same delivery failed, the message was rejected, and the DLQ reconciler marked the request terminal error without checking that the batch had already handed its outcome to the land. The merge then landed anyway.

Observed on a demo run: the land was published, the same delivery failed, the DLQ marked the request error, and the merge appeared on main three seconds later.

Expected

A request whose batch has handed its outcome to the land must not be moved to error by DLQ reconciliation.

Status

No fixing change has been identified on main. The same class is still reachable through the land and landsignal DLQs; see the companion issue linked from #819.

Dominant language
Go
Stars
247
Forks
16
Avg merge
2d 1h
Merged PRs (30d)
125

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from uber/submitqueue

All issues in uber/submitqueue

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.