lifecycle/stream: fetchMessages doesn't check done before calling fetch again
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- go
- Domain
- stream-processing
Research direction
Start in fetchMessages in pkg/lifecycle/stream/base.go and read how the loop selects on done when handing off results, then see where fetch(ctx) is called. Add a non-blocking check of done at the top of the loop, before fetch. Done means a test with a fetcher that returns empty batches shows the goroutine exits after cleanup.
Written by the indexing model from the issue text.
Description
Follow-up to #2969 / #2975 (pre-existing behaviour, not introduced by the fix).
fetchMessages in pkg/lifecycle/stream/base.go selects on done when handing a message or error over, but does not check done before calling fetch again. A source that keeps returning empty batches (no messages, no error) after the node has torn down never reaches a hand-over, so the goroutine spins calling Source.Read until ctx is cancelled.
Suggested fix: check done (non-blocking select) at the top of the loop, before fetch(ctx). Add a test with a fetcher that returns empty batches and assert the goroutine exits after cleanup.
Not data loss: no record is acked or dropped by this; it is a CPU/goroutine leak after the node stopped.
🤖 Generated with Claude Code
- Dominant language
- Go
- Stars
- 611
- Forks
- 63
- Avg merge
- 1h 10m
- Merged PRs (30d)
- 30
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ConduitIO/conduit
-
processor-plugins install/uninstall --help list wrong exit codesPossibly taken @devarismeroxa claimed this 1 day ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
housekeeping
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
Maintainers usually reply within 1 day
All issues in ConduitIO/conduit
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
siyuan-note/siyuan#20353 ·
Maintainers usually reply within 1 day
-
attributes-natural-language "en-US" is rejected by PAPPL >= 1.4.12 printers (RFC 8011 requires lowercase)Possibly taken @ChrisEdgington claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 84/100
OpenPrinting/ipp-usb#140 ·
-
Discriminator mapping keys are listed in a random orderPossibly taken @reuvenharrison claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Idle compaction monitors LIST the replica every tick when the newest destination file spans more than one TXIDPossibly taken @pishuv claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
benbjohnson/litestream#1563 ·
Maintainers usually reply within 2 days
-
triage needed
Difficulty 1/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 2 days