benchkit/sweep: measure and reduce per-run orchestration overhead
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 35/100
- Issue type
- Refactor
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- go
- Domain
- cli, performance
Research direction
Start with the perRunOverhead constant in benchkit/cmd/sweep/eta.go, then read the run loop in benchkit/cmd/sweep/main.go and launchAndWait in deploy.go. Check AwaitPeersDoneOrGrace in benchkit/benchmark/symmetric.go to see when runs fall back to the grace deadline. Done means before/after per-run timings on representative cluster sizes, with readiness and result completeness unchanged and the static estimate kept as specified.
Written by the indexing model from the issue text.
Description
The benchkit TODO records roughly 15 seconds of orchestration overhead per sweep run, which is too costly for large sweeps.
Measure the current overhead and reduce the remaining avoidable work between measurement windows.
The current perRunOverhead = 15 * time.Second in eta.go is an allowance in the static completion estimate, not a fixed sleep.
The benchmark already signals peer completion and can exit before its grace deadline through AwaitPeersDoneOrGrace.
Focused tests confirm the early-exit path, but no fresh remote measurement was taken during this review, so 15 seconds should not be treated as a verified current cost.
Work to do:
- Measure end-to-end per-run overhead on representative cluster sizes, separating one-time build/deployment costs from repeated run costs.
- Break down cleanup/port checks, launch/readiness and clock synchronization, completion/teardown, result collection, and local processing.
- Check how often healthy runs fall back to the peer-completion grace deadline and identify why.
- Remove unnecessary waits, repeated setup, or serialized work where measurements show a benefit, while preserving readiness, complete results, and bounded failure handling.
- Record before/after measurements and update the static overhead allowance if the evidence supports it.
Keep the immediate static earliest–latest estimate, with measurement time alone as its lower bound.
Do not recalibrate that estimate from the first run or count lowering the estimate constant as a performance improvement.
Relevant orchestration: main.go and launchAndWait.
- Dominant language
- Go
- Stars
- 149
- Forks
- 15
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 43
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from relab/gorums
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
Maintainers usually reply within 1 day
-
Difficulty 4/5 3-5 days Newbie friendliness 22/100
Maintainers usually reply within 1 day
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
Maintainers usually reply within 1 day
-
doc
Difficulty 3/5 Half a day Newbie friendliness 45/100
Maintainers usually reply within 1 day
Similar issues
-
date_time, url_encode, html_escape, to_unix_time, contains_all and contains_any panic when called without argumentsPossibly taken @antcybersec claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
projectdiscovery/dsl#372 ·
Maintainers usually reply within 1 day
-
blocked-needs-validation
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
projectcapsule/capsule#2265 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
JuliusBrussee/caveman#1449 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
bot-found bug priority: P3
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
madenvel/KalinkaPlayer#498 ·