Drift baselines never form on bulk-ingested SDK data (baseline null after 21k spans)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 45/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- python
- Domain
- observability
Research direction
Start by reading tokenjam/core/drift.py to trace when SDK agents' baselines are computed and what triggers evaluation during bulk ingest. Reproduce the reported case with bulk-ingested spans, then verify baseline formation and drift detection on a synthetic drifting stream. Done when the root cause is documented and either bulk ingest forms baselines or the warm-up behavior is explicit and tested.
Written by the indexing model from the issue text.
Description
Severity: Low
Model: sonnet
Severity: Low
Where: tokenjam/ — tokenjam/core/drift.py baseline computation
Observed: After ingesting 9 full agent runs (21,562 spans) into a fresh tj serve, drift baseline was null for every agent; no drift evaluation occurred. Likely baselines need accumulated history across multiple ingest passes or a compute trigger that a single bulk ingest into a short-lived server never hits. Not fully root-caused (timeboxed).
Fix direction: Investigate when/how drift baselines are computed for SDK agents; ensure a bulk-ingested or newly-onboarded agent gets a baseline within a bounded window (or document the intended warm-up), then verify drift fires on a synthetic drifting stream.
Done when: root cause documented and either baselines form after bulk ingest or the warm-up behavior is explicit and tested.
Ref: SDK benchmark dogfood (HAL trace replay), 2026-07-17.
- Dominant language
- Python
- Stars
- 122
- Forks
- 35
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 8
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Metabuilder-Labs/tokenjam
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Metabuilder-Labs/tokenjam#786 ·
Maintainers usually reply within 1 day
-
enhancement good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Metabuilder-Labs/tokenjam#759 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Metabuilder-Labs/tokenjam#728 ·
Maintainers usually reply within 1 day
-
bug help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Metabuilder-Labs/tokenjam#591 ·
Maintainers usually reply within 1 day
-
bug good first issue help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Metabuilder-Labs/tokenjam#559 ·
Maintainers usually reply within 1 day
All issues in Metabuilder-Labs/tokenjam
Similar issues
-
Project submission:Open
Difficulty 1/5 Under an hour Newbie friendliness 72/100
slavakurilyak/awesome-ai-agents#845 ·
Maintainers usually reply within 1 day
-
tests: pin that the atomic writers in utils/paths.py stage the temporary file in the target's directoryPossibly taken A pull request linked to this issue is open or already merged. Opendifficulty:2 enhancement good first issue
Difficulty 1/5 1-3 hours Newbie friendliness 92/100
MakazhanAlpamys/Soup#2178 ·
Maintainers usually reply within 1 day
-
[BUG] dcg.waggr permutation p-values are folded twice, so near-zero scores get the smallest p-valuesOpen
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
scverse/rapids-singlecell#854 ·
Maintainers usually reply within 1 day
-
Slang input: int8_t4_packed and uint8_t4_packed are copied into CrossGLPossibly taken @ninglinLiu claimed this today. Openarea:frontend backend:slang bug direction:input
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
random_elements(unique=True) on an OrderedDict: ZeroDivisionError once only zero weights remain, AssertionError with use_weighting=FalsePossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 13 days