engine: measure the legacy-vs-new gap on recorded decision tapes
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 15/100
Research direction
Start with the valuation_cursor loop in engine.py (around lines 2369-2390) to see how the legacy engine marks bars against the current fill_plan.bar, then read HourlyBacktester to find where decision tapes are recorded. Before any replay, the marking rule to score must be chosen and stated, since it decides whether D1 is counted in the gap. Done means the methodology page reports the legacy-vs-new gap on My Agents tapes only, with the chosen marking rule documented.
Written by the indexing model from the issue text.
Description
Once the new core passes conformance groups F, L, T and CA, replay recorded decision tapes through both engines and report the gap. That number, not fidelity to the legacy engine, is what the rewrite is for. It belongs on the methodology page.
Before comparing, decide which marking rule is scored. In intraday mode the legacy engine marks every source bar through the current fill_plan.bar after executing bar i (the valuation_cursor loop, engine.py:2369-2390). A fill's slippage therefore already shows on up to an hour of earlier curve points. This is conformance defect D1, owned by case M01. A core that marks causally will diverge at every fill, so the report has to state which rule it scores, or separate the D1 component out.
What can be replayed: only My Agents runs (HourlyBacktester) record tapes. The contest and live-board loops (domain/leaderboard/) record none, so their published medians cannot be replayed. The comparison set is My Agents tapes. The Competition board is retired rather than re-run (direction spec §6).
Source: decision-tape spec, follow-ups 4 and 5.
- Dominant language
- Python
- Stars
- 767
- Forks
- 156
- Avg merge
- 21h 57m
- Merged PRs (30d)
- 92
Getting set up
- Ships a Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Open-Finance-Lab/AgenticTrading
-
fix(llm): a non-ASCII execution handoff raises UnicodeEncodeError instead of ExecutionHandoffErrorPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Open-Finance-Lab/AgenticTrading#659 ·
Maintainers usually reply within 1 day
-
fix(backtests): an LLM action with trailing whitespace ("buy ") is silently droppedPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Open-Finance-Lab/AgenticTrading#658 ·
Maintainers usually reply within 1 day
-
fix(llm): Gemini adapter drops thinking tokens from the output countPossibly taken A pull request linked to this issue is open or already merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Open-Finance-Lab/AgenticTrading#657 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
Open-Finance-Lab/AgenticTrading#591 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Open-Finance-Lab/AgenticTrading#586 ·
Maintainers usually reply within 1 day
All issues in Open-Finance-Lab/AgenticTrading
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
MISP/misp-workbench#496 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
fossasia/visdom-dev#69 ·
Maintainers usually reply within 3 days
-
[Bug] Parameters named like a property break to_xcal()Possibly taken @atharvasalunke1267 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
collective/icalendar#2018 · 2 comments ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day