Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

ops(backtests): re-argue MAX_ACTIVE_DASHBOARD_BACKTESTS on spend, not RAM

Open
  • Is something wrong?
#473 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
backend

Research direction

Read the implementation of MAX_ACTIVE_DASHBOARD_BACKTESTS and DEFAULT_MAX_CONCURRENT_BACKTESTS, then inspect credit_ledger_entries for settled tokens by run and call. Estimate the cost of a representative dashboard backtest and agree on an acceptable concurrent spend rate before choosing the cap. Done means the cap and its comment reflect that decision while preserving the stated quota, zero-value, and invalid-value behavior.

Written by the indexing model from the issue text.

Description

MAX_ACTIVE_DASHBOARD_BACKTESTS (default 5) was sized as a RAM bound against a 512MB free instance. Prod moved to Standard 1c-2g on 2026-09-11 (#308, PR #472), which raised the ceiling without establishing where it now sits — nothing has ever measured one child's resident set, and the ~1.1GB of headroom above the observed 882MB peak belongs to the instance, shared across however many slots this value opens. So RAM is not ruled out here; it has only stopped being the obviously binding constraint. That measurement is #475.

The bound that can be argued without a measurement is operator LLM spend, and nobody has ever argued that number either. Before the slot ledger existed the runner was single-flight and the ceiling was exactly 1; the value multiplies concurrent billable API cost by the same factor.

What this issue wants

A number chosen on cost, not inherited from a dead memory constraint:

  • Measure the per-run LLM cost of a representative dashboard backtest (credit_ledger_entries already records settled tokens per (run_id, call_index, attempt_index) — the data is there).
  • Decide the acceptable concurrent burn rate, then set the cap from it.
  • Update the comment so the stated reason matches the real one.
Constraints to keep
  • It is deliberately equal to DEFAULT_MAX_CONCURRENT_BACKTESTS (5) so one default-entitlement account can reach its own quota. Raising one without the other makes the entitlement unreachable or the global cap unreachable.
  • 0 disables all dashboard backtests.
  • Junk/negative falls back to 5 with a log line — an unparseable value used to kill app boot.
  • It is env-overridable, so a change needs no deploy.
Related

  • #308 — the OOM that produced the original 512MB sizing (closed; prod is 2GB).

  • #363 — the cap is per-process, not server-wide.

  • #364 — the per-owner cap above it is session-keyed, so it is an incentive fix.

  • PR #472 — corrected the stale rationale in the comments without moving any value.

  • #475 — the per-child peak-RSS measurement. Deliberately not bundled with this issue: that one is gated on a measured resident set, this one on money. Raising this value wants both.

Dominant language
Python
Stars
767
Forks
156
Avg merge
21h 57m
Merged PRs (30d)
92

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Open-Finance-Lab/AgenticTrading

All issues in Open-Finance-Lab/AgenticTrading

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.