[experimental chore/agentx-v0.2-aiperf-testing branch] agentic build_replay_cmd: --cache-bust system_prefix mismatched with current aiperf scenario (requires first_turn_prefix) / agentic build_replay_cmd 的 --cache-bust system_prefix 与当前 aiperf 场景不匹配(需要 first_turn_prefix)
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Quiet
- Domain
- performance
Research direction
Start in benchmarks/benchmark_lib.sh at build_replay_cmd around line 1000, and review the launcher comment near line 950 alongside the inferencex-agentx-mvp validation described in the issue. Update the cache-bust setting and comment consistently, then run a DURATION=1800 sweep with the relevant offloading configurations; done means aiperf validation passes without launcher errors.
Written by the indexing model from the issue text.
Description
Summary
benchmarks/benchmark_lib.sh (function build_replay_cmd, around line 1000 of v0.2 tip) passes --cache-bust system_prefix to aiperf profile, but the current utils/aiperf submodule (70fecb2e) hardcodes first_turn_prefix as the only valid value for the inferencex-agentx-mvp scenario. The validation is fatal:
Value error, Scenario invariants violated (1 conflict):
- --cache-bust: got 'system_prefix', required 'first_turn_prefix'
(scenario 'inferencex-agentx-mvp' requires
cache_bust.target=first_turn_prefix; got system_prefix)
Pass --unsafe-override to convert to warnings (run will be marked
submission_valid=false).
This is masked for short smoke runs because build_replay_cmd auto-adds --unsafe-override when DURATION < 900. Production sweep configs use DURATION=1800 and hit the error directly — the run flips to "invalid" or fails at validation, depending on flags.
Repro
# Any cpu/noffload sweep with DURATION=1800 (the production default)
podman run ... \
-e MODEL=MiniMaxAI/MiniMax-M2.5 -e TP=4 -e CONC=16 \
-e OFFLOADING=cpu -e TOTAL_CPU_DRAM_GB=1200 \
-e DURATION=1800 \
... /workspace/benchmarks/single_node/agentic/minimaxm2.5_fp8_mi355x.sh
# vllm starts cleanly (Application startup complete, /health 200), then
# aiperf profile bails immediately with the validator error above.
Suggested fix
- REPLAY_CMD+=" --cache-bust system_prefix"
+ REPLAY_CMD+=" --cache-bust first_turn_prefix"
…or, if backwards compat with older aiperf submodules matters, source the value from the scenario plugin via aiperf scenario describe inferencex-agentx-mvp and inject it dynamically.
After this patch, the full 30-min sweep runs cleanly (verified with both OFFLOADING=none and OFFLOADING=cpu, 11 + 8 conc points respectively, 0 launcher errors).
The launcher comment block at line 950 should also be updated — it currently says:
# The scenario plugin locks: --cache-bust system_prefix,
which became stale when the scenario validator was tightened.
Environment
- Branch:
chore/agentx-v0.2-aiperf-testing(tipc8dfb585) - aiperf submodule:
70fecb2e refactor(input): move --max-context-length filter into WekaTraceLoader
中文说明
benchmark_lib.sh 中 build_replay_cmd 向 aiperf profile 传递了 --cache-bust system_prefix,但当前 aiperf 子模块仅允许 first_turn_prefix 作为 inferencex-agentx-mvp 场景的有效值,验证失败是致命错误。短时间冒烟测试因自动添加 --unsafe-override 而被掩盖,但生产扫描配置使用 DURATION=1800 会直接触发此错误。建议将 system_prefix 改为 first_turn_prefix。
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 303
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 284
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from SemiAnalysisAI/InferenceX
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
SemiAnalysisAI/InferenceX#2125 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
SemiAnalysisAI/InferenceX#1587 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
SemiAnalysisAI/InferenceX#1369 · 3 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
SemiAnalysisAI/InferenceX#3322 ·
-
Difficulty 5/5 Over a week Newbie friendliness 30/100
SemiAnalysisAI/InferenceX#3122 · 3 comments ·
All issues in SemiAnalysisAI/InferenceX
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100