Benchmark aggregation reports skipped workloads as successful timing samples
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 68/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- php, typescript, wordpress
- Domain
- backend, performance, testing
Research direction
Reproduce the issue with tests/bench/boot-timing.php through the decision benchmark profile without BENCH_BOOT_PHASE, then trace the result through the real dispatcher and the owning aggregation layer. Exercise executed, skipped, and failed callables and verify that skipped workloads produce no timing sample, retain their reason, and leave required matrix completeness incomplete.
Written by the indexing model from the issue text.
Description
Observed failure
The MDI decision matrix completed with PASS for both backends, including a boot-timing workload that returned skipped: missing_bench_boot_phase. Aggregation reported approximately 0.004 ms as its duration, presenting the skip path as a benchmark result rather than a skipped scenario.
Reproduction workload: Automattic/markdown-database-integration, tests/bench/boot-timing.php, matrix source 21267e18d41c88396e4883e227422158ba5c16b2. Invoke the decision benchmark profile without BENCH_BOOT_PHASE. Operator evidence: Homeboy job 7018d7ce-fdf0-442d-a95c-ad96eedac2c5, SQLite run b28d11dd-e33d-40cf-8944-683aacb5fbc3, native run 2cc3b041-9291-4e8b-823f-e05b92376607. The source and normal workload invocation are the portable reproduction; IDs require operator access.
Expected contract
The workload result boundary preserves typed skip status and reason. Skipped workloads contribute no performance sample and are explicitly represented in matrix completeness. A required skipped scenario leaves acceptance incomplete, even when other executed scenarios pass. Test aggregation with executed, skipped, and failed callables through the real dispatcher.
Inspect owning layers before implementation: if WP Codebox already preserves skip status and the loss occurs downstream, route the repair to that aggregator rather than duplicating logic.
AI Assistance
GPT-6 Astra through OpenCode inspected the benchmark output and workload source, searched existing trackers, and drafted this report under Chris Huber’s direction. Direct OpenCode repair runs use openai/gpt-5.6-terra.
- Dominant language
- TypeScript
- Stars
- 17
- Forks
- 4
- Avg merge
- 1h 39m
- Merged PRs (30d)
- 45
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Automattic/wp-codebox
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Automattic/wp-codebox#2531 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Automattic/wp-codebox#2467 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Automattic/wp-codebox#2466 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Automattic/wp-codebox#2105 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Automattic/wp-codebox#2061 ·
Maintainers usually reply within 1 day
All issues in Automattic/wp-codebox
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
AOSSIE-Org/DebateAI#611 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
openzim/mwoffliner#2933 ·
Maintainers usually reply within 1 day
-
Use the README category name for website links and submissionsPossibly taken @dajiaohuang claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
birobirobiro/awesome-shadcn-ui#647 ·
Maintainers usually reply within 2 days
-
Twake Drive picker: closePicker() never destroys the intent (stop() is on the promise returned by start(), not by create())Possibly taken @chibenwa claimed this today. Openclaude
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
linagora/twake-calendar-frontend#1498 · 1 comment ·
Maintainers usually reply within 1 day
-
check:passed streams:add
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day