Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

M4: two static baselines of one session differ by 0.0092 px on the same footage, twelve times M3's spread, unexplained

Open
  • Is something wrong?
#123 7 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
10/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
javascript, typescript
Domain
performance, testing-qa

Research direction

Start with compareExports in examples/js/bench-metrics.mjs and the two round-1 static exports named in the issue, then recompute jitterPx on their 168 common media times. The issue's desk checks are corner agreement between those baselines and a per-window residual breakdown, and the calibration plan in docs/benchmarks/README.md sets the criteria. Done means a written account of the 0.0092 px gap, not a fix; the issue asks for an explanation, so expect to coordinate with the maintainer before changing anything.

Written by the indexing model from the issue text.

Description

benchmarks

Two static runs of one configuration, in one session, read 0.0092 px apart in jitterPx on the same footage. That is about twelve times the whole range of M3's four static runs, and nothing in the record accounts for it. This issue holds the question until the calibration block of M4's plan answers it (#122: docs/benchmarks/README.md, "The null control's jitter clause (2026-10-14)", item 3).

The observation

Round 1's two static baselines: 2026-09-28-tab9-tuning-r1-baseline-static.json and -baseline-static-repeat.json.

  • One configuration: Tab_9_WiFi, Chrome 153, the 64-patch target 4af6a7fb…, the tracker's defaults.
  • One session: they were its first and last runs, exported at 12:32:14 and 12:51:05, 19 minutes apart.
  • On their 168 common media times: compareExports (examples/js/bench-metrics.mjs) reads jitterPx 0.1476 and 0.1568, which is 0.0092 px apart.
  • Over their whole windows: 0.1476 → 0.1552, which is 0.0076 px apart. So a window covering different footage is not the cause.

M3's four static runs: 2026-10-01-tab9-m3-{sync,worker}-static-{1,2}.json.

  • Configuration: the 48-patch target 9e8eb486…, run with ?loops=4.
  • When: 07:35:46 to 07:46:07.
  • runSummary.jitterPx: 0.14163, 0.14175, 0.14192 and 0.14240. The range is 0.14240 − 0.14163 = 0.00077 px.

The ratio: 0.0092 ÷ 0.00077 = 11.9.

What the ratio is, and what it is not

It compares one pair's difference with the range of four runs:

  • the pair's difference is taken over 212 and 207 frames of common footage (a media time a window reaches again after a wrap counts each time);
  • the range of four is taken over 815 to 822 counted-loop frames per run.

From one spread, the range of four is expected to be wider than one pair's difference, and the smaller sample is expected to be noisier. So the ratio states the size of the question. It is not a ratio of two measured spreads.

Two more facts in the record that bear on it

  • Round 2's three static baselines (2026-09-29-tab9-tuning-r2-baseline-static-{1,2,3}.json): one session, 64 patches, the window procedure.
    • Over their windows they read 0.14984, 0.15676 and 0.15647, a range of 0.15676 − 0.14984 = 0.00692 px.
    • That is over windows, not common footage, but it is of the order of round 1's, not M3's.
  • M3's frame-resampling figure. M3's record (docs/benchmarks/2026-10-01-m3-device-session.md, "What was looked at before the decision") reports a check that kept each processed frame with probability 0.9.
    • That moves a run's jitterPx with a standard deviation of 0.0013–0.0015 px, about twice the four runs' whole range (0.0014 ÷ 0.00077 = 1.8).
    • Nothing in the record accounts for how a resampling spread that wide sits beside a run-to-run range that narrow. It is one more reason not to take 0.00077 px as the norm.

Candidates, none of them shown

  • The time and the device's thermal state. Round 1's pair spans 19 minutes of a session; M3's four ran in about 10.
  • The configuration. Round 1 ran 64 patches under the window procedure (a run ended by Stop, its last 300 ticks kept); M3 ran 48 patches with ?loops=4.
  • Sampling. The common-footage sample is the smaller one.
  • The build and the harness, which differed between 2026-09-28 and 2026-10-01.

Why it matters

  • Round 1 does not transfer. It ran 64 patches under the window procedure, so its figure cannot be assumed to hold for the campaign's configuration.
  • The calibration block is where we find out which figure is normal for the campaign's configuration. It is eight static runs on Tab_9_WiFi at 37dc6d4, distributed in time as counted runs will be.
  • If the variability is real and reappears in M4, the paired design itself is at risk. A treatment's difference from its baseline would then carry run-to-run movement an order of magnitude above what M3's four suggest.

Desk checks it could start with (none run yet)

  • Corner agreement at common media times between round 1's two baselines. M3's record measured 0.0003–0.0006 px at the median for its four runs.
  • Round 2's three baselines, compared on their common media times.
  • Where the 0.0092 px sits: the residuals window by window over the common footage, to see whether the difference is spread across the clip or concentrated.

What decides it: the calibration block's eight runs, read by the plan's rule and its within-run check, beside M3's four.

Dominant language
TypeScript
Stars
11
Forks
3
Avg merge
3h 13m
Merged PRs (30d)
48

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from webarkit/webarkit

All issues in webarkit/webarkit

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.