snapshot: `<video>` frames come out ~4% darker than the render on Chrome 152 (snapshot's frame PNGs keep the BT.709 cICP tag)

La extracción de fotogramas vive en packages/cli/src/commands/snapshot.ts (extractVideoFrameToBuffer); compárala con packages/engine/src/services/videoFrameExtractor.ts, que usa SDR_CANVAS_PASSTHROUGH_FILTER. Ejecuta repro.sh para reproducir la diferencia de luma y luego añade el mismo filtro de color de ffmpeg al export PNG de snapshot. Listo cuando measure.py muestre ~0.4% de diferencia de luma en stills de <video> en Chrome 152.

Summary

hyperframes snapshot shows footage darker than hyperframes render does at the same time. On the managed Chrome 152 headless shell, the video area of every still is 3.8% darker in mean luma than the rendered frame. Shadows take the biggest hit: a channel value of 16 in the render comes out as about 4 in the still. CSS colours are identical in both outputs. Only the <video> pixels differ.

This still happens on the latest release, 0.8.189.

It looks like the snapshot-side counterpart of #4549. That fix retags the render extractor's frames as sRGB. Snapshot extracts its own frames, which still carry the source's BT.709 transfer tag, and Chrome 152 colour-manages that tag.

Environment

• macOS 27.0.1 (26A434), arm64 (Apple Silicon)
• Node v22.23.2, ffmpeg 9.0.1
• hyperframes 0.8.177, 0.8.178, 0.8.186, 0.8.189
• chrome-headless-shell 152.0.7977.30 (the managed one in ~/.cache/hyperframes/chrome) and 131.0.6778.204 (an older one in ~/.cache/puppeteer)
• Env for every run: HYPERFRAMES_NO_UPDATE_CHECK=1 HYPERFRAMES_NO_AUTO_INSTALL=1 HYPERFRAMES_SKIP_SKILLS=1

Repro

The repro uses a 1080x1920 composition with two flat CSS fields, a generated 16:9 testsrc2 clip with object-fit: cover (starting at 0.5 s), and a bordered box.

assets/testsrc2.mp4, BT.709 limited range with every tag set:

ffmpeg -f lavfi -i testsrc2=size=1280x720:rate=30:duration=3 -c:v libx264 -pix_fmt yuv420p \
  -x264-params colorprim=bt709:transfer=bt709:colormatrix=bt709:range=tv assets/testsrc2.mp4

index.html (lint: 0 errors, 0 warnings):

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=1080, height=1920" />
    <script src="https://cdn.jsdelivr.net/npm/[email protected]/dist/gsap.min.js"></script>
    <style>
      body { margin: 0; background: #000; }
      #root { position: relative; width: 100%; height: 100%; overflow: hidden; background: #000; }
      #grey { position: absolute; left: 0; top: 0; width: 540px; height: 480px; background: #808080; }
      #blue { position: absolute; left: 540px; top: 0; width: 540px; height: 480px; background: #3070c0; }
      #clip { position: absolute; left: 0; top: 480px; width: 1080px; height: 1440px; object-fit: cover; }
      #box { position: absolute; left: 340px; top: 1500px; width: 400px; height: 300px; box-sizing: border-box; border: 12px solid #ffffff; }
    </style>
  </head>
  <body>
    <div id="root" data-composition-id="main" data-start="0" data-width="1080" data-height="1920" data-duration="3">
      <div id="grey"></div>
      <div id="blue"></div>
      <video id="clip" src="assets/testsrc2.mp4" muted playsinline data-start="0.5" data-duration="2" data-track-index="0"></video>
      <div id="box" class="clip" data-start="1" data-duration="1.5" data-track-index="1"></div>
    </div>
    <script>
      window.__timelines["main"] = gsap.timeline({ paused: true });
    </script>
  </body>
</html>

Commands:

npx --yes [email protected] render project --format png-sequence -o out/render
npx --yes [email protected] snapshot project --at 0.2,0.467,0.5,0.55,1,1.55,2,2.05 --describe false --no-end -o out/stills
python3 measure.py out/render out/stills

Each still is compared with the rendered frame showing at the same instant (frame_{floor(t*30)+1}). The script reports:

• whole-frame mean absolute difference (0-255)
• mean BT.709 luma of the video area, as a percentage difference
• which neighbouring render frame (±3) matches best
• the translation of the video area, from phase correlation

repro.sh (under Repro files, below) runs the steps above for several versions.

import re, sys
from pathlib import Path
import numpy as np
from PIL import Image

VIDEO = (slice(480, 1920), slice(0, 1080))  # the <video> box: rows, cols

def load(p): return np.asarray(Image.open(p).convert("RGB"), np.float64)
def luma(a): return a @ np.array([0.2126, 0.7152, 0.0722])

def shift(a, b):
    fa, fb = np.fft.fft2(luma(a)), np.fft.fft2(luma(b))
    r = fa * np.conj(fb)
    c = np.abs(np.fft.ifft2(r / (np.abs(r) + 1e-9)))
    dy, dx = np.unravel_index(c.argmax(), c.shape)
    h, w = c.shape
    return int(dx - w if dx > w // 2 else dx), int(dy - h if dy > h // 2 else dy)

rdir, sdir, fps = Path(sys.argv[1]), Path(sys.argv[2]), 30.0
frames = {int(p.stem.split("_")[1]): p for p in rdir.glob("frame_*.png")}
for p in sorted(sdir.glob("frame-*-at-*.png")):
    t = float(re.search(r"at-([\d.]+)s", p.name).group(1))
    k = int(t * fps + 1e-6) + 1  # frame_000001 is t = 0
    s = load(p); sv = s[VIDEO]
    cands = {d: load(frames[k + d]) for d in range(-3, 4) if k + d in frames}
    mads = {d: np.abs(sv - r[VIDEO]).mean() for d, r in cands.items()}
    best = min(mads, key=lambda d: (round(mads[d], 3), abs(d)))
    r = cands[0]; yr, ys = luma(r[VIDEO]).mean(), luma(sv).mean()
    print(p.name, f"mad {np.abs(s - r).mean():.2f}", f"dY {100*(ys-yr)/max(yr,1e-9):+.2f}%", f"best {best:+d}", "shift", shift(sv, cands[best][VIDEO]))

Expected

With the video showing (0.5 to 2.5 s), each still's video area should match the rendered frame at the same time, as it did on Chrome 131: about 1 mean absolute difference and 0.4% luma.

Actual

All renders are on Chrome 152. Renders from 0.8.177, 0.8.178 and 0.8.189 are bit-identical: 90 frames, maximum pixel difference 0.

• No video on screen (t = 0.2 s and 0.467 s): the stills match the render exactly in every version (mad 0.00). The #808080 and #3070c0 fields are identical in every still.
• Timing and position: in every case the best-matching render frame is the expected one, and the translation is (0, 0).
• Tone change at t = 1 s, 0.8.189 (mean still value over pixels where the render channel is v): 16 → 4.0, 32 → 15.1, 64 → 50.6, 128 → 121.2, 192 → 189.4, 255 → 255.0. On 0.8.177 / Chrome 131: 16 → 14.8, 64 → 60.3, 128 → 128.1, 192 → 192.8.

Bisect

The bad output first appears in 0.8.178 (0.8.177 is good), but in this setup it follows the Chrome build, not the HyperFrames version.

• snapshot.ts and the composition-page code are the same in 0.8.177 and 0.8.178 (dist diff, ignoring chunk hashes).

• What 0.8.178 changed is the browser lookup order in src/browser/manager.ts findFromCache(). 0.8.177 used a chrome-headless-shell found in ~/.cache/puppeteer before its own ~/.cache/hyperframes one; 0.8.178 checks its own first. On this machine that moves snapshot from Chrome 131 to 152. render already used 152 in both.

• Swapping the browser with HYPERFRAMES_BROWSER_PATH moves the darkening with it:

  • 0.8.177 on Chrome 152: -3.77 to -3.82% (bad)
  • 0.8.178 on Chrome 131: -0.40 to -0.41% (good)

  A machine without an old puppeteer cache probably shows this on 0.8.177 too.

Likely cause (a guess: not confirmed by patching HyperFrames)

extractVideoFrameToBuffer in packages/cli/src/commands/snapshot.ts runs ffmpeg to PNG with no colour filter. ffmpeg copies the source's tags into the PNG as cICP 01/01/00/01 (BT.709 transfer), plus gAMA.

The render extractor (packages/engine/src/services/videoFrameExtractor.ts) appends SDR_CANVAS_PASSTHROUGH_FILTER (setparams=color_primaries=bt709:color_trc=iec61966-2-1). Its PNGs are tagged sRGB, so Chrome leaves them alone. That is the #4549 fix, and snapshot doesn't seem to share it.

Chrome-only check (chrome-check/, under Repro files, below):

• Same frame, same decoded pixels, two tags: ffmpeg's default (cICP 01/01/00/01) and the render filter (sRGB).
• Both shown as <img> and screenshotted with --force-color-profile=srgb:

So Chrome 131 ignored the cICP chunk and Chrome 152 applies it. That matches the -3.8% seen in snapshot.

A possible fix is to give snapshot's frame extraction the same SDR colour handling as the render extractor, or to have snapshot call the engine's extractor. Either way the two paths can't drift apart again.

Repro files

project/index.html and measure.py are inline above; project/assets/testsrc2.mp4 comes from the ffmpeg command above.

#!/bin/sh
# Render the project once, snapshot it with each version, compare every still with the rendered frame.
#   ./repro.sh [out_dir] [snapshot versions...]     (needs ffmpeg, python3 with numpy + Pillow)
set -e
here=$(cd "$(dirname "$0")" && pwd)
out=${1:-$here/out}; [ $# -gt 0 ] && shift
versions=${*:-0.8.177 0.8.178 0.8.189}
render_version=0.8.189
times=0.2,0.467,0.5,0.55,1,1.55,2,2.05
export HYPERFRAMES_NO_UPDATE_CHECK=1 HYPERFRAMES_NO_AUTO_INSTALL=1 HYPERFRAMES_SKIP_SKILLS=1

clip=$here/project/assets/testsrc2.mp4
[ -f "$clip" ] || ffmpeg -hide_banner -loglevel error -f lavfi -i testsrc2=size=1280x720:rate=30:duration=3 \
  -c:v libx264 -pix_fmt yuv420p -x264-params colorprim=bt709:transfer=bt709:colormatrix=bt709:range=tv "$clip"

mkdir -p "$out"
[ -d "$out/render" ] || npx --yes hyperframes@$render_version render "$here/project" --format png-sequence -o "$out/render" >"$out/render.log" 2>&1
for v in $versions; do
  npx --yes hyperframes@$v snapshot "$here/project" --at $times --describe false --no-end -o "$out/stills-$v" >"$out/snapshot-$v.log" 2>&1
  echo "== snapshot $v vs render $render_version, browser: $(npx --yes hyperframes@$v browser path 2>/dev/null | tail -1)"
  python3 "$here/measure.py" "$out/render" "$out/stills-$v"
done

#!/bin/sh
# ./check.sh <chrome-headless-shell>...   (needs ffmpeg, node with puppeteer-core resolvable here, python3 + numpy + Pillow)
set -e
cd "$(dirname "$0")"
clip=../project/assets/testsrc2.mp4
# what snapshot's extractVideoFrameToBuffer writes: ffmpeg copies the source tags into cICP 1/1/0/1 (BT.709 transfer)
ffmpeg -hide_banner -loglevel error -ss 0.5 -i $clip -frames:v 1 -y snapshot-style.png
# what render's extractor writes: SDR_CANVAS_PASSTHROUGH_FILTER retags the transfer as sRGB
ffmpeg -hide_banner -loglevel error -ss 0.5 -i $clip -vf setparams=color_primaries=bt709:color_trc=iec61966-2-1 -frames:v 1 -y render-style.png
for exe in "$@"; do
  v=$(node shot.mjs "$exe" shot.png)
  python3 -c "
import numpy as np; from PIL import Image
y = np.asarray(Image.open('shot.png').convert('RGB'), float) @ [0.2126, 0.7152, 0.0722]
a, b = y[:360].mean(), y[360:].mean()
print(f'$v: snapshot-style (cICP BT.709) {a:.2f}, render-style (sRGB) {b:.2f}, diff {100*(a-b)/b:+.2f}%')"
done

// Screenshot page.html (the same PNG pixels twice, with two different colour tags) in the given Chrome.
import puppeteer from "puppeteer-core";
const [exe, out] = process.argv.slice(2);
const b = await puppeteer.launch({ executablePath: exe, headless: true, args: ["--force-color-profile=srgb"] });
const p = await b.newPage();
await p.setViewport({ width: 640, height: 720 });
await p.goto(new URL("page.html", import.meta.url).href, { waitUntil: "load" });
await p.screenshot({ path: out });
console.log(await b.version());
await b.close();

<!doctype html><html><body style="margin:0;background:#000"><img src="snapshot-style.png" style="display:block;width:640px;height:360px"><img src="render-style.png" style="display:block;width:640px;height:360px"></body></html>