drawElement self-verify PSNR gate cannot detect missing low-contrast content
维护者通常 1 天内回复
评估
这个 Issue 还没有评估数据。
描述
Describe the bug
The drawElement self-verify gate cannot detect a missing low-contrast element, because
PSNR is magnitude-weighted. An entire element can be absent from a rendered frame and
still score well above the pass threshold, so the damaged frame ships with a clean exit
code.
The gate is in packages/producer/src/services/render/stages/captureStreamingStage.ts:341:
db = await psnrDb(buf, truth);
...
if (db < verifyMinDb) { throw new DrawElementVerificationError(...) }
psnrDb takes FFmpeg's average: PSNR across components
(packages/engine/src/utils/psnr.ts:13), and verifyMinDb defaults to 32
(psnr.ts:43). PSNR falls as the magnitude of the difference grows. A whole element
that is merely low-contrast against its background produces a small magnitude over a
modest area, so it scores high and passes.
The failure mode this permits is the worst kind we can ship: a video that is wrong, with
no warning and exit code 0. High-contrast damage in the same frame is caught, which is
why this reads as "some elements render and some silently do not" rather than as a
general breakage.
Steps to reproduce
No project needed. This measures the gate itself against synthetic 4K frames, using the
same comparison psnrDb performs.
#!/usr/bin/env bash
set -uo pipefail
FF="${FF:-ffmpeg}"; W=/tmp/psnr; rm -rf $W; mkdir -p $W; cd $W
BG="0x12122E"; BAND=324 # 3840x324 = 1,244,160 px = 15.0% of a 4K frame
mkframe(){
if [ -z "$2" ]; then
$FF -v error -y -f lavfi -i "color=c=$BG:s=3840x2160:d=1" -frames:v 1 "$1"
else
$FF -v error -y -f lavfi -i "color=c=$BG:s=3840x2160:d=1" \
-vf "drawbox=x=0:y=900:w=3840:h=$BAND:color=$2:t=fill" -frames:v 1 "$1"
fi
}
cmp_psnr(){ $FF -hide_banner -i "$1" -i "$2" -lavfi psnr -f null - 2>&1 \
| grep -o 'average:[^ ]*' | head -1; }
mkframe truth.jpg ""
for spec in "0x161632:~4/255 luma delta" "0x1E1E42:~10/255 luma delta" \
"0x262556:~22/255 luma delta" "0xFFFFFF:white (control)"; do
col="${spec%%:*}"; label="${spec#*:}"
mkframe damaged.jpg "$col"
db=$(cmp_psnr damaged.jpg truth.jpg)
printf " %-26s %-20s %s\n" "$label" "$db" \
"$(awk -v d="${db#average:}" 'BEGIN{print (d+0>=32)?"PASSES 32 dB gate -> damage ships":"fails gate -> caught"}')"
done
truth.jpg is the background with the panel missing; damaged.jpg is the correct frame.
The PSNR between them is exactly what the gate computes when the panel fails to draw.
Expected behavior
A frame missing an entire visible element fails verification, regardless of that
element's contrast against its background.
Actual behavior
~4/255 luma delta average:44.141763 PASSES 32 dB gate -> damage ships
~10/255 luma delta average:35.739625 PASSES 32 dB gate -> damage ships
~22/255 luma delta average:31.063362 fails gate -> caught
white (control) average:10.743118 fails gate -> caught
An element covering 15% of a 4K frame can go completely missing and still score 44 dB,
12 dB clear of the floor. The gate only starts catching it somewhere around a 12/255
luma delta at that coverage, and the threshold moves with area: a smaller element needs
proportionally more contrast before the gate notices it is gone.
Two separate holes
1. The metric is wrong for the question. PSNR answers "how far off are these pixels
on average", but the gate is asking "did any content go missing". Those come apart
precisely when the missing content is low-contrast. Raising HF_DE_VERIFY_MIN_DB is not
a fix: 44 dB would need a floor above 44, which would then reject ordinary JPEG encoder
noise. A presence-sensitive check is needed instead, for example a per-region or
worst-block comparison rather than a whole-frame average, so a fully wrong region cannot
be averaged away by a correct background.
2. The gate does not run on most frames. It is guarded by
const truth = session.deVerifyFrames?.get(idx) (captureStreamingStage.ts:337), so
frames without a sampled truth image are not checked at all. Even damage severe enough
to fail the threshold ships if it lands on an unsampled frame.
Scope of what is proven here
This demonstrates that the verification gate is blind to this class of damage. It does
not explain why drawElement omits a low-contrast element in the first place. Those are
separate bugs, and this one is worth fixing on its own: without a gate that can see the
damage, the underlying drop stays invisible in production no matter what causes it.
Until the metric is fixed, it is worth considering whether the streaming path should
fail loudly rather than proceed when it cannot verify a frame, since a hard failure is
recoverable and a silently wrong deliverable is not.
Environment
FFmpeg 7.0.2-static (linux x64)
Node.js v22.22.2
Gate packages/engine/src/utils/psnr.ts resolveDeVerifyMinDb() default 32
Call packages/producer/src/services/render/stages/captureStreamingStage.ts:341
Also packages/engine/src/services/parallelCoordinator.ts:613 (same gate, disk path)
- 主要语言
- TypeScript
- 星标
- 54.1k
- 派生
- 4.9k
- 平均合并
- 7 小时 18 分钟
- 30 天内合并 PR
- 784
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
heygen-com/hyperframes 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 85/100
heygen-com/hyperframes#5027 ·
维护者通常 1 天内回复
-
fix(producer): propagate useGpu to HDR layered streaming encoder可能已有人在做 @Monster-GM 于 1 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 87/100
heygen-com/hyperframes#5002 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
heygen-com/hyperframes#4702 · 1 条评论 · 1 个 reaction ·
维护者通常 1 天内回复
-
Studio catalog prompt editor has no accessible name可能已有人在做 @lorenzozanee 于 12 天前认领。 未关闭bug difficulty/easy triage/ready
难度 2/5 1-3 小时 新手友好度 78/100
heygen-com/hyperframes#4384 ·
维护者通常 1 天内回复
-
lint: validate composition variables declared on supported root elements可能重新可做 关联的 PR 已关闭且未合并。 未关闭bug difficulty/easy triage/ready
难度 2/5 1-3 小时 新手友好度 88/100
heygen-com/hyperframes#4383 ·
维护者通常 1 天内回复
查看 heygen-com/hyperframes 的全部 Issue
相似的 Issue
-
bot:ai-assisted status:untriaged
难度 2/5 1-3 小时 新手友好度 85/100
midnightntwrk/midnight-js#1424 ·
维护者通常 1 天内回复
-
enhancement
难度 2/5 1-3 小时 新手友好度 68/100
mksglu/context-mode#1268 ·
维护者通常 5 天内回复
-
Edit:未关闭check:failed streams:edit
难度 2/5 1-3 小时 新手友好度 60/100
维护者通常 1 天内回复
-
[Table] reserveSelectedRowOnPaginate=false 时表头全选包含其他页数据可能已有人在做 @dvd233 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 75/100
Tencent/tdesign-react#4416 · 2 条评论 ·
维护者通常 1 天内回复
-
area:widget bug
难度 2/5 1-3 小时 新手友好度 66/100
interledger/publisher-tools#894 ·
维护者通常 1 天内回复