Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Specify alpha association for VideoFrame buffer input and copyTo() output

Đang mở
  • Có gì không ổn?
#972 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
35/100
Loại issue
Tính năng
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
html, javascript
Lĩnh vực
api, web-dev

Hướng nghiên cứu

Read the WebCodecs sections linked in the issue for the VideoFrame buffer and image source constructors, copyTo(), and Convert to RGB frame. Compare their alpha association rules with the cited HTML canvas and ImageBitmap requirements; the issue proposes defining observable behavior, conversion precision, zero-alpha pixels, and YUVA semantics. Done means the specification resolves those cases and adds distinguishing translucent regression tests.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

The WebCodecs Editor's Draft checked on 2026-10-12 defines channel order and sample sizes for RGBA and BGRA, but does not explicitly say whether their RGB samples are premultiplied by alpha. The buffer constructor and copyTo() copy pixel data without specifying an alpha-association conversion. When an RGB output format is explicitly requested, copyTo() calls Convert to RGB frame, which specifies conversion of color space and pixel format, but does not explicitly define alpha association either.

This also matters for frames constructed from canvases and ImageBitmaps. The image-source constructor copies or references their bitmap data. HTML requires 2D canvas output bitmaps to use premultiplied alpha; ImageBitmap's premultiplyAlpha processing allows different representations. WebCodecs needs to define the observable buffer input/output semantics across these sources.

Measurements on Windows 11, rerun on 2026-10-12 with fresh profiles in headless mode:

  • Chrome 154.0.8037.99, with default graphics flags and with --disable-gpu.
  • Firefox Nightly 160.0a1, build ID 20261011213519.

The probe uses a 2x2 RGBA buffer whose pixels are (200, 100, 50, 128), and a 2x2 2D OffscreenCanvas filled with the corresponding CSS color. Each readback below is the first pixel. The Chrome columns identify launch settings; the measurements alone do not identify every internal graphics path.

(async () => {
  function readBack(source) {
    const canvas = new OffscreenCanvas(2, 2);
    const ctx = canvas.getContext("2d");
    ctx.drawImage(source, 0, 0);
    return Array.from(ctx.getImageData(0, 0, 1, 1).data);
  }

  async function copyBytes(frame, options) {
    const out = new Uint8Array(frame.allocationSize(options));
    await frame.copyTo(out, options);
    return Array.from(out.subarray(0, 4));
  }

  const data = new Uint8Array(16);
  for (let i = 0; i < data.length; i += 4) {
    data.set([200, 100, 50, 128], i);
  }
  const bufferFrame = new VideoFrame(data, {
    format: "RGBA", codedWidth: 2, codedHeight: 2, timestamp: 0,
  });
  const canvas = new OffscreenCanvas(2, 2);
  const ctx = canvas.getContext("2d");
  ctx.fillStyle = `rgba(200, 100, 50, ${128 / 255})`;
  ctx.fillRect(0, 0, 2, 2);
  const canvasFrame = new VideoFrame(canvas, {timestamp: 0});
  try {
    console.log({
      ua: navigator.userAgent,
      bufferFrame_drawImage: readBack(bufferFrame),
      bufferFrame_copyTo: await copyBytes(bufferFrame),
      canvas_getImageData: Array.from(ctx.getImageData(0, 0, 1, 1).data),
      canvasFrame_format: canvasFrame.format,
      canvasFrame_drawImage: readBack(canvasFrame),
      canvasFrame_copyTo: await copyBytes(canvasFrame),
      canvasFrame_copyTo_RGBA: await copyBytes(canvasFrame, {format: "RGBA"}),
    });
  } finally {
    bufferFrame.close();
    canvasFrame.close();
  }
})();
Observation Chrome, default graphics flags Chrome, --disable-gpu Firefox Nightly
buffer frame, drawImage + getImageData (RGBA) 199,100,50,128 199,100,50,128 142,199,100,128
buffer frame, copyTo(out) (RGBA) 200,100,50,128 200,100,50,128 200,100,50,128
source canvas, getImageData (RGBA) 199,100,50,128 199,100,50,128 199,100,50,128
canvas frame, format RGBA BGRA BGRA
canvas frame, copyTo(out) RGBA: 199,100,50,128 BGRA: 25,50,100,128 BGRA: 25,50,100,128
canvas frame, copyTo(out, {format: "RGBA"}) 199,100,50,128 100,50,26,128 100,50,25,128
canvas frame, drawImage + getImageData (RGBA) 199,100,50,128 100,50,26,128 199,100,50,128

These observations are consistent with straight-alpha interpretation of the buffer in Chrome, and premultiplied-alpha interpretation when drawing the buffer in Firefox. For canvas frames, default-format copyTo() returns values consistent with straight alpha in the default Chrome run and premultiplied alpha in the other two runs. With --disable-gpu, Chrome also draws the canvas frame with roughly halved RGB values, consistent with interpreting premultiplied samples as straight. These are interpretations of this probe's output, not evidence that all frames follow the same internal path. Small rounding differences are separate from the much larger color errors.

The Firefox buffer-drawing issue is tracked in bug 2079836. D332178, currently awaiting review, changes RGBA/BGRA buffer handling to preserve straight-alpha colors when drawing. The existing local build tested during this audit (160.0a1, build ID 20261009205724) gives (201, 101, 51, 128) for the buffer drawing row and retains the canvas copyTo() values shown above. This local-build observation is not a controlled before/after test of that patch.

Proposal: explicitly define RGBA/BGRA buffer input and copyTo() output as non-premultiplied (straight) alpha, including output from Convert to RGB frame and from frames created with canvas or ImageBitmap sources. Specify the necessary conversions at observable API boundaries while allowing implementations to retain premultiplied resources internally. The clarification should address conversion precision and zero-alpha pixels. Alpha association should also be made explicit for the YUVA formats (I420A, I422A, I444A, and their higher-bit-depth variants); this RGB probe does not test them.

Relevant prior discussion and related API behavior:

  • WebCodecs issue #508 explicitly argues that RGBA/BGRA copyTo() output should be unpremultiplied, while internal premultiplication should not affect consumers. It also discusses createImageBitmap() for callers requesting premultiplication. This supports the proposed semantics, but is discussion rather than a normative WebCodecs requirement.
  • WebGPU's importExternalTexture() algorithm specifies conversion of the source image to the requested color space "with unpremultiplied alpha". This specifies the imported texture's values, not the representation returned by VideoFrame.copyTo().
  • The naming discussion in issue #200 uses ImageData as the model for RGBA byte order. That comment does not itself specify alpha association.

The upstream webcodecs/videoFrame-alpha.any.js checked for this report tests opaque colors and transparent black. Those particular pixels do not distinguish the two alpha representations. D332178 adds translucent RGBA/BGRA buffer tests for drawing and related consumers, but does not add the translucent canvas-to-copyTo() checks illustrated here. Those cases would also be useful regression tests once the required semantics are agreed.

Ngôn ngữ chính
HTML
Star
1.3k
Fork
192
Merge trung bình
21 giờ 7 phút
Pull request đã merge (30 ngày)
8

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của w3c/webcodecs

Tất cả issue của w3c/webcodecs

Issue tương tự

Thêm issue về Backend & API Design

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.