Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Specify alpha association for VideoFrame buffer input and copyTo() output

Abierto
  • ¿Algo no está bien?
#972 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
html, javascript
Área
api, web-dev

Línea de trabajo

Read the WebCodecs sections linked in the issue for the VideoFrame buffer and image source constructors, copyTo(), and Convert to RGB frame. Compare their alpha association rules with the cited HTML canvas and ImageBitmap requirements; the issue proposes defining observable behavior, conversion precision, zero-alpha pixels, and YUVA semantics. Done means the specification resolves those cases and adds distinguishing translucent regression tests.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

The WebCodecs Editor's Draft checked on 2026-10-12 defines channel order and sample sizes for RGBA and BGRA, but does not explicitly say whether their RGB samples are premultiplied by alpha. The buffer constructor and copyTo() copy pixel data without specifying an alpha-association conversion. When an RGB output format is explicitly requested, copyTo() calls Convert to RGB frame, which specifies conversion of color space and pixel format, but does not explicitly define alpha association either.

This also matters for frames constructed from canvases and ImageBitmaps. The image-source constructor copies or references their bitmap data. HTML requires 2D canvas output bitmaps to use premultiplied alpha; ImageBitmap's premultiplyAlpha processing allows different representations. WebCodecs needs to define the observable buffer input/output semantics across these sources.

Measurements on Windows 11, rerun on 2026-10-12 with fresh profiles in headless mode:

  • Chrome 154.0.8037.99, with default graphics flags and with --disable-gpu.
  • Firefox Nightly 160.0a1, build ID 20261011213519.

The probe uses a 2x2 RGBA buffer whose pixels are (200, 100, 50, 128), and a 2x2 2D OffscreenCanvas filled with the corresponding CSS color. Each readback below is the first pixel. The Chrome columns identify launch settings; the measurements alone do not identify every internal graphics path.

(async () => {
  function readBack(source) {
    const canvas = new OffscreenCanvas(2, 2);
    const ctx = canvas.getContext("2d");
    ctx.drawImage(source, 0, 0);
    return Array.from(ctx.getImageData(0, 0, 1, 1).data);
  }

  async function copyBytes(frame, options) {
    const out = new Uint8Array(frame.allocationSize(options));
    await frame.copyTo(out, options);
    return Array.from(out.subarray(0, 4));
  }

  const data = new Uint8Array(16);
  for (let i = 0; i < data.length; i += 4) {
    data.set([200, 100, 50, 128], i);
  }
  const bufferFrame = new VideoFrame(data, {
    format: "RGBA", codedWidth: 2, codedHeight: 2, timestamp: 0,
  });
  const canvas = new OffscreenCanvas(2, 2);
  const ctx = canvas.getContext("2d");
  ctx.fillStyle = `rgba(200, 100, 50, ${128 / 255})`;
  ctx.fillRect(0, 0, 2, 2);
  const canvasFrame = new VideoFrame(canvas, {timestamp: 0});
  try {
    console.log({
      ua: navigator.userAgent,
      bufferFrame_drawImage: readBack(bufferFrame),
      bufferFrame_copyTo: await copyBytes(bufferFrame),
      canvas_getImageData: Array.from(ctx.getImageData(0, 0, 1, 1).data),
      canvasFrame_format: canvasFrame.format,
      canvasFrame_drawImage: readBack(canvasFrame),
      canvasFrame_copyTo: await copyBytes(canvasFrame),
      canvasFrame_copyTo_RGBA: await copyBytes(canvasFrame, {format: "RGBA"}),
    });
  } finally {
    bufferFrame.close();
    canvasFrame.close();
  }
})();
Observation Chrome, default graphics flags Chrome, --disable-gpu Firefox Nightly
buffer frame, drawImage + getImageData (RGBA) 199,100,50,128 199,100,50,128 142,199,100,128
buffer frame, copyTo(out) (RGBA) 200,100,50,128 200,100,50,128 200,100,50,128
source canvas, getImageData (RGBA) 199,100,50,128 199,100,50,128 199,100,50,128
canvas frame, format RGBA BGRA BGRA
canvas frame, copyTo(out) RGBA: 199,100,50,128 BGRA: 25,50,100,128 BGRA: 25,50,100,128
canvas frame, copyTo(out, {format: "RGBA"}) 199,100,50,128 100,50,26,128 100,50,25,128
canvas frame, drawImage + getImageData (RGBA) 199,100,50,128 100,50,26,128 199,100,50,128

These observations are consistent with straight-alpha interpretation of the buffer in Chrome, and premultiplied-alpha interpretation when drawing the buffer in Firefox. For canvas frames, default-format copyTo() returns values consistent with straight alpha in the default Chrome run and premultiplied alpha in the other two runs. With --disable-gpu, Chrome also draws the canvas frame with roughly halved RGB values, consistent with interpreting premultiplied samples as straight. These are interpretations of this probe's output, not evidence that all frames follow the same internal path. Small rounding differences are separate from the much larger color errors.

The Firefox buffer-drawing issue is tracked in bug 2079836. D332178, currently awaiting review, changes RGBA/BGRA buffer handling to preserve straight-alpha colors when drawing. The existing local build tested during this audit (160.0a1, build ID 20261009205724) gives (201, 101, 51, 128) for the buffer drawing row and retains the canvas copyTo() values shown above. This local-build observation is not a controlled before/after test of that patch.

Proposal: explicitly define RGBA/BGRA buffer input and copyTo() output as non-premultiplied (straight) alpha, including output from Convert to RGB frame and from frames created with canvas or ImageBitmap sources. Specify the necessary conversions at observable API boundaries while allowing implementations to retain premultiplied resources internally. The clarification should address conversion precision and zero-alpha pixels. Alpha association should also be made explicit for the YUVA formats (I420A, I422A, I444A, and their higher-bit-depth variants); this RGB probe does not test them.

Relevant prior discussion and related API behavior:

  • WebCodecs issue #508 explicitly argues that RGBA/BGRA copyTo() output should be unpremultiplied, while internal premultiplication should not affect consumers. It also discusses createImageBitmap() for callers requesting premultiplication. This supports the proposed semantics, but is discussion rather than a normative WebCodecs requirement.
  • WebGPU's importExternalTexture() algorithm specifies conversion of the source image to the requested color space "with unpremultiplied alpha". This specifies the imported texture's values, not the representation returned by VideoFrame.copyTo().
  • The naming discussion in issue #200 uses ImageData as the model for RGBA byte order. That comment does not itself specify alpha association.

The upstream webcodecs/videoFrame-alpha.any.js checked for this report tests opaque colors and transparent black. Those particular pixels do not distinguish the two alpha representations. D332178 adds translucent RGBA/BGRA buffer tests for drawing and related consumers, but does not add the translucent canvas-to-copyTo() checks illustrated here. Those cases would also be useful regression tests once the required semantics are agreed.

Lenguaje dominante
HTML
Estrellas
1.3k
Forks
192
Merge medio
21 h 7 min
PR fusionados (30 d)
8

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de w3c/webcodecs

Todos los issues de w3c/webcodecs

Issues similares

Más issues de Backend & API Design

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.