Audio channel ordering is underspecified: "same as in the EncodedAudioChunk" is ambiguous for AAC, and layout information has no path through the API

Start with the WebCodecs Audio channel ordering section and the AudioDecoderConfig, AudioEncoderConfig, and AudioData definitions; compare them with Web Audio’s Channel Ordering requirements and the linked WPT PR. The issue proposes changes to channel ordering and possibly API fields for layout, but leaves the choice among proposals open. Done means reaching agreement on the required behavior and updating the specification and interoperability tests accordingly.

Problem

§ Audio channel ordering states:

When decoding, the ordering of the audio channels in the resulting AudioData MUST be the same as what is present in the EncodedAudioChunk. [...] In other terms, no channel reordering is performed when encoding and decoding.

NOTE: The container either implies or specifies the channel mapping: the channel attributed to a particular channel index.

This doesn't give a deterministic, interoperable result for common content, in particular AAC in MP4.

1. "Order present in the EncodedAudioChunk" is not well-defined for AAC

For AAC, the bitstream order of syntactic elements is not the decoder's output order. With channelConfiguration = 6 (5.1), the elements appear as SCE (C), CPE (L, R), CPE (Ls, Rs), LFE, so the "chunk order" is C, L, R, Ls, Rs, LFE.

Commonly used decoders differ in output order. Some reorder to WAVE/SMPTE order (L, R, C, LFE, Ls, Rs) such as Windows or FFmpeg, others output AAC-native order (Apple's AudioToolbox). Both readings arguably conform to the current text. The same file decoded in two conforming implementations can therefore produce different channel orders in AudioData.

The same ambiguity applies to Opus with channel mapping family 1 (Vorbis order) and to Vorbis. The spec doesn't say whether "the order in the chunk" means the coded stream order or the order the codec's own specification defines for output.

2. The NOTE is contradicted by the API: container information cannot reach the codec

The section ends with:

NOTE: The container either implies or specifies the channel mapping: the channel attributed to a particular channel index.

This puts the channel mapping in the container, but WebCodecs never sees the container. The only inputs a decoder receives are:

• AudioDecoderConfig: codec, sampleRate, numberOfChannels, and description. description is codec-level configuration such as the AAC AudioSpecificConfig, OpusHead or FLAC STREAMINFO, not container metadata.
• EncodedAudioChunk: type, timestamp, duration, and the coded payload.

Neither has a field for a container-level layout. Concretely:

• AAC in ISOBMFF: a chnl box (ISO/IEC 14496-12 / 23001-8) or a QuickTime chan box in the sample entry has nowhere to go.
• PCM in ISOBMFF (ipcm/fpcm, ISO/IEC 23003-5): the chnl box is the only layout information, and the PCM registration accepts no description.
• Encoding: AudioEncoderConfig has no layout field and AudioData carries none, so the encoder can't produce output a muxer could write a correct chnl/chan box from. The produced EncodedAudioChunkMetadata.decoderConfig carries no layout either.

The NOTE can only be read as "the application must interpret channel indices itself, using container data it parsed". That fails too. Per section 1, the application can't know whether the decoder output follows bitstream order, the codec's canonical order, or a reordered one, so it has no reliable way to apply the container's mapping to AudioData channel indices.

The NOTE therefore describes a mechanism that doesn't exist in the API. It should either be removed, or backed by an actual way to carry layout information (see proposal 2).

3. AudioData exposes no channel layout

AudioData only exposes numberOfChannels. Because of (1), an application can't determine which speaker a given channel index represents, even if it parsed the container itself. It doesn't know whether the decoder reordered.

Impact

An application that decodes 5.1 AAC from MP4 and renders it through Web Audio, which defines a specific 5.1 order (L, R, C, LFE, SL, SR), cannot write portable code. It must guess or sniff the user agent. The same applies to encoding: there is no way to tell an AudioEncoder which layout the input AudioData uses, so the encoder can't signal it correctly in the bitstream (e.g. AAC channelConfiguration or a PCE, Opus mapping table).

Proposals (not mutually exclusive)

1. Adopt the Web Audio channel ordering. Web Audio § Channel Ordering already defines a normative order (SPEAKER_FRONT_LEFT, SPEAKER_FRONT_RIGHT, SPEAKER_FRONT_CENTER, SPEAKER_LOW_FREQUENCY, SPEAKER_BACK_LEFT, ...) and requires implementations to "present the channels provided in the order defined below, skipping over those channels not present". User agents already apply this when decoding the same codecs via decodeAudioData(), so the reordering logic already exists in implementations.

   WebCodecs should require:

   • Decoding: when the codec configuration signals a speaker layout (e.g. AAC channelConfiguration or a PCE with defined positions, Opus mapping family 1, Vorbis, FLAC), AudioData channels MUST be output in Web Audio channel order.
   • Encoding: input AudioData channels are interpreted in Web Audio channel order, and the encoder MUST map them to the codec's own order and signalling.
   • Discrete layouts: when no speaker layout is signalled (e.g. Opus mapping families 2 and 255, PCM without layout information), channels are passed through in coded order with no speaker semantics.

2. Add an optional channel layout to AudioDecoderConfig and AudioEncoderConfig, so container-level layout information (e.g. chnl) can be supplied and encoders can signal it.

3. Expose the channel layout on AudioData, so consumers can interpret channel indices without codec- or UA-specific knowledge.

Interop data

WPT PR: https://github.com/web-platform-tests/wpt/pull/64040

The test decodes 5.1 streams in which each channel carries a distinct tone, then identifies the speaker of each output channel.

	AAC + AudioSpecificConfig (MP4)	AAC (ADTS)	Opus (mapping family 1)
Chrome	FL FR FC LFE BL BR	FL FR FC LFE BL BR	FL FR FC LFE BL BR
Firefox	FL FR FC LFE BL BR	FL FR FC LFE BL BR	FL FR FC LFE BL BR
Safari	FC FL FR BL BR LFE	FC FL FR BL BR LFE	not supported (Safari doesn't currently decode more than 2 channels of Opus)

Coded channel order in the test streams:

• AAC, channelConfiguration 6: C, L, R, Ls, Rs, LFE
• Opus, channel mapping family 1 (Vorbis order): L, C, R, Ls, Rs, LFE

Chrome and Firefox already reorder the decoded output of both codecs to SMPTE/WAVE order, which is also the Web Audio order. That contradicts the current spec text, which says the order MUST be "the same as what is present in the EncodedAudioChunk" and that "no channel reordering is performed". Safari follows the spec text for AAC and outputs the coded order.

So the only engine that follows the spec produces output that no other engine matches, and that Web Audio consumers can't use without UA-specific remapping. The spec should be changed to match the majority behaviour (proposal 1), not the other way around.

Editorial

In the same section, "as what is preset in the given AudioData" should be "present".