Sonic (16-bit path): `findPitchPeriodInRange` compares in 32-bit ints that overflow on loud audio, so a badly matching pitch period can win

オープン 初心者向け
#3,565 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 4 日以内に返信

まだ誰も着手していません。

評価

難易度
2/5
見積もり時間
1〜3時間
初心者へのやさしさ
78/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
android, java

調査の方向性

libraries/common/src/main/java/androidx/media3/common/audio/Sonic.java、特に1114~1124行付近のSonicShortImpl.findPitchPeriodInRangeを読みます。2つの整数乗算チェックを、issueに示されたlong演算と比較し、速度1.1で大音量の16ビットPCM再現を実行するか、適宜調整します。比較でオーバーフローを回避し、ピッチ周期の選択がlong演算の結果と一致すれば完了です。

索引モデルが issue の本文から書いたものです。

説明

bug needs triage
Version

Media3 1.10.1

More version details

The same code is in Media3 1.11.1 and on main (e765dd45dbf58e67a578ae003ca64eaa449bcc28):

https://github.com/androidx/media/blob/e765dd45dbf58e67a578ae003ca64eaa449bcc28/libraries/common/src/main/java/androidx/media3/common/audio/Sonic.java#L1114-L1124

Devices that reproduce the issue

Every device: the issue is integer arithmetic in Sonic.java.

  • Measured on the JVM, with Media3 1.10.1's own Sonic class (from the media3-common AAR)
  • Recorded on an Android 12 emulator (API 31, x86_64), playing through ExoPlayer at speed 1.1
  • Heard on a Samsung Galaxy Note10+ running Android 12
Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Not tested

Reproduction steps
  1. Take loud 16-bit PCM at 44.1 kHz, stereo: speech mastered to full scale, or full-scale white noise.

  2. Run it through Sonic at speed 1.1, pitch 1 (for example SonicAudioProcessor.setSpeed(1.1f), or an ExoPlayer at setPlaybackSpeed(1.1f)).

  3. Run the same input through a copy of Sonic whose two comparisons in SonicShortImpl.findPitchPeriodInRange are made in long:

    if ((long) diff * bestPeriod < (long) minDiff * period) {
    ...
    if ((long) diff * worstPeriod > (long) maxDiff * period) {
    
  4. Compare the two outputs: they differ. Scale the input down by 64, so that no product can reach 2^31, and they are identical, sample for sample.

The arithmetic alone, with values of the second pass: a poor candidate beats a good match.

int bestPeriod = 610, minDiff = 610 * 1_000; // the best so far: mean |s - p| of 1,000
int period = 600, diff = 600 * 8_000;        // a poor candidate: mean |s - p| of 8,000
diff * bestPeriod < minDiff * period                // true:  -1366967296 < 366000000
(long) diff * bestPeriod < (long) minDiff * period  // false: 2928000000 < 366000000
Expected result

The pitch search keeps the best-matching period at any level, as it does on quieter audio, in SonicFloatImpl (which holds diff, minDiff and maxDiff in doubles), and in the C library it was ported from (waywardgeek/sonic declares them unsigned long).

Suggested fix: the two casts of step 3.

Actual result

In SonicShortImpl.findPitchPeriodInRange the variables diff, minDiff, maxDiff, bestPeriod, worstPeriod and period are ints, so diff * bestPeriod < minDiff * period and diff * worstPeriod > maxDiff * period are evaluated in 32 bits. The comment above them says diff is a 24-bit number "since we skip samples". That holds for the first, down-sampled pass of findPitchPeriod. It does not hold for the second pass, which runs at the full rate (downSampleInput(positionFrames, 1) for more than one channel, the input buffer for mono) over periods of up to maxPeriod, 678 samples at 44.1 kHz. There diff sums up to 678 absolute differences of 16-bit samples, and its product with a period passes 2^31 once the mean absolute difference per sample is above about 2^31 / period² (about 13,400 at a period of 400, 4,700 at 678): any loud passage. A wrapped product decides the comparison, and a period that matches badly can be kept as the best one.

What it does to the sound: between 1x and 2x, Sonic blends the period it found into the next one. With a wrong period the two pieces are unalike, the blend partly cancels, and the level dips for a few milliseconds: a faint, rapid fluttering on a sustained voice.

Measured at speed 1.1 on a speech recording mastered to full scale (44.1 kHz stereo, −13 dBFS RMS, 120 s), 308 of 1,491 splices (21%) take another period once the comparisons are made in long, and the level dips at the splices fall:

Level dips over 3 dB, per second (0.3–2 kHz) as shipped compared in long
the loud recording (−13 dBFS RMS, 120 s) 1.78 0.97
a quieter recording (−20 dBFS RMS, 175 s) 1.49 1.15

A dip here is a run of 5 ms output frames whose level in the 0.3–2 kHz band is more than 3 dB below the best-matching place of the input; 306 of 310 dips over 1 dB lay within 12 ms of a splice.

Media

Any loud 16-bit PCM shows it (full-scale white noise is enough), so no file is attached. The measurements above used two speech recordings decoded to 44.1 kHz stereo 16-bit PCM.

No adb bugreport will follow: the issue is arithmetic and does not depend on the device.

Bug Report
  • You will email the zip file produced by adb bugreport to [email protected] after filing this issue.
主要言語
Java
スター
3.1k
フォーク
1k
平均マージ
6日 4時間
マージ済み PR(30日)
5

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

androidx/media のほかの issue

androidx/media の issue をすべて見る

似ている issue

Java の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。