`kaldi.fbank` does not work with non-contiguous input when `snip_edges=False`
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 64/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- python
- Domain
- audio-video-rtc
Research direction
Inspect torchaudio/compliance/kaldi.py, especially _get_strided(), and run the provided Python reproducer with contiguous and non-contiguous inputs. Verify that fbank(..., snip_edges=False) produces matching results for both inputs without the as_strided bounds error.
Written by the indexing model from the issue text.
Description
🐛 Describe the bug
from torchaudio.compliance.kaldi import fbank
import torch
x = torch.rand(1, 16_000 * 2) * (1 << 15)
x = x[:, ::2]
torch.testing.assert_close(fbank(x.contiguous(), snip_edges=False), fbank(x, snip_edges=False))
File ~/miniconda3/envs/vas_2.4/lib/python3.10/site-packages/torchaudio/compliance/kaldi.py:177, in _get_window(waveform, padded_window_size, window_size, window_shift, window_type, blackman_coeff, snip_edges, raw_energy, energy_floor, dither, remove_dc_offset, preemphasis_coefficient)
174 epsilon = _get_epsilon(device, dtype)
176 # size (m, window_size)
--> 177 strided_input = _get_strided(waveform, window_size, window_shift, snip_edges)
179 if dither != 0.0:
180 rand_gauss = torch.randn(strided_input.shape, device=device, dtype=dtype)
File ~/miniconda3/envs/vas_2.4/lib/python3.10/site-packages/torchaudio/compliance/kaldi.py:83, in _get_strided(waveform, window_size, window_shift, snip_edges)
80 waveform = torch.cat((waveform[-pad:], pad_right), dim=0)
82 sizes = (m, window_size)
---> 83 return waveform.as_strided(sizes, strides)
RuntimeError: setStorage: sizes [100, 400], strides [320, 2], storage offset 0, and itemsize 4 requiring a storage size of 129916 are out of bounds for storage of size 128480
I encountered this problem while implementing a batched version of kaldi.fbank (btw I'm also willing to contribute my batch support back to torchaudio if the maintainers are interested). The problem lies in _get_strided() function. It first obtains the stride of original waveform
However, when snip_edges=False, there is a copy via torch.cat(), which forces waveform to be contiguous, if it was not originally so
Hence, there is a mismatch between the original stride (before padding) and the new stride (padding).
The solution is to move the stride calculation line after padding
# padding...
strides = (window_shift * waveform.stride(0), waveform.stride(0))
sizes = (m, window_size)
return waveform.as_strided(sizes, strides)
Versions
PyTorch 2.4, torchaudio 2.4
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 799
- Avg merge
- 58m
- Merged PRs (30d)
- 3
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from pytorch/audio
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 1/5 Under an hour Newbie friendliness 65/100
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
-
transforms.Vad silently returns an empty tensor when the waveform contains one non-finite sample Open
Difficulty 3/5 1-2 days Newbie friendliness 70/100
-
rnnt_loss: per-sequence logits offset is computed in int and overflows for large-vocabulary batches Open
Difficulty 3/5 1-2 days Newbie friendliness 68/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100