Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[Bug]: Time-stretched proxies lag by the Signalsmith latency and go silent at every loop pass

Open
#474 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
22/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
cpp

Research direction

Start with StretchSegment and ProxyRenderingInfo::render in modules/tracktion_engine/model/clips/tracktion_AudioClipBase.cpp, then SignalsmithStretcher in modules/tracktion_engine/timestretch/tracktion_TimeStretch.cpp for the latency handling. Reproduce with offline renders of a 120 BPM click or sine loop in a 127.3 BPM edit and measure onset offset and RMS at pass boundaries. Done means onsets land within ±0.5 ms and the seam dip is near 0 dB, with the TestRunner suite still green.

Written by the indexing model from the issue text.

Description

Found while building a loop-based app on Tracktion Engine; measured with offline renders of click and sine loops.

Summary. An auto-tempo clip's time-stretched proxy is ~117 ms behind the beat (loop at 120 BPM in a 127.3 BPM edit; 120 ms even at a stretch ratio of 1). For looped clips, a sustained loop (sine, pad) drops to digital silence for ~100 ms at every pass boundary.

Cause. StretchSegment starts every segment (every loop pass) with a new TimeStretcher and never compensates its latency (inputLatency() / rate + outputLatency(), ~5760 samples at 48 kHz with presetDefault). The 12 ms segment crossfade then fades from the old segment into the new stretcher's latency region, which is silent.

Repro. 48 kHz WAV, 4 beats at 120 BPM with an ACID chunk, containing a 220 Hz sine with a whole number of periods. Insert as an auto-tempo looped clip over 8 bars in a 127.3 BPM edit and render offline. RMS in 5 ms windows around each pass boundary: −390 dB. A click loop rendered the same way has every onset 117 ms late (102–124 ms depending on the ratio).

Proposal (two patches below, our tests attached to them pass and the TestRunner suite is green):

  1. Offline (realtime == false) the Signalsmith wrapper pre-computes its latency with outputSeek() on the first block after a reset, so output sample 0 lines up with input sample 0; getLatencySamples() then returns 0. Only the proxy renderer creates offline Signalsmith stretchers.
  2. ProxyRenderingInfo::render feeds contiguous segments to a single stretcher back to back instead of one per segment, so a loop's end flows into its start as in the source. Used only for Signalsmith modes and gap-free segment lists; everything else keeps the existing path.

Result: onsets within ±0.5 ms of a repitched reference, no level dip at the seams (−0.4 dB, same as unstretched).

Note: repeated passes are then no longer sample-identical (the phase vocoder starts each pass in a different phase). Rendering one pass and repeating it can't work for tonal material: a seamless loop with N periods has N/ratio periods after stretching, so each repeat would jump in phase.

Environment: develop @07e2d674 with its JUCE 8.0.13, macOS arm64, 48 kHz, TRACKTION_ENABLE_TIMESTRETCH_SIGNALSMITH=1. The code in question is unchanged on develop @38d5a275.

Patch: 003-signalsmith-offline-ohne-latenz.patch
diff --git a/modules/tracktion_engine/timestretch/tracktion_TimeStretch.cpp b/modules/tracktion_engine/timestretch/tracktion_TimeStretch.cpp
index 9fe50e2..d1a44b9 100644
--- a/modules/tracktion_engine/timestretch/tracktion_TimeStretch.cpp
+++ b/modules/tracktion_engine/timestretch/tracktion_TimeStretch.cpp
@@ -907,7 +907,10 @@ struct SignalsmithStretcher  : public TimeStretcher::Stretcher
     SignalsmithStretcher (double sourceSampleRate, int samplesPerBlock, int numChannels_,
                           TimeStretcher::Mode mode, bool realtime)
         : numChannels (numChannels_),
-          samplesPerOutputBuffer (samplesPerBlock)
+          samplesPerOutputBuffer (samplesPerBlock),
+          alignOutputToInput (! realtime),
+          needsOutputSeek (! realtime),
+          offsetInputs ((size_t) numChannels_, nullptr)
     {
         CRASH_TRACER
 
@@ -926,6 +929,7 @@ struct SignalsmithStretcher  : public TimeStretcher::Stretcher
         stretch.reset();
         hasDoneFinalBlock = false;
         inputRemainder = 0.0;
+        needsOutputSeek = alignOutputToInput;
     }
 
     bool setSpeedAndPitch (float speedRatio_, float semitonesUp) override
@@ -940,7 +944,8 @@ struct SignalsmithStretcher  : public TimeStretcher::Stretcher
         // The fraction of an input sample left over from earlier blocks is carried forward. Rounding
         // each block on its own lets the input run ahead of or behind the output by up to half a
         // sample per block, e.g. 0.43 samples per 256 at 120 -> 127.3 BPM, about 80 samples a second.
-        return (int) std::floor (inputRemainder + exactFramesNeeded() + 0.5);
+        return (int) std::floor (inputRemainder + exactFramesNeeded() + 0.5)
+                + (needsOutputSeek ? outputSeekLength() : 0);
     }
 
     int getMaxFramesNeeded() const override
@@ -951,6 +956,25 @@ struct SignalsmithStretcher  : public TimeStretcher::Stretcher
     int processData (const float* const* inChannels, int numSamples, float* const* outChannels) override
     {
         CRASH_TRACER
+
+        if (needsOutputSeek)
+        {
+            // Offline (e.g. proxy rendering), the first block after a reset pre-computes the
+            // stretcher's latency with Signalsmith's outputSeek(), so the first output sample lines
+            // up with the first input sample. Without this the output lags by inputLatency / rate +
+            // outputLatency, about 120 ms at 48 kHz with the default preset.
+            const int seekLength = outputSeekLength();
+            jassert (numSamples >= seekLength);
+            stretch.outputSeek (inChannels, seekLength);
+
+            for (int i = 0; i < numChannels; ++i)
+                offsetInputs[(size_t) i] = inChannels[i] + seekLength;
+
+            inChannels = offsetInputs.data();
+            numSamples -= seekLength;
+            needsOutputSeek = false;
+        }
+
         stretch.process (inChannels, numSamples, outChannels, samplesPerOutputBuffer);
         inputRemainder += exactFramesNeeded() - numSamples;
         return samplesPerOutputBuffer;
@@ -968,7 +992,8 @@ struct SignalsmithStretcher  : public TimeStretcher::Stretcher
 
     int getLatencySamples() const override
     {
-        return stretch.inputLatency() + stretch.outputLatency();
+        // Offline the output is aligned to the input (see processData)
+        return alignOutputToInput ? 0 : stretch.inputLatency() + stretch.outputLatency();
     }
 
 private:
@@ -978,8 +1003,12 @@ private:
     float speedRatio = 1.0f;
     double inputRemainder = 0.0;
     bool hasDoneFinalBlock = false;
+    const bool alignOutputToInput;
+    bool needsOutputSeek;
+    std::vector<const float*> offsetInputs;
 
     double exactFramesNeeded() const        { return samplesPerOutputBuffer / (double) speedRatio; }
+    int outputSeekLength() const            { return stretch.outputSeekLength (1.0f / speedRatio); }
 
     JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (SignalsmithStretcher)
 };
Patch: 004-proxy-durchgehend-strecken.patch
diff --git a/modules/tracktion_engine/model/clips/tracktion_AudioClipBase.cpp b/modules/tracktion_engine/model/clips/tracktion_AudioClipBase.cpp
index 6e8964e..c4bc051 100644
--- a/modules/tracktion_engine/model/clips/tracktion_AudioClipBase.cpp
+++ b/modules/tracktion_engine/model/clips/tracktion_AudioClipBase.cpp
@@ -2310,6 +2310,204 @@ struct StretchSegment
     JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (StretchSegment)
 };
 
+//==============================================================================
+/** Stretches all segments of a clip with one stretcher, one after the other.
+
+    The per-segment rendering above starts every segment (e.g. every pass of a looped clip) with a
+    fresh stretcher and cross-fades into it. Each fresh stretcher only produces sound after its
+    latency, so a sustained loop dropped to silence at every pass for about 100 ms. Here the
+    segments' source ranges are fed to a single stretcher back to back,
+    so a loop's end flows into its start like it does in the source file. Only used when the
+    segments follow each other without gaps and the stretcher aligns its output to its input
+    offline (Signalsmith, see SignalsmithStretcher::processData).
+*/
+struct ContinuousStretch
+{
+    struct Piece
+    {
+        SampleCount sourceStart = 0, sourceLength = 0;   // in the source file
+        SampleCount outputStart = 0, outputEnd = 0;      // in the proxy
+        float transpose = 0.0f;
+    };
+
+    static bool canRender (const AudioSegmentList& list, TimeStretcher::Mode mode)
+    {
+        if (mode != TimeStretcher::signalsmithDefault && mode != TimeStretcher::signalsmithCheaper)
+            return false;
+
+        auto& segments = list.getSegments();
+
+        if (segments.isEmpty() || segments.getReference (0).getRange().getStart() != TimePosition())
+            return false;
+
+        // Every segment but the last must run into the next one (crossFadeSegments() marks those
+        // with a fade out); a gap means silence in between and needs the per-segment rendering
+        for (int i = 0; i < segments.size() - 1; ++i)
+            if (! segments.getReference (i).hasFadeOut())
+                return false;
+
+        return true;
+    }
+
+    static std::vector<Piece> createPieces (const AudioSegmentList& list, double sampleRate)
+    {
+        std::vector<Piece> pieces;
+        auto& segments = list.getSegments();
+        const auto crossfade = list.getCrossfadeLength();
+
+        for (int i = 0; i < segments.size(); ++i)
+        {
+            auto& seg = segments.getReference (i);
+            auto range = seg.getRange();
+
+            // crossFadeSegments() lengthened the segment (and its source range) by the cross-fade
+            auto ownLength = seg.hasFadeOut() ? range.getLength() - crossfade : range.getLength();
+            auto sourceLength = seg.getSampleRange().getLength();
+
+            if (seg.hasFadeOut() && range.getLength() > TimeDuration())
+                sourceLength = (SampleCount) std::llround ((double) sourceLength * (ownLength / range.getLength()));
+
+            Piece p;
+            p.sourceStart = seg.getSampleRange().getStart();
+            p.sourceLength = std::max ((SampleCount) 0, sourceLength);
+            p.outputStart = toSamples (range.getStart(), sampleRate);
+            p.outputEnd = toSamples (range.getStart() + ownLength, sampleRate);
+            p.transpose = seg.getTranspose();
+
+            if (! pieces.empty())
+                p.outputStart = pieces.back().outputEnd;   // contiguous by construction
+
+            if (p.outputEnd > p.outputStart && p.sourceLength > 0)
+                pieces.push_back (p);
+        }
+
+        return pieces;
+    }
+
+    ContinuousStretch (Engine& engine, const AudioFile& file, const AudioClipBase::ProxyRenderingInfo& info,
+                       std::vector<Piece> piecesToUse, int blockSize)
+        : pieces (std::move (piecesToUse)),
+          numChannels (juce::jlimit (1, maxNumChannels, file.getInfo().numChannels)),
+          outputBlockSize (blockSize)
+    {
+        reader = engine.getAudioFileManager().cache.createReader (file);
+
+        if (reader != nullptr)
+        {
+            timestretcher.initialise (file.getSampleRate(), outputBlockSize, numChannels, info.mode, info.options, false);
+            jassert (timestretcher.isInitialised());
+            output.setSize (numChannels, outputBlockSize);
+        }
+    }
+
+    bool isValid() const    { return reader != nullptr && timestretcher.isInitialised() && ! pieces.empty(); }
+
+    /** Renders the next block (blockSize samples) of the proxy. */
+    void renderNextBlock (juce::AudioBuffer<float>& dest)
+    {
+        CRASH_TRACER
+        dest.clear();
+
+        const auto blockStart = outputPosition;
+        const auto blockEnd = outputPosition + outputBlockSize;
+        outputPosition = blockEnd;
+
+        if (blockStart >= pieces.back().outputEnd)
+            return;
+
+        // Input for this block: each piece contributes its share at its own ratio
+        double inputForBlock = 0.0;
+        float transpose = pieces.front().transpose;
+
+        for (auto& p : pieces)
+        {
+            const auto overlap = std::min (blockEnd, p.outputEnd) - std::max (blockStart, p.outputStart);
+
+            if (overlap > 0)
+                inputForBlock += (double) overlap * (double) p.sourceLength / (double) (p.outputEnd - p.outputStart);
+
+            if (p.outputStart <= blockStart && blockStart < p.outputEnd)
+                transpose = p.transpose;
+        }
+
+        // Past the end of the last piece the ratio of the last piece continues (the clip ends there)
+        if (blockEnd > pieces.back().outputEnd)
+        {
+            auto& last = pieces.back();
+            inputForBlock += (double) (blockEnd - std::max (blockStart, last.outputEnd))
+                               * (double) last.sourceLength / (double) (last.outputEnd - last.outputStart);
+        }
+
+        timestretcher.setSpeedAndPitch ((float) (outputBlockSize / std::max (inputForBlock, 1.0e-6)), transpose);
+
+        const int needed = timestretcher.getFramesNeeded();
+        AudioScratchBuffer scratch (numChannels, std::max (1, needed));
+        scratch.buffer.clear();
+        readInput (scratch.buffer, needed);
+
+        const float* ins[maxNumChannels] = {};
+        float* outs[maxNumChannels] = {};
+
+        for (int i = 0; i < numChannels; ++i)
+        {
+            ins[i] = scratch.buffer.getReadPointer (i);
+            outs[i] = output.getWritePointer (i);
+        }
+
+        const int numDone = timestretcher.processData (ins, needed, outs);
+        jassert (numDone == outputBlockSize);
+
+        // Silence after the end of the clip
+        const auto numToCopy = (int) std::min ((SampleCount) numDone, pieces.back().outputEnd - blockStart);
+
+        for (int i = 0; i < dest.getNumChannels(); ++i)
+            dest.copyFrom (i, 0, output, std::min (i, numChannels - 1), 0, numToCopy);
+    }
+
+private:
+    static constexpr int maxNumChannels = 8;
+
+    std::vector<Piece> pieces;
+    const int numChannels, outputBlockSize;
+    AudioFileCache::Reader::Ptr reader;
+    TimeStretcher timestretcher;
+    juce::AudioBuffer<float> output;
+    SampleCount outputPosition = 0;
+    size_t inputPiece = 0;
+    SampleCount inputOffset = 0;   // within inputPiece, may run past the end of the last piece
+
+    /** The pieces' source ranges back to back; after the last piece the file just continues. */
+    void readInput (juce::AudioBuffer<float>& buffer, int numSamples)
+    {
+        const auto channels = ChannelConfiguration::canonical (numChannels);
+        int done = 0;
+
+        while (done < numSamples)
+        {
+            auto& p = pieces[inputPiece];
+            const bool isLast = inputPiece == pieces.size() - 1;
+            const auto available = isLast ? (SampleCount) (numSamples - done) : p.sourceLength - inputOffset;
+            const auto numThisTime = (int) std::min ((SampleCount) (numSamples - done), available);
+
+            if (numThisTime > 0)
+            {
+                reader->setReadPosition (p.sourceStart + inputOffset);
+                reader->readSamples (numThisTime, buffer, channels, done, channels, 5000);
+                done += numThisTime;
+                inputOffset += numThisTime;
+            }
+
+            if (! isLast && inputOffset >= p.sourceLength)
+            {
+                ++inputPiece;
+                inputOffset = 0;
+            }
+        }
+    }
+
+    JUCE_DECLARE_NON_COPYABLE_WITH_LEAK_DETECTOR (ContinuousStretch)
+};
+
 //==============================================================================
 std::unique_ptr<AudioClipBase::ProxyRenderingInfo> AudioClipBase::createProxyRenderingInfo()
 {
@@ -2333,19 +2531,41 @@ bool AudioClipBase::ProxyRenderingInfo::render (Engine& engine, const AudioFile&
     if (audioSegmentList->getSegments().isEmpty() || ! sourceFile.isValid())
         return false;
 
-    juce::OwnedArray<StretchSegment> segments;
-
     auto sampleRate = sourceFile.getSampleRate();
+    const int samplesPerBlock = 1024;
+    juce::AudioBuffer<float> buffer (sourceFile.getNumChannels(), samplesPerBlock);
+    auto numBlocks = 1 + (int) (clipTime.getLength().inSeconds() * sampleRate / samplesPerBlock);
+
+    if (ContinuousStretch::canRender (*audioSegmentList, mode))
+    {
+        ContinuousStretch stretch (engine, sourceFile, *this, ContinuousStretch::createPieces (*audioSegmentList, sampleRate), samplesPerBlock);
+
+        if (stretch.isValid())
+        {
+            for (int i = 0; i < numBlocks; ++i)
+            {
+                if (job != nullptr && job->shouldExit())
+                    return false;
+
+                stretch.renderNextBlock (buffer);
+
+                if (! writer.appendBuffer (buffer, samplesPerBlock))
+                    return false;
+
+                progress = static_cast<float> (i) / (float) numBlocks;
+            }
+
+            return true;
+        }
+    }
+
+    juce::OwnedArray<StretchSegment> segments;
 
     for (auto& segment : audioSegmentList->getSegments())
         segments.add (new StretchSegment (engine, sourceFile, *this, sampleRate, segment));
 
-    const int samplesPerBlock = 1024;
-    juce::AudioBuffer<float> buffer (sourceFile.getNumChannels(), samplesPerBlock);
     double time = 0.0;
 
-    auto numBlocks = 1 + (int) (clipTime.getLength().inSeconds() * sampleRate / samplesPerBlock);
-
     for (int i = 0; i < numBlocks; ++i)
     {
         if (job != nullptr && job->shouldExit())

(003 applies on top of the patch in #473, which touches the same struct.)

Dominant language
C++
Stars
1.5k
Forks
222
Avg merge
15h 44m
Merged PRs (30d)
33

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Tracktion/tracktion_engine

All issues in Tracktion/tracktion_engine

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.