Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[BUG] Inactive layer1x1 drops the activation from the WaveNet residual path

Open
  • Is something wrong?
#341 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@andrenepomuceno is already working on this.

Since Sep 27, 2026.

  • #343 by @andrenepomuceno — open

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
cpp
Domain
machine-learning

Research direction

Start in NAM/wavenet/model.cpp at _Layer::Process and inspect the inactive-layer1x1 branch in both the Eigen and NAM_USE_INLINE_GEMM paths. Compare its residual handling with nam/models/wavenet/_layer_array.py, lines 755–773, where the trainer adds post_activation to x. Done means the core output matches the trainer for an inactive layer1x1 model, including the reported residual-path comparison.

Written by the indexing model from the issue text.

Description

Summary

When a WaveNet layer has "layer1x1": {"active": false} (.nam 0.7 format), the core passes the residual input through unchanged instead of adding the layer's activation output to it. The trainer (neural-amp-modeler) computes x + post_activation in that case, so the core's output diverges from the trainer's for such models.

Where

  • Core (main @ 0b3d3c97b0, also v0.5.4): NAM/wavenet/model.cpp, _Layer::Process… inactive-layer1x1 branch (around line 385): 
    // If layer1x1 is inactive, residual connection is just the input (identity)
...memcpy(... input.data() ...)            // NAM_USE_INLINE_GEMM path
this->_output_next_layer.leftCols(num_frames).noalias() = input.leftCols(num_frames);  // Eigen path
  • Trainer (main @ 0072676419): nam/models/wavenet/_layer_array.py, lines ~755–773: 
    layer_output = post_activation
if self._layer1x1 is not None:
    layer_output = self._layer1x1(layer_output)
    ...
residual = x[:, :, -layer_output.shape[2]:] + layer_output
    i.e. with layer1x1 inactive the residual is x + post_activation, not x.

Impact

On a model exported by neural-amp-modeler 0.13.0 with layer1x1 inactive on one layer array, the core's output differed from the trainer's PyTorch output (pad_start=True) by up to ~0.4 on that array and ~1.9e-3 at the model output. With the residual add restored, the core matches the trainer to ~6e-8.

Suggested fix

In the inactive branch, add the activation output (the bottleneck rows of _z) to the input instead of copying it, in both the Eigen path and the NAM_USE_INLINE_GEMM path, e.g. for the Eigen path:

this->_output_next_layer.leftCols(num_frames).noalias() =
  input.leftCols(num_frames) + this->_z.topRows(bottleneck).leftCols(num_frames);

(Assuming bottleneck == channels is required when layer1x1 is inactive, as the residual shapes imply.)

Dominant language
C++
Stars
969
Forks
185
Avg merge
36m
Merged PRs (30d)
4

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from sdatkinson/NeuralAmpModelerCore

All issues in sdatkinson/NeuralAmpModelerCore

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.