Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Apertus xIELU formula / param-space mismatch — forward pass NaN's on real Apertus-8B Q4_K_S

Ouverte
  • Quelque chose ne va pas ?
#103 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

Personne n'a encore pris cette issue.

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
30/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Calme
Stack technique
kotlin
Domaine
backend, machine-learning

Piste de recherche

Read ApertusXIELU.kt at line 27 and compare its activation formula with the Hugging Face Apertus modeling code; inspect the layer 0 parameters in the reference checkpoint and compare them with the GGUF values. The issue also points to ApertusWeightLoader.kt, the DSL wiring in apertusNetwork(), and ApertusRealGgufLoadingTest for a forward pass probe. Done means identifying the source of the NaNs and confirming the corrected activation produces finite logits.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Summary

After PR #102 the load → convert → fromWeights → forward chain runs to completion against unsloth/Apertus-8B-Instruct-2509-GGUF (Q4_K_S) — but the logit vector is all-NaN at the first forward pass. The propagation source is ApertusXIELU.xielu(buf, params) in layer 0.

Diagnosis

ApertusXIELU.kt evaluates:

alpha_p_eff = softplus(alpha_p)
alpha_n_eff = beta + softplus(alpha_n)

if x > 0:  alpha_p_eff * x*x + beta * x
else:      (expm1(min(x, eps)) - x) * alpha_n_eff + beta * x

with softplus(x) = if (x > 20) x else ln(1 + exp(x)).

The unsloth GGUF stores per-layer xIELU params (verified via gguf Python lib):

layer alpha_p alpha_n beta eps
0 166.0 40.75 0.5 -9.98e-7
1 174.0 31.625 0.5 -9.98e-7
2 128.0 22.875 0.5 -9.98e-7
… (decaying) (decaying) 0.5 -9.98e-7
11 -0.477 … 0.5 -9.98e-7
… (small / negative) … 0.5 -9.98e-7

For layer 0, softplus(166) ≈ 166 (the > 20 shortcut applies), so the activation collapses to 166·x² + 0.5·x. Any input value |x| past a few causes the FFN output to overflow, and the post-FFN matmul + residual chain NaN-poisons the rest of the forward.

Hypothesis

Either:

  1. The unsloth GGUF stores alpha_p / alpha_n in a different transform space than ApertusXIELU.kt expects. softplus(stored) may already be applied (i.e. these are the post-softplus alpha_p_eff values, not the raw learned parameters). In that case the right thing is to drop the softplus calls in the activation and use the stored values directly — which would still leave layer 0 with alpha_p_eff = 166, and the activation would still overflow for any non-trivial input.
  2. ApertusXIELU.kt's formula diverges from the Apertus-8B reference. Possible candidates:
    • The reference may scale alpha_p by some normalization factor.
    • The if x > 0 branch might use alpha_p_eff * (e^x - x - 1) (an ELU-like form) rather than alpha_p_eff * x².
    • beta, eps, etc. may participate in a different combination.
  3. The unsloth converter is upstream-buggy and writes the wrong values in the wrong fields.

What to verify

  • Pull the HF reference checkpoint swiss-ai/Apertus-8B-Instruct-2509 and inspect the actual act_fn parameter values for layer 0 — see whether they match the unsloth-stored values or are very different (e.g. small).
  • Diff ApertusXIELU.xielu against the activation forward in the HF Apertus modeling code (modeling_apertus.py or similar).
  • Quick sanity probe: replace ApertusXIELU with a no-op linear (or with silu) and re-run the forward pass. If logits become finite, the failure is fully inside xIELU.
  • If a different unsloth quant of Apertus is available, repro under it to rule out a unsloth-conversion glitch isolated to Q4_K_S.

Where the formula lives

  • llm-inference/apertus/src/commonMain/kotlin/sk/ainet/models/apertus/ApertusXIELU.kt:27
  • The DSL wiring is apertusNetwork() line 65: stage.xielu(id = \"act_fn\"). The DSL-emitted op is in skainet-lang-core; need to confirm it computes the same thing as ApertusXIELU.xielu reference.
  • Per-layer params are loaded by extractXIELUParams / extractXIELUParamsFromStreamingMeta in ApertusWeightLoader.kt, then converted to scalar tensors named blk.N.mlp.act_fn.{alpha_p,alpha_n,beta,eps} by ApertusNetworkLoader.applyWeightsToNetwork.

Out of scope

This issue is only about the activation. Loader / module construction / Q4_K dispatch are all correct (PR #102 verified loader + fromGguf().load() end-to-end on real Apertus-8B). When the activation is fixed, forward / generate / tool calling smoke tests in ApertusRealGgufLoadingTest should pass without further loader work.

Langage dominant
Kotlin
Étoiles
8
Forks
3
Merge moyen
19 h 45 min
PR mergées (30 j)
38

Préparer son environnement

Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de SKaiNET-developers/SKaiNET-transformers

Toutes les issues de SKaiNET-developers/SKaiNET-transformers

Issues similaires

Plus d'issues Kotlin

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.