Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Apertus xIELU formula / param-space mismatch — forward pass NaN's on real Apertus-8B Q4_K_S

Abierto
  • ¿Algo no está bien?
#103 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
30/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Tranquilo
Stack tecnológico
kotlin
Área
backend, machine-learning

Línea de trabajo

Read ApertusXIELU.kt at line 27 and compare its activation formula with the Hugging Face Apertus modeling code; inspect the layer 0 parameters in the reference checkpoint and compare them with the GGUF values. The issue also points to ApertusWeightLoader.kt, the DSL wiring in apertusNetwork(), and ApertusRealGgufLoadingTest for a forward pass probe. Done means identifying the source of the NaNs and confirming the corrected activation produces finite logits.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

After PR #102 the load → convert → fromWeights → forward chain runs to completion against unsloth/Apertus-8B-Instruct-2509-GGUF (Q4_K_S) — but the logit vector is all-NaN at the first forward pass. The propagation source is ApertusXIELU.xielu(buf, params) in layer 0.

Diagnosis

ApertusXIELU.kt evaluates:

alpha_p_eff = softplus(alpha_p)
alpha_n_eff = beta + softplus(alpha_n)

if x > 0:  alpha_p_eff * x*x + beta * x
else:      (expm1(min(x, eps)) - x) * alpha_n_eff + beta * x

with softplus(x) = if (x > 20) x else ln(1 + exp(x)).

The unsloth GGUF stores per-layer xIELU params (verified via gguf Python lib):

layer alpha_p alpha_n beta eps
0 166.0 40.75 0.5 -9.98e-7
1 174.0 31.625 0.5 -9.98e-7
2 128.0 22.875 0.5 -9.98e-7
… (decaying) (decaying) 0.5 -9.98e-7
11 -0.477 … 0.5 -9.98e-7
… (small / negative) … 0.5 -9.98e-7

For layer 0, softplus(166) ≈ 166 (the > 20 shortcut applies), so the activation collapses to 166·x² + 0.5·x. Any input value |x| past a few causes the FFN output to overflow, and the post-FFN matmul + residual chain NaN-poisons the rest of the forward.

Hypothesis

Either:

  1. The unsloth GGUF stores alpha_p / alpha_n in a different transform space than ApertusXIELU.kt expects. softplus(stored) may already be applied (i.e. these are the post-softplus alpha_p_eff values, not the raw learned parameters). In that case the right thing is to drop the softplus calls in the activation and use the stored values directly — which would still leave layer 0 with alpha_p_eff = 166, and the activation would still overflow for any non-trivial input.
  2. ApertusXIELU.kt's formula diverges from the Apertus-8B reference. Possible candidates:
    • The reference may scale alpha_p by some normalization factor.
    • The if x > 0 branch might use alpha_p_eff * (e^x - x - 1) (an ELU-like form) rather than alpha_p_eff * x².
    • beta, eps, etc. may participate in a different combination.
  3. The unsloth converter is upstream-buggy and writes the wrong values in the wrong fields.

What to verify

  • Pull the HF reference checkpoint swiss-ai/Apertus-8B-Instruct-2509 and inspect the actual act_fn parameter values for layer 0 — see whether they match the unsloth-stored values or are very different (e.g. small).
  • Diff ApertusXIELU.xielu against the activation forward in the HF Apertus modeling code (modeling_apertus.py or similar).
  • Quick sanity probe: replace ApertusXIELU with a no-op linear (or with silu) and re-run the forward pass. If logits become finite, the failure is fully inside xIELU.
  • If a different unsloth quant of Apertus is available, repro under it to rule out a unsloth-conversion glitch isolated to Q4_K_S.

Where the formula lives

  • llm-inference/apertus/src/commonMain/kotlin/sk/ainet/models/apertus/ApertusXIELU.kt:27
  • The DSL wiring is apertusNetwork() line 65: stage.xielu(id = \"act_fn\"). The DSL-emitted op is in skainet-lang-core; need to confirm it computes the same thing as ApertusXIELU.xielu reference.
  • Per-layer params are loaded by extractXIELUParams / extractXIELUParamsFromStreamingMeta in ApertusWeightLoader.kt, then converted to scalar tensors named blk.N.mlp.act_fn.{alpha_p,alpha_n,beta,eps} by ApertusNetworkLoader.applyWeightsToNetwork.

Out of scope

This issue is only about the activation. Loader / module construction / Q4_K dispatch are all correct (PR #102 verified loader + fromGguf().load() end-to-end on real Apertus-8B). When the activation is fixed, forward / generate / tool calling smoke tests in ApertusRealGgufLoadingTest should pass without further loader work.

Lenguaje dominante
Kotlin
Estrellas
8
Forks
3
Merge medio
1 d 16 h
PR fusionados (30 d)
48

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de SKaiNET-developers/SKaiNET-transformers

Todos los issues de SKaiNET-developers/SKaiNET-transformers

Issues similares

Más issues de Kotlin

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.