Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Faulted link observed during MUPdate with limited retrospective debugging

Abierto
  • ¿Algo no está bien?
#253 0 comentarios 0 reacciones 1 asignado Ver en GitHub

Los mantenedores suelen responder en 1 día

@rcgoodfellow ya está trabajando en esto.

Desde el 1/4/2026.

Evaluación

Este issue todavía no se ha evaluado.

Descripción

We observed failures during initial manufacturing MUPdate of sled 22 in a rack. I'm leaving the issue title here open ended since I am relatively unfamiliar with the system. The things we'd like are (1) a mechanism to understand why a link (e.g. rear22/0) is in a Faulted state and (2) a mechanism (e.g. logs) to get this retrospectively - for example, after ignition cycle when links are back up.

See internal ticket for details here: https://github.com/oxidecomputer/mfg-troubleshooting/issues/926.

/staff/adam/dendrite-fault-rear22/ contains the output of swadm link history and the dendrite service logs.

adam@atrium ~ $ find /staff/adam/dendrite-fault-rear22/ -type f
/staff/adam/dendrite-fault-rear22/XXXXXXXX/swadm-link-history-rear220.txt
/staff/adam/dendrite-fault-rear22/XXXXXXXX/oxide-dendrite:default.log
/staff/adam/dendrite-fault-rear22/YYYYYYYY/oxide-dendrite:default.log
/staff/adam/dendrite-fault-rear22/YYYYYYYY/swadm-link-history-rear220.txt

MUPdate was hung downloading installinator because both links were down:

XXXXXXXX # dladm
LINK        CLASS     MTU    STATE    BRIDGE     OVER
cxgbe0      phys      1500   down     --         --
cxgbe1      phys      1500   down     --         --
bootstrap\_stub0 etherstub 9000 up     --         --
bootstrap0  vnic      1500   up       --         bootstrap\_stub0
XXXXXXXX #

swadm link ls in the switch zones showed rear22/0 in a Faulted state.

support@oxz_switch0:~$ swadm link ls
...
rear22/0   Copper  100G   RS    true     true     Faulted  xx:xx:xx:xx:xx:xx
...

We ignition cycled multiple times and XXXXXXXX / 22 rebooted with all links up:

XXXXXXXX # dladm
LINK        CLASS     MTU    STATE    BRIDGE     OVER
cxgbe0      phys      9000   up       --         --
cxgbe1      phys      9000   up       --         --
XXXXXXXX #

Likewise, we lost the Faulted state in swadm link ls:

support@oxz_switch0:~$ swadm link ls
...
rear22/0   Copper  100G   RS    true     true     Up       xx:xx:xx:xx:xx:xx
...

dendrite logs show the link transitioning out of Faulted:

20:17:41.822Z DEBG dpd: Link update
    link_id = 0
    new = Up
    old = Faulted
    port_id = rear22
    state = LinkUp
    unit = callback_handler

swadm link history showed loops of:

14241983  PortFSM  Idle                         -
14242053  PortFSM  Abort                        -
14247600  PortFSM  WaitAutoNegLinkTrainingDone  -
14250248  PortFSM  WaitAutoNegDone              -

until ignition cycles resulted in the link going up:

Time      Class    Subclass                     Channel  Details
3746629   PortFSM  LinkUp                       -
3746639   PortFSM  LinkDown                     -
3747009   PortFSM  WaitAutoNegLinkTrainingDone  -
4197483   PortFSM  WaitAutoNegDone              -
4198486   PortFSM  Idle                         -
Lenguaje dominante
Rust
Estrellas
21
Forks
3
Merge medio
8 h 29 min
PR fusionados (30 d)
2

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de oxidecomputer/dendrite

Todos los issues de oxidecomputer/dendrite

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.