Dendrite could automatically clear power faults

未关闭

#71 创建于 2025年7月14日 0 条评论 0 个 reaction 已指派 1 人在 GitHub 查看

维护者通常 5 天内回复

@bnaecker 已经在做这个了。

开始于 2025年7月14日。

评估

这个 Issue 还没有评估数据。

描述

Originally reported by @bnaecker

While bringing up the next rack, we noticed that one transceiver was faulted, reporting power lost:

BRM27230053-switch # swadm sp xcvr ls
Port   State     Reset Interrupt Power  Power control Vendor           Part No.         Serial No.
qsfp0  supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65A12X
qsfp1  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5NBW41
qsfp2  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5QA1LJ
qsfp3  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X57BJGF
qsfp4  supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5MBDQ4
qsfp5  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5PB5JE
qsfp6  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X72B2BL
qsfp7  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5KBMCX
qsfp8  supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X69BVNJ
qsfp9  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X52AC5S
qsfp10 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X64A0GT
qsfp11 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X72AZ71
qsfp12 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X64A1HF
qsfp13 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X64BWFW
qsfp14 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5EBGDU
qsfp15 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5SB7J4
qsfp16 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65BPU1
qsfp17 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5DBJZT
qsfp18 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65B603
qsfp19 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5QAPG4
qsfp20 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5RA07S
qsfp21 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5PALLC
qsfp22 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65AM09
qsfp23 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65BLFZ
qsfp24 faulted (PowerLost)
qsfp25 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X72B3LF
qsfp26 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X61ABER
qsfp27 supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5QAAJV
qsfp28 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5PABV4
qsfp29 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5QB2MY
qsfp30 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65A14C
qsfp31 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5PAQHE
BRM27230053-switch # xcvradm -i sidecar0 -t 24 vendor-info
Port Identifier               Vendor           Part             Rev  Serial           Mfg date
Some operations failed, errors below
Port Error
  24 Hardware error accessing module 24: FPGA reported an I2C error
BRM27230053-switch # xcvradm -i sidecar0 -t 24 status
Port Status
  24 PRESENT | ENABLED | LOW_POWER_MODE | INTERRUPT | FAULT_POWER_LOST
BRM27230053-switch #

@Aaron-Hartwig and I recalled that this fault is latched, so at some point we lost power and all downstream operations are destined to fail. We cleared the latch manually, and things started working fine:

BRM27230053-switch # xcvradm -i sidecar0 -t 24 clear-power-fault
BRM27230053-switch # xcvradm -i sidecar0 -t 24 status
Port Status
  24 PRESENT | ENABLED | LOW_POWER_MODE | INTERRUPT | POWER_GOOD
BRM27230053-switch #
BRM27230053-switch # xcvradm -i sidecar0 -t 24 vendor-info
Port Identifier               Vendor           Part             Rev  Serial           Mfg date
  24 QsfpPlusCmis (0x1e)      FINISAR CORP.    FTCC1112E2PCL    A    X5DA0P3          210901
BRM27230053-switch # swadm sp xcvr ls
Port   State     Reset Interrupt Power  Power control Vendor           Part No.         Serial No.
qsfp0  supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65A12X
qsfp1  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5NBW41
qsfp2  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5QA1LJ
qsfp3  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X57BJGF
qsfp4  supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5MBDQ4
qsfp5  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5PB5JE
qsfp6  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X72B2BL
qsfp7  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5KBMCX
qsfp8  supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X69BVNJ
qsfp9  supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X52AC5S
qsfp10 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X64A0GT
qsfp11 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X72AZ71
qsfp12 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X64A1HF
qsfp13 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X64BWFW
qsfp14 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5EBGDU
qsfp15 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5SB7J4
qsfp16 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65BPU1
qsfp17 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5DBJZT
qsfp18 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65B603
qsfp19 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5QAPG4
qsfp20 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5RA07S
qsfp21 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5PALLC
qsfp22 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65AM09
qsfp23 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65BLFZ
qsfp24 supported false true      low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5DA0P3
qsfp25 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X72B3LF
qsfp26 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X61ABER
qsfp27 supported false false     low    Hardware      FINISAR CORP.    FTCC1112E2PCL    X5QAAJV
qsfp28 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5PABV4
qsfp29 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5QB2MY
qsfp30 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X65A14C
qsfp31 supported false true      high   Software      FINISAR CORP.    FTCC1112E2PCL    X5PAQHE

This is a pretty bad failure mode, because there is no way to automatically recover from this, requiring operator intervention. Even if the module's power is actually fine, it cannot work until the fault is cleared. Dendrite should consider doing this automatically. During its transceiver monitoring loop, it could note any modules with a power fault and clear that, up to some number of attempts. Should we decide to do this, it needs to be logged, and ideally we'd have a way to control this disposition through the API.

主要语言: Rust
星标: 20
派生: 3
平均合并: 8 小时 29 分钟
30 天内合并 PR: 2

环境准备

我们还没有检查这个项目的环境配置文件。先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

先读完整个 Issue，再读项目的贡献指南。
在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
Fork 仓库，在一个分支上完成修改。
提交 Pull Request，并在描述里引用这个 Issue 编号。

oxidecomputer/dendrite 的其他 Issue

Inverted logic for assigning DHCPv6 addresses to techports works by accident
未关闭
- 哪里不对？
难度 2/5 1-3 小时新手友好度 78/100

oxidecomputer/dendrite#380 · 2026年9月26日创建

维护者通常 5 天内回复
Deleting a link shouldn't fail if the transceiver isn't present
未关闭
- 哪里不对？
难度 3/5 1-2 天新手友好度 65/100

oxidecomputer/dendrite#375 · 2026年9月23日创建

维护者通常 5 天内回复
Revisiting `reset_all` and `reset_all_tagged`
未关闭
- 哪里不对？
难度 5/5 一周以上新手友好度 35/100

oxidecomputer/dendrite#369 · 2026年9月3日创建

维护者通常 5 天内回复
Include `tools` dir in shellcheck CI
未关闭
- 哪里不对？
难度 3/5 1-2 天新手友好度 58/100

oxidecomputer/dendrite#368 · 2026年9月3日创建

维护者通常 5 天内回复
Oximeter types probably shouldn't be in the Dendrite API
未关闭
- 哪里不对？
难度 4/5 3-5 天新手友好度 52/100

oxidecomputer/dendrite#359 · 1 条评论 · 2026年8月31日创建

维护者通常 5 天内回复

查看 oxidecomputer/dendrite 的全部 Issue

相似的 Issue

`categorize_command` has no `uv` arm, so every `rtk uv …` row counts as `other` in the ecosystem mix
未关闭
- 哪里不对？
area:api bug good first issue priority:low

难度 1/5 1 小时以内新手友好度 92/100

rtk-ai/rtk#4316 · 1 条评论 · 2026年9月28日创建

维护者通常 1 天内回复
buzz-pair: panics on wss:// relays — rustls CryptoProvider never installed
未关闭
- 哪里不对？
难度 2/5 1-3 小时新手友好度 92/100

block/buzz#7927 · 2026年9月27日创建

维护者通常 1 天内回复
Installer: `curl https://install.spiceai.org | bash` asks for sudo on a first install and fails without a TTY
未关闭
- 哪里不对？
area/cli kind/bug

难度 2/5 1-3 小时新手友好度 90/100

spiceai/spiceai#14445 · 2026年9月27日创建

维护者通常 1 天内回复
Renovate config extends the shared preset
未关闭
- 哪里不对？
enhancement

难度 2/5 1-3 小时新手友好度 72/100

alunduil/zellij-claude-pair#44 · 2026年9月27日创建
cerberus-3: debug LEDs and button
未关闭
- 哪里不对？
good first issue open-endedness: low type: new feature

难度 2/5 1-3 小时新手友好度 72/100

Northeastern-Electric-Racing/h5-projects-rs#17 · 2026年9月27日创建

更多 Rust Issue

Dendrite could automatically clear power faults

哪里不对？

评估

描述

环境准备

从这里开始

oxidecomputer/dendrite 的其他 Issue

哪里不对？

哪里不对？

哪里不对？

哪里不对？

哪里不对？

相似的 Issue

哪里不对？

哪里不对？

哪里不对？

哪里不对？

哪里不对？

把新 issue 发到你的邮箱