oxlog: stop listing /var/dendrite as an extra log directory
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in dev-tools/oxlog/src/lib.rs at Zones::load and trace ExtraLogDir::Dendrite and the switch zone's extra log directory. Remove the obsolete directory entry, then run the relevant oxlog checks or tests and confirm that oxlog logs oxz_switch --extra no longer lists zlog-cfg-cur.
Written by the indexing model from the issue text.
Description
oxlog lists /zone/oxz_switch/root/var/dendrite as an "extra" log directory for the dendrite service (Zones::load in dev-tools/oxlog/src/lib.rs), but on current systems it holds no logs.
What's in the directory
dpd's start method (svc-dpd in dendrite) runs dpd from /var/dendrite after copying misc/zlog-cfg there as zlog-cfg-cur. That zlog configuration sends the Tofino SDE's driver logging to bf_drivers.log ("bf_drivers.log", 5M * 5), in dpd's working directory.
However, dpd now sends the SDE's log messages to its own structured log instead. In dendrite, asic/src/tofino_asic/sde_log.rs defines bf_sys_log_callback ("an FFI-compatible routine that the SDE's C code can call when it has a message to be logged"), and asic/src/tofino_asic/mod.rs installs it with a logger tagged "unit" => "bf-sde".
What we see on rack2 (switch zone on BRM42220014)
- dpd runs with
/var/dendriteas its working directory. The directory contains onlyzlog-cfg-cur, and dpd has no files open in it. - No
bf_drivers.log*exists anywhere in the switch zone. - SDE messages appear in dpd's SMF log,
oxide-dendrite:default.log, e.g.{"msg":"QSFP 62 : FLAG_RX_LOS change (0x0 -> 0xf)", …, "unit":"bf-sde","module":"Pltfm"}. - An archived dpd log from an older boot (file
mtimeMarch 2024) contains zlog-formatted SDE lines on stdout, e.g.1986-12-28 00:01:16.244179 BF_PLTFM ERROR - QSFP 1 : …. The current boot's log has none, so SDE logging appears to have moved from zlog to dpd's callback at some point.
Since the SDE's logs now live in oxide-dendrite:default.log, oxlog already reports them as the dendrite service's SMF logs, including in support bundles.
Proposal
- Remove
ExtraLogDir::Dendrite, and the switch zone's extra log directory, from oxlog. - This changes the output of
oxlog logs oxz_switch --extra: it would no longer listzlog-cfg-cur. Support bundles don't collect that file today anyway: sled-diagnostics' extra-log parser skips it, because it isn't named like a log file.
Open questions
- Is there any configuration or platform (e.g. softnpu, or a lab setup) where dpd still writes
bf_drivers.logto/var/dendrite? If so, the directory should stay, and it would need a naming rule for zlog's rotated files. - For the dendrite owners: does
svc-dpdstill need to copyzlog-cfginto/var/dendrite, now that SDE logging goes throughbf_sys_log_callback? (Outside omicron; just flagging it.)
Found while working on #11372.
- Dominant language
- Rust
- Stars
- 610
- Forks
- 104
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 120
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from oxidecomputer/omicron
-
development Testing & Analysis
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
oxidecomputer/omicron#11582 ·
Maintainers usually reply within 1 day
-
fault-management
Difficulty 2/5 1-3 hours Newbie friendliness 64/100
oxidecomputer/omicron#11444 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
oxidecomputer/omicron#11397 ·
Maintainers usually reply within 1 day
-
non-important non-blocker Testing & Analysis
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
oxidecomputer/omicron#11374 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
oxidecomputer/omicron#11269 ·
Maintainers usually reply within 1 day
All issues in oxidecomputer/omicron
Similar issues
-
area/backends bug session-analysis
Difficulty 2/5 1-3 hours Newbie friendliness 69/100
Maintainers usually reply within 1 day
-
docs good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
crash
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 2 days
-
Difficulty 1/5 Under an hour Newbie friendliness 65/100
aaif-goose/goose#13125 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Maintainers usually reply within 1 day