[BUG] Host SBOM scan fails on any executable dd-agent can't read: the deferred analysis pass doesn't skip permission errors
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 56/100
Research direction
Start with pkg/collector/corechecks/sbom/processor.go and trace the deferred analysis path into the pinned Trivy fork’s AnalyzerGroup.DeferredAnalyze and parallel.WalkDir; the issue also points to handler/sysfile for path normalization. Reproduce with host SBOM enabled and os plus languages analyzers on a file the agent cannot read. Done when permission-denied files no longer fail the scan and dpkg-owned paths are skipped, with tests covering both cases.
Written by the indexing model from the issue text.
Description
Agent version
7.84.2 (also seen on 7.84.1)
Bug Report
With host SBOM enabled and sbom.host.analyzers: [os, languages], the agent, running as dd-agent, fails the whole host scan at the first non-empty file that has an exec bit, sits outside the compiled-in skip list, and that dd-agent can't open:
ERROR | (pkg/collector/corechecks/sbom/processor.go:305 in processHostScanResult) | Scan error: unable to marshal report to sbom format, err: unable to scan rootfs, err: trivy scan failed: trivy scan failed: failed analysis: deferred analysis error: deferred analysis error: file open error: open /var/lib/cloud/instances/<instance-id>/scripts/runcmd: permission denied
No SBOM is produced, so the host gets no vulnerability findings at all, which in Datadog looks the same as a host with nothing to report. Every scan fails the same way until the file goes away.
Expected: the file is skipped, as it is everywhere else in the scan. The walker's ErrorCallback (ddtrivy's errorCallback) ignores fs.ErrPermission, AnalyzerGroup.AnalyzeFile breaks on it, and 7.84.0's release note fixed the same class for the static paths (/root/buildinfo/content_manifests).
Cause, in the Trivy fork the agent pins (DataDog/trivy 13735d029a7b, branch djc/main-dd-069). Since 5d0baf780031, "Defer per file application analyzers past OS package detection", the languages and executable analyzers run in AnalyzerGroup.DeferredAnalyze, after the walk, through parallel.WalkDir over the composite FS. During the walk, CompositeFS.CreateLink only records the file without opening it. So the first open is in parallel.walk, which returns file open error for any fsys.Open error. That ends WalkDir and fails the artifact.
A second problem in the same pass. DeferredAnalyze means to skip files an OS package owns (result.SystemInstalledFiles), but nothing is skipped for dpkg. dpkgAnalyzer.parseDpkgMd5sums records each file as "/"+file, while the composite FS's paths are relative, and mapfs.FS.Filter compares them with slices.Contains. The fork's test case "deferred analyzer skips a file owned by an OS package" feeds a relative path, so it passes. Observed: once the first file was readable, the scan stopped at /usr/libexec/samba/smbspool_krb5_wrapper. That file is 0700, from Debian's smbclient (maintainer Debian Samba Maintainers), which isThirdPartyPackage treats as first-party, so it should have been skipped.
Suggested fix:
- In the deferred walk, handle
fs.ErrPermissionasAnalyzeFiledoes: log it and skip the file. One way is to passopts.WalkErrCallbackthrough toparallel.WalkDir. - Trim the leading
/fromSystemInstalledFilesbeforeFilter, ashandler/sysfilealready does ("Trim leading slashes to be the same format as the path in container images").
Reproduction Steps
- Boot a Debian 13 (trixie) cloud image with any cloud-init
runcmd. Ours is one line of vendor data,runcmd: [[systemctl, start, qemu-guest-agent.service]], and cloud-init writes it to/var/lib/cloud/instances/<instance-id>/scripts/runcmdas0700 root:root. - Install agent 7.84.2 with the configuration below.
- Start the agent. The host scan runs at start, and
agent.logshows the error above for every scan.Successfully generated SBOM for hostnever appears.
The same happens on a Debian 13 host with HPE's ssacli package installed, whose binaries under /opt/smartstorageadmin/ssacli/bin are 0500.
Workarounds that work: give dd-agent read on the file (setfacl -m u:dd-agent:r <file>), after which the scan succeeds. Or scan with analyzers: [os], which uses the static paths only.
Agent configuration
sbom:
enabled: true
host:
enabled: true
analyzers:
- os
- languages
Operating System
Debian GNU/Linux 13 (trixie); also Proxmox VE 9 on Debian 13
Other environment details
Agent from the Debian package, running as dd-agent. Trivy fork DataDog/trivy@13735d029a7b, ddtrivy bf6bcaf2f9b7, both still pinned by main's go.mod.
- Dominant language
- Go
- Stars
- 3.8k
- Forks
- 1.5k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 594
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DataDog/datadog-agent
-
oss/0 pending team/windows-products
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
DataDog/datadog-agent#58197 ·
Maintainers usually reply within 1 day
-
[BUG] superfluous response.WriteHeader calls made by telemetryWriterWrapperPossibly taken A pull request linked to this issue is open or already merged. Openoss/0 pending team/container-platform
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
DataDog/datadog-agent#55745 · 1 comment · 1 reaction ·
Maintainers usually reply within 1 day
-
component/config oss/0 pending team/agent-configuration team/fleet-automation
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
DataDog/datadog-agent#55426 ·
Maintainers usually reply within 1 day
-
[BUG] CWS event_monitor fails on Talos Linux: `attach_recursive_mnt` kprobe kernel symbol only present as `attach_recursive_mnt.isra.0`Possibly taken @ChipWolf claimed this 151 days ago. Openoss/0 team/agent-security
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
DataDog/datadog-agent#48510 · 2 comments ·
Maintainers usually reply within 1 day
-
[cluster-agent] KSM-core extended/rollout collectors watch on the 10s-timeout API client, so watches reconnect every 10sPossibly taken @L3n41c claimed this 1 day ago. Openoss/0 pending team/container-integrations
Difficulty 3/5 1-2 days Newbie friendliness 50/100
DataDog/datadog-agent#58111 ·
Maintainers usually reply within 1 day
All issues in DataDog/datadog-agent
Similar issues
-
documentation hacktoberfest priority: low skill: good first issue status: ready for dev
Difficulty 2/5 Half a day Newbie friendliness 85/100
hyperledger-labs/fabric-smart-client#2142 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
numberly/vault-db-injector#107 ·
Maintainers usually reply within 1 day
-
area/import kind/bug
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
[DOC] Wrong sg_egress_ports range in the Terraform Databricks Guide for AWS Hub & Spoke with Network FirewallPossibly taken @akhil9tiet claimed this today. Opendocs
Difficulty 1/5 Under an hour Newbie friendliness 72/100
databricks/terraform-provider-databricks#6080 · 1 comment ·
Maintainers usually reply within 3 days
-
spqr-coordinator usage shows 'run --config', but 'run' is not a subcommand (unknown command "run")Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
pg-sharding/spqr#3092 · 1 comment ·
Maintainers usually reply within 1 day