Threat detection fails the run when it was skipped and `continue-on-error: false`
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Start by reading conclude_threat_detection.sh and the detection job's guard and install steps; reproduce with continue-on-error: false and no safe outputs. Check how the conclusion handles RUN_DETECTION: false when installation was skipped. Done when detection reports skipped without failing in that case, while a missing detector still errors when detection was expected to run.
Written by the indexing model from the issue text.
Description
Problem
With safe-outputs.threat-detection.continue-on-error: false, a run in which the agent emits no safe output at all ends with a failed detection job:
ERR_SYSTEM: threat-detect binary not found on PATH
##[error]Process completed with exit code 1.
Nothing was there to analyze. The guard step says so ("Detection skipped: no agent outputs or patches to analyze"), and the run fails anyway, with a message that points at the detector install rather than at the agent.
This happens every time the agent stops before writing to outputs.jsonl. In our case the agent hit max-ai-credits (403 Maximum AI credits exceeded), so outputs.jsonl was never created, and the only failure shown on the run was the threat-detect one.
Seen on v0.89.21 and v0.91.5.
Repro
- A workflow with any safe output and:
safe-outputs: threat-detection: continue-on-error: false - Make the agent end without calling any safe-output tool. For instance, set a
max-ai-creditslow enough to stop it early, or use a prompt that ends without calling any tool,noopincluded. - Look at the
detectionjob:Check if detection needed:OUTPUT_TYPES:empty,HAS_PATCH: false, "Detection skipped: no agent outputs or patches to analyze".Install threat-detect binary: skipped.Conclude threat detection: env hasRUN_DETECTION: false,THREAT_DETECT_INSTALL_OUTCOME: skipped,DETECTION_AGENTIC_EXECUTION_OUTCOME: skipped, thenERR_SYSTEM: threat-detect binary not found on PATHand exit code 1.
Expected
When the guard skipped detection because there was nothing to analyze, the conclusion should report it as skipped, as the comment in conclude_threat_detection.sh describes, whatever continue-on-error is set to. A missing detector should only be reported as an error when detection was actually expected to run.
- Dominant language
- Go
- Stars
- 5.4k
- Forks
- 576
- Avg merge
- 8h 22m
- Merged PRs (30d)
- 798
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- Ships a Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from github/gh-aw
-
agentic-workflows maintenance
Difficulty 1/5 1-3 hours Newbie friendliness 77/100
github/gh-aw#66635 · 6 comments ·
Maintainers usually reply within 1 day
-
agentic-workflows
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
[uk-ai-resilience] Missing .github/CODEOWNERS for security-sensitive compiler/CLI paths (Tier B)Openai-generated cookie high-priority security
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
ai-generated automation testing
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
agentic-workflows maintenance
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
github/gh-aw#34892 · 2 comments ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
Maintainers usually reply within 1 day
-
product/backlog
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
TencentBlueKing/bk-nodemgr#3645 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
modelcontextprotocol/go-sdk#1387 ·
Maintainers usually reply within 1 day
-
docs(prompt): filer socket fragment hardcodes an issue-intent cap of 8 that Chore runs overrideOpenagent-butler-finding chore
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
jordansmall/spindrift#5128 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 84/100
Maintainers usually reply within 2 days