[awf] network: isolation/topology mode + Tailscale → Squid 503 TCP_TUNNEL:HIER_NONE on api.githubcopilot.com egress
Maintainers usually reply within 1 day
Assessment
This issue has not been assessed yet.
Description
Problem
In network-isolation / topology mode, when a workflow also brings up Tailscale, the api-proxy's egress CONNECT to api.githubcopilot.com:443 fails at Squid with 503 TCP_TUNNEL:HIER_NONE. The request never leaves the sandbox, so the Copilot engine reports Failed to get response from the AI model; retried 5 times … Last error: 503 Service Unavailable and the agent job fails.
The same workflow works on the older host-access stack, and a non-Tailscale workflow works in the same isolation mode — so it's the isolation-mode × Tailscale combination, not the Copilot API, the model, or the request.
Where observed
- Workflow:
github/copilot-code-review→datadog-alert-triage(Copilot engine,claude-opus-4.8). It brings up Tailscale viagithub/setup-tailscaleto reach 5 Splunk*.ts.netMCP servers, plus Datadog/Sentry/Kusto/GitHub MCP. - Reproduced on gh-aw
v0.82.15,v0.83.1,v0.83.4(api-proxy0.27.37→0.27.42, Copilot CLI1.0.71→1.0.75). Works onmain's stack (gh-awv0.81.6, api-proxy0.27.11, host-access mode).
Evidence
Squid access log — failing run (api-proxy 0.27.37): every CONNECT to the Copilot API fails:
api.githubcopilot.com:443 -:- 1.1 CONNECT 503 TCP_TUNNEL:HIER_NONE api.githubcopilot.com:443 "-"
→ 56/56 CONNECTs = 503.
Same workflow, working run (api-proxy 0.27.11, host-access): 42/42 CONNECTs = 200.
Resolved firewall config diff (audit/awf-resolved-config.json):
| failing (new) | working (old) | |
|---|---|---|
networkIsolation |
true |
false |
enableHostAccess |
false |
true |
api-proxy OTEL span (failing): GET /models → http.response.status_code: 503 (the proxy just propagates the Squid CONNECT failure; the fatal one is the chat request).
Copilot harness classification: failureClass=partial_execution, isModelNotSupportedError=false, isCAPIQuotaExceededError=false, isAuthError=false.
Environment: Squid 6.14; nameserver 168.63.129.16 (Azure DNS).
Run IDs (github/copilot-code-review Actions):
- Fail — v0.82.15 / 0.27.37:
30401644133 - Fail — v0.83.4 / 0.27.42:
30397968304 - Fail — v0.83.1 / 0.27.38:
30390759136,30394229055 - Work — main / 0.27.11:
30393390874,30402440480
Isolation matrix (what rules causation in/out)
| firewall mode | Tailscale up? | api.githubcopilot.com egress |
|---|---|---|
| host-access (0.27.11) | yes | ✅ 200 |
| isolation/topology (0.27.37) | no — github/copilot-metrics project-triage, claude-opus-4.8, succeeds daily |
✅ |
| isolation/topology (0.27.37–0.27.42) | yes — our workflow | ❌ 503 HIER_NONE |
This rules out: the Copilot API (never receives the request), the model / retirement (isModelNotSupportedError=false; opus-4.8 serves fine elsewhere), request size (the CONNECT fails before any payload), and repo/token (works on host-access with identical secrets). The only differentiator is isolation-mode + Tailscale.
Suspected root cause
In isolation/topology mode the agent + api-proxy egress traverses Squid with no host route (per #6657). When Tailscale comes up it rewrites routing/DNS; in the isolated topology this appears to black-hole the route/resolution to api.githubcopilot.com, so Squid has no forwarding path for the CONNECT and returns HIER_NONE. In host-access mode the host route sidestepped the collision. (HIER_NONE is a forward/resolve failure, distinct from an ACL TCP_DENIED.) Maintainers can confirm the exact interaction — e.g. whether Tailscale is capturing the default route or the Azure DNS (168.63.129.16) path used by Squid.
Related
- #6670 (open) — same subsystem: Squid ACL doesn't auto-allow
topologyAttachhosts (internal MCP gateway blocked). Likely a sibling gap in topology-mode egress/ACL handling. - #6657 (merged) — host-access × network-isolation; documents that in topology mode the agent runs on an internal Docker network with no host route.
Impact
Blocks upgrading github/copilot-code-review's opus-based FR/alert-triage fleet past the current host-access stack — every newer stack tested 503s. Any Tailscale-using workflow in isolation mode is likely affected.
Possible mitigations to validate
- Exclude allowlisted external egress domains (e.g.
*.githubcopilot.com) from Tailscale's route table / DNS interception when isolation mode is active. - Bring Tailscale up so it doesn't capture the default route / DNS resolver Squid uses for egress.
- Provide a host-access egress path for the api-proxy in topology mode (cf. #6657).
Filed after an end-to-end investigation; happy to share full firewall log artifacts or run a controlled Tailscale-removed repro on request.
- Dominant language
- TypeScript
- Stars
- 149
- Forks
- 77
- Avg merge
- 4h 20m
- Merged PRs (30d)
- 436
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from github/gh-aw-firewall
-
agentic-workflows automated dependencies
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
github/gh-aw-firewall#3837 ·
Maintainers usually reply within 1 day
-
agentic-workflows
Difficulty 4/5 1-2 days Newbie friendliness 35/100
github/gh-aw-firewall#10315 ·
Maintainers usually reply within 1 day
-
Router: port the routing core and CLI to Go (built and tested, not shipped)Possibly taken @rabeckett claimed this today. Openenhancement refactoring
Difficulty 5/5 Over a week Newbie friendliness 1/100
github/gh-aw-firewall#10312 ·
Maintainers usually reply within 1 day
-
agentic-workflows
Difficulty 2/5 1-3 hours Newbie friendliness 48/100
github/gh-aw-firewall#10309 ·
Maintainers usually reply within 1 day
-
agentic-workflows
Difficulty 3/5 1-2 days Newbie friendliness 35/100
github/gh-aw-firewall#10308 ·
Maintainers usually reply within 1 day
All issues in github/gh-aw-firewall
Similar issues
-
area:workflow bug ready-for-human
Difficulty 1/5 Under an hour Newbie friendliness 78/100
fil-donadoni/tolaria#6108 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/rocketmq-dashboard#7234 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
vitest-dev/vitest#11656 ·
Maintainers usually reply within 1 day
-
e2e-failure ready-to-code
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
redhat-developer/rhdh-plugin-export-overlays#4460 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
ehmpathy/rhachet-roles-bhrain#655 ·
Maintainers usually reply within 1 day