Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

One client disconnect closes the shared WASM guard and gives 502 for the rest of the session

Open
  • Is something wrong?
#14,050 1 comment 1 reaction 2 assignees View on GitHub

Maintainers usually reply within 1 day

@lpcox is already working on this.

Since Sep 28, 2026.

  • #14094 by @copilot-swe-agent — open

Assessment

This issue has not been assessed yet.

Description

Summary

The proxy calls the WASM guard with the HTTP request context (r.Context()). The wazero runtime uses WithCloseOnContextDone(true). When a client disconnects while its request waits for g.mu, the next guard call on that request uses a canceled context. wazero then closes the module that all requests share. isWasmTrap classifies the resulting sys.ExitError (exit code ExitCodeContextCanceled = 0xffffffff) as a trap, so the gateway sets g.failed = true. After that, every request gets HTTP 502 resource labeling failed for the rest of the session.

Thus one client that stops unexpectedly (crash, timeout, Ctrl-C) disables the proxy for all other clients.

Seen on v0.4.25 (CLI proxy mode, gh-aw-firewall 0.28.23). The code path is the same on v0.4.9 and on main (81d6a4d5).

Code path

  1. internal/proxy/handler.go handleWithDIFC: ctx := r.Context() goes to guard.RunPipelinePrePhases and guard.RunPipelinePhase4.
  2. internal/guard/wasm_lifecycle.go callWasmGuardFunction: g.mu.Lock(). Requests wait here one at a time. The context of a waiting request can be canceled while it waits.
  3. internal/guard/wasm_exec.go tryCallWasmFunction: g.wasmAlloc(ctx, …) calls alloc with the canceled context. With WithCloseOnContextDone(true), wazero (v1.12.0, ModuleInstance.CloseModuleOnCanceledOrTimeout) closes the module and returns sys.ExitError "module closed with context canceled".
  4. internal/guard/wasm_lifecycle.go isWasmTrap: exitErr.ExitCode() != 0 is true, so callWasmFunction sets g.failed = true.
  5. All later calls return WASM guard 'github' is unavailable after a previous trap.

The cleanup code already uses context.WithoutCancel(ctx) for dealloc, but alloc and the guard function call use the request context.

Evidence

A script sent 6 parallel gh api requests through the CLI proxy. Some gh processes in the CLI proxy container crashed with runtime: failed to create new OS thread (have 5 already; errno=11) (a separate thread-limit problem). One of them (a GraphQL pull_request_read) had a request in progress:

CLI proxy access.jsonl:

02:57:40.202Z exec_start api graphql (PR A)
02:57:40.580Z exec_done  api graphql (PR A) exit=2  "runtime: failed to create new OS thread (have 5 already; errno=11)"

Gateway proxy.log: three GraphQL requests arrive together and wait for the guard lock. The request of the crashed client gets the lock about 640 ms after the client stopped:

02:57:40.382Z [proxy:handler] incoming POST /api/graphql
02:57:40.388Z [proxy:handler] incoming POST /api/graphql
02:57:40.390Z [proxy:handler] incoming POST /api/graphql
...
02:57:41.215Z [guard:wasm] label_resource input JSON: 98 bytes
02:57:41.216Z [guard:wasm] Using guard allocator path: guard=github
02:57:41.218Z [ERROR] [backend] WASM guard trap: guard=github, func=label_resource, error=failed to allocate WASM input buffer: module closed with context canceled
02:57:41.219Z [guard:wasm] callWasmFunction: guard=github is unavailable after a previous trap
02:57:41.222Z [ERROR] [proxy] Request rejected: status=502 code=bad_gateway message=resource labeling failed ...

There was one trap in the log, followed by 277 rejected requests until the run stopped. The two other waiting requests got the coarse-grained fallback in Phase 4. All later requests failed in Phase 1.

Proposed fix

Any of these prevents the failure. The first is the smallest change:

  • Call the guard with context.WithoutCancel(ctx) (or a new context with a short timeout) instead of the request context. A guard call is short and changes shared state, so a canceled client must not be able to stop it partway through.
  • Before callWasmFunction, check ctx.Err() after the lock is acquired and return that error. Do not call into the module with a context that is already canceled.
  • In isWasmTrap, treat sys.ExitCodeContextCanceled and sys.ExitCodeDeadlineExceeded as host errors, and reinstantiate the module (the NewSessionGuard path already does this) instead of setting g.failed for the full session.

Related

  • #3711: a guest panic disabled the guard for the full session. The failure mode is the same, but the trigger here is on the host side, and it applies to every guard.
Dominant language
Go
Stars
176
Forks
51
Avg merge
6h 40m
Merged PRs (30d)
258

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from github/gh-aw-mcpg

All issues in github/gh-aw-mcpg

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.