agent-substrate/substrate

E2E test flaky: TestActorDirectAccess/via_ingress

Open

#723 opened on Aug 4, 2026

View on GitHubContributor guide
 (0 comments) (0 reactions) (0 assignees)Go (195 forks)github user discovery
area/dev-infragood first issuekind/bug

Repository metrics

Stars
 (969 stars)
PR merge metrics
 (PR metrics pending)

Description

The E2E test TestActorDirectAccess/via_ingress in internal/e2e/suites/networking/networking_test.go:53 fails intermittently in CI with an ingress connection timeout.

Error

networking_test.go:53: Actor access through ingress returned HTTP 503, want 200; body: upstream connect error or disconnect/reset before headers. reset reason: connection timeout
--- FAIL: TestActorDirectAccess (5.72s)
    --- FAIL: TestActorDirectAccess/via_ingress (5.03s)
FAIL    github.com/agent-substrate/substrate/internal/e2e/suites/networking

Sample failures

Both fail with the same 503/connection timeout against the ingress path. The direct subtest passes in both cases — only via_ingress is affected.

Observed pass rate

Passed in runs 30835856539, 30818259089, 30817878192 — fails roughly 2 out of every 6 observed runs.

Likely cause

The ingress controller or Envoy proxy is not fully ready when the test probes it, causing an intermittent 503. The test likely needs a readiness wait or retry loop before asserting the 200 response via ingress.

Contributor guide

Research direction
Trace the via ingress test path, understand the ingress setup, and add a readiness retry loop to prevent intermittent 503 timeouts.
Tech stack
go
Domain
backendnetworkingtesting
Issue type
Test
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
GoGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65