Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

xds: CdsLoadBalancer2 keeps stale READY picker after EDS update with no usable endpoints

Offen
  • Stimmt etwas nicht?
#13,141 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

@lujiajing1126 arbeitet bereits daran.

Seit 28.10.2026.

  • #13142 von @lujiajing1126 — offen

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Anfängerfreundlichkeit
64/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
java
Bereich
backend, networking

Rechercherichtung

Start by reading CdsLoadBalancer2.acceptResolvedAddresses() and trace how the empty-address case updates balancing state; compare it with the existing fail() callers and the earlier ClusterResolverLoadBalancer behavior described in the issue. Add a regression test for an empty EDS update after the cluster is READY, checking that picks fail with UNAVAILABLE and that a later non-empty update recovers.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

What version of gRPC-Java are you using?

1.85.1 (the same code is still on master as of 2026-10-27).

What is your environment?

macOS / Linux, JDK 17. xDS control plane: Istio 1.25 (proxyless), also reproduced with an in-process fake ADS server.

What did you expect to see?

When a cluster that was previously READY receives an EDS update (ACKed) whose ClusterLoadAssignment has no usable endpoints, the CDS policy should go to TRANSIENT_FAILURE with an error picker (UNAVAILABLE: No usable endpoint from cluster: ...) and RPCs routed to that cluster should fail fast.

This is what gRPC-Java did before ClusterResolverLoadBalancer was merged into CdsLoadBalancer2. In 1.56.x, ClusterResolverLoadBalancer.handleEndpointResourceUpdate() did:

if (addresses.isEmpty()) {
  ...
  helper.updateBalancingState(TRANSIENT_FAILURE, new ErrorPicker(endpointNotFound));
  if (childLb != null) {
    childLb.shutdown();
    childLb = null;
  }
  return;
}
What did you see instead?

The cluster keeps its previous READY picker and keeps picking the endpoints that were removed by EDS.

In CdsLoadBalancer2.acceptResolvedAddresses(), the empty-address branch calls handleNameResolutionError on the delegate instead of failing:

if (addresses.isEmpty()) {
  Status status = Status.UNAVAILABLE
      .withDescription("No usable endpoint from cluster: " + clusterName);
  delegate.handleNameResolutionError(status);
  return status;
}

The error goes down to PriorityLoadBalancer -> ... -> MultiChildLoadBalancer.handleNameResolutionError(), which only publishes TRANSIENT_FAILURE when the current state is not READY. So a READY subtree keeps its old picker, even though the endpoint list is now authoritatively empty.

Impact: the stale picker stays in use for as long as the old subchannels stay READY. For example, an endpoint can be removed from EDS but still be alive. This happens with subset/label changes, draining, or Istio scaling a subset to 0 while the connection hasn't been torn down yet. In those cases RPCs keep going to endpoints the control plane has removed, instead of failing fast with UNAVAILABLE.

This looks like it came in with ebb9420948 ("xds: Merge ClusterResolverLB into CdsLB2").

Steps to reproduce the bug
  1. Configure an xDS channel with a route to an EDS cluster with a few endpoints (round_robin; also reproduced with a custom LB policy, so it is independent of the leaf policy).
  2. Make RPCs until the cluster is READY.
  3. From the control plane, push an empty ClusterLoadAssignment for that cluster (cluster_name set, no endpoints). Confirm via XdsClient.getSubscribedResourcesMetadataSnapshot() that it is ACKED with 0 endpoint groups.
  4. Make more RPCs to that cluster.

Result: 20/20 picks still return the old endpoints (same result whether or not the EDS version changes).

With this one-line change, all picks fail fast with UNAVAILABLE, other clusters are unaffected, and a later non-empty EDS update recovers normally:

       if (addresses.isEmpty()) {
         Status status = Status.UNAVAILABLE
             .withDescription("No usable endpoint from cluster: " + clusterName);
-        delegate.handleNameResolutionError(status);
-        return status;
+        return fail(status);
       }

(fail() shuts down the delegate, which tears down the priority children, and publishes TF + an error picker. On the next non-empty update the same PriorityLoadBalancer instance rebuilds its children, which is the same recovery path the existing fail() callers already rely on. RPCs already in flight are not affected, because subchannel shutdown is graceful.)

Ambient errors don't reach this branch, since XdsDependencyManager keeps them separate from the data. So this change only affects the authoritative "no usable endpoints" case.

Happy to send a PR with a test if this approach looks right.

Vorherrschende Sprache
Java
Sterne
12.1k
Forks
4k
Ø Merge
2 T. 1 Std.
Gemergte PRs (30 T.)
48

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus grpc/grpc-java

Alle Issues in grpc/grpc-java

Ähnliche Issues

Weitere Issues zu Java

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.