Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

xds: CdsLoadBalancer2 keeps stale READY picker after EDS update with no usable endpoints

Aperta
  • Qualcosa non va?
#13,141 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@lujiajing1126 ci sta già lavorando.

Dal 28/10/2026.

  • #13142 di @lujiajing1126 — aperta

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
64/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
java
Ambito
backend, networking

Direzione di ricerca

Start by reading CdsLoadBalancer2.acceptResolvedAddresses() and trace how the empty-address case updates balancing state; compare it with the existing fail() callers and the earlier ClusterResolverLoadBalancer behavior described in the issue. Add a regression test for an empty EDS update after the cluster is READY, checking that picks fail with UNAVAILABLE and that a later non-empty update recovers.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What version of gRPC-Java are you using?

1.85.1 (the same code is still on master as of 2026-10-27).

What is your environment?

macOS / Linux, JDK 17. xDS control plane: Istio 1.25 (proxyless), also reproduced with an in-process fake ADS server.

What did you expect to see?

When a cluster that was previously READY receives an EDS update (ACKed) whose ClusterLoadAssignment has no usable endpoints, the CDS policy should go to TRANSIENT_FAILURE with an error picker (UNAVAILABLE: No usable endpoint from cluster: ...) and RPCs routed to that cluster should fail fast.

This is what gRPC-Java did before ClusterResolverLoadBalancer was merged into CdsLoadBalancer2. In 1.56.x, ClusterResolverLoadBalancer.handleEndpointResourceUpdate() did:

if (addresses.isEmpty()) {
  ...
  helper.updateBalancingState(TRANSIENT_FAILURE, new ErrorPicker(endpointNotFound));
  if (childLb != null) {
    childLb.shutdown();
    childLb = null;
  }
  return;
}
What did you see instead?

The cluster keeps its previous READY picker and keeps picking the endpoints that were removed by EDS.

In CdsLoadBalancer2.acceptResolvedAddresses(), the empty-address branch calls handleNameResolutionError on the delegate instead of failing:

if (addresses.isEmpty()) {
  Status status = Status.UNAVAILABLE
      .withDescription("No usable endpoint from cluster: " + clusterName);
  delegate.handleNameResolutionError(status);
  return status;
}

The error goes down to PriorityLoadBalancer -> ... -> MultiChildLoadBalancer.handleNameResolutionError(), which only publishes TRANSIENT_FAILURE when the current state is not READY. So a READY subtree keeps its old picker, even though the endpoint list is now authoritatively empty.

Impact: the stale picker stays in use for as long as the old subchannels stay READY. For example, an endpoint can be removed from EDS but still be alive. This happens with subset/label changes, draining, or Istio scaling a subset to 0 while the connection hasn't been torn down yet. In those cases RPCs keep going to endpoints the control plane has removed, instead of failing fast with UNAVAILABLE.

This looks like it came in with ebb9420948 ("xds: Merge ClusterResolverLB into CdsLB2").

Steps to reproduce the bug
  1. Configure an xDS channel with a route to an EDS cluster with a few endpoints (round_robin; also reproduced with a custom LB policy, so it is independent of the leaf policy).
  2. Make RPCs until the cluster is READY.
  3. From the control plane, push an empty ClusterLoadAssignment for that cluster (cluster_name set, no endpoints). Confirm via XdsClient.getSubscribedResourcesMetadataSnapshot() that it is ACKED with 0 endpoint groups.
  4. Make more RPCs to that cluster.

Result: 20/20 picks still return the old endpoints (same result whether or not the EDS version changes).

With this one-line change, all picks fail fast with UNAVAILABLE, other clusters are unaffected, and a later non-empty EDS update recovers normally:

       if (addresses.isEmpty()) {
         Status status = Status.UNAVAILABLE
             .withDescription("No usable endpoint from cluster: " + clusterName);
-        delegate.handleNameResolutionError(status);
-        return status;
+        return fail(status);
       }

(fail() shuts down the delegate, which tears down the priority children, and publishes TF + an error picker. On the next non-empty update the same PriorityLoadBalancer instance rebuilds its children, which is the same recovery path the existing fail() callers already rely on. RPCs already in flight are not affected, because subchannel shutdown is graceful.)

Ambient errors don't reach this branch, since XdsDependencyManager keeps them separate from the data. So this change only affects the authoritative "no usable endpoints" case.

Happy to send a PR with a test if this approach looks right.

Lingua principale
Java
Stelle
12.1k
Fork
4k
Merge medio
2g 1h
PR unite (30g)
48

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di grpc/grpc-java

Tutte le issue di grpc/grpc-java

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.