NullPointerException in NettyClientHandler.onHeadersRead when a HEADERS frame races concurrent stream teardown
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 76/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- grpc, java
- Ambito
- backend-api-design, networking
Direzione di ricerca
Start in NettyClientHandler.java with onHeadersRead, then compare onDataRead and onRstStreamRead with the server-side guard pattern from #10384. Verify behavior when clientStream() returns null after stream teardown; done means cleared TransportState is handled without a null dereference across the affected callbacks.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
What version of gRPC-Java are you using?
1.69.0 (confirmed the same unguarded code is still present at HEAD of the latest release, 1.84.0)
What did you see instead?
A NullPointerException inside NettyClientHandler.onHeadersRead, wrapped by google-cloud-pubsub's StreamingSubscriberConnection:
com.google.api.gax.rpc.UnknownException: io.grpc.StatusRuntimeException: UNKNOWN
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:119)
...
Caused by: io.grpc.StatusRuntimeException: UNKNOWN
at io.grpc.Status.asRuntimeException(Status.java:532)
... 16 common frames omitted
Caused by: java.lang.NullPointerException: Cannot invoke "io.grpc.netty.NettyClientStream$TransportState.tag()" because "stream" is null
at io.grpc.netty.NettyClientHandler.onHeadersRead(NettyClientHandler.java:400)
at io.grpc.netty.NettyClientHandler$FrameListener.onHeadersRead(NettyClientHandler.java:981)
at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(DefaultHttp2ConnectionDecoder.java:409)
...
Observed on a com.google.cloud:google-cloud-pubsub:1.134.2 Subscriber (StreamingPull), during a burst where ~16 subscriber connections were being established concurrently within a ~10s window (a batch of pods restarting and each re-subscribing to many Pub/Sub subscriptions at once).
What did you expect to see?
No NPE — a stream whose TransportState has already been cleared should be handled gracefully, the same way it's handled elsewhere in this same class.
Root cause (traced from source)
NettyClientHandler.java:399—NettyClientStream.TransportState stream = clientStream(requireHttp2Stream(streamId));NettyClientHandler.java:400—PerfMark.event("NettyClientHandler.onHeadersRead", stream.tag());— dereferencesstreamwith no null check.NettyClientHandler.java:930-931—clientStream():
This returnsprivate NettyClientStream.TransportState clientStream(Http2Stream stream) { return stream == null ? null : (NettyClientStream.TransportState) stream.getProperty(streamKey); }nullwhenever theHttp2Streamobject is still live in the connection (sorequireHttp2Streamdoesn't throw) but itsTransportStateproperty was already cleared — e.g. a stream-cancel/RST race. That's exactly the condition hit when a HEADERS frame for that stream arrives concurrently with its own teardown.
The same unguarded clientStream(requireHttp2Stream(streamId)) → immediate dereference pattern also exists at:
NettyClientHandler.java:436-437(onDataRead)NettyClientHandler.java:448-450(onRstStreamRead)
Suggested fix
This looks like the same class of bug as #10364, which was fixed server-side in #10384 by adding a null guard right after the serverStream(requireHttp2Stream(streamId)) call:
NettyServerStream.TransportState stream = serverStream(requireHttp2Stream(streamId));
if (stream == null) {
return;
}
The same guard pattern would apply to NettyClientHandler's onHeadersRead (and likely onDataRead/onRstStreamRead, which have the identical shape).
Impact observed
Low — this was WARN-level in our application logs (not fatal), and the underlying gax retry logic (UNKNOWN status is retryable) recovered the stream automatically. Filing for visibility since the null-dereference itself is a real, reachable bug with a known-good fix pattern already established for the server-side sibling.
- Lingua principale
- Java
- Stelle
- 12.1k
- Fork
- 4k
- Merge medio
- 2g 3h
- PR unite (30g)
- 32
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di grpc/grpc-java
-
enhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
grpc/grpc-java#13063 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
grpc/grpc-java#13052 · 4 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Support of `dns:name` URIsApertadocs enhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
grpc/grpc-java#10824 · 8 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 3/5 1-2 giorni Idoneità per principianti 65/100
grpc/grpc-java#13075 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
question
Difficoltà 5/5 Più di una settimana Idoneità per principianti 25/100
grpc/grpc-java#13066 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di grpc/grpc-java
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
beehive-lab/TornadoVM#1151 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
I maintainer di solito rispondono entro 1 giorno
-
(cbor) `maxStringLength` not consistently checked for chunked (indefinite-length) text valuesAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 77/100
FasterXML/jackson-dataformats-binary#823 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
I maintainer di solito rispondono entro 1 giorno
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 1 giorno