NullPointerException in NettyClientHandler.onHeadersRead when a HEADERS frame races concurrent stream teardown
Maintainer antworten meist innerhalb von 1 Tag
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 3/5
- Geschätzter Aufwand
- 1-2 Tage
- Anfängerfreundlichkeit
- 76/100
- Issue-Typ
- Bug
- Klarheit
- Klar beschrieben
- Aktivitätsstatus
- Aktiv
- Tech-Stack
- grpc, java
- Bereich
- backend-api-design, networking
Rechercherichtung
Start in NettyClientHandler.java with onHeadersRead, then compare onDataRead and onRstStreamRead with the server-side guard pattern from #10384. Verify behavior when clientStream() returns null after stream teardown; done means cleared TransportState is handled without a null dereference across the affected callbacks.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
What version of gRPC-Java are you using?
1.69.0 (confirmed the same unguarded code is still present at HEAD of the latest release, 1.84.0)
What did you see instead?
A NullPointerException inside NettyClientHandler.onHeadersRead, wrapped by google-cloud-pubsub's StreamingSubscriberConnection:
com.google.api.gax.rpc.UnknownException: io.grpc.StatusRuntimeException: UNKNOWN
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:119)
...
Caused by: io.grpc.StatusRuntimeException: UNKNOWN
at io.grpc.Status.asRuntimeException(Status.java:532)
... 16 common frames omitted
Caused by: java.lang.NullPointerException: Cannot invoke "io.grpc.netty.NettyClientStream$TransportState.tag()" because "stream" is null
at io.grpc.netty.NettyClientHandler.onHeadersRead(NettyClientHandler.java:400)
at io.grpc.netty.NettyClientHandler$FrameListener.onHeadersRead(NettyClientHandler.java:981)
at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onHeadersRead(DefaultHttp2ConnectionDecoder.java:409)
...
Observed on a com.google.cloud:google-cloud-pubsub:1.134.2 Subscriber (StreamingPull), during a burst where ~16 subscriber connections were being established concurrently within a ~10s window (a batch of pods restarting and each re-subscribing to many Pub/Sub subscriptions at once).
What did you expect to see?
No NPE — a stream whose TransportState has already been cleared should be handled gracefully, the same way it's handled elsewhere in this same class.
Root cause (traced from source)
NettyClientHandler.java:399—NettyClientStream.TransportState stream = clientStream(requireHttp2Stream(streamId));NettyClientHandler.java:400—PerfMark.event("NettyClientHandler.onHeadersRead", stream.tag());— dereferencesstreamwith no null check.NettyClientHandler.java:930-931—clientStream():
This returnsprivate NettyClientStream.TransportState clientStream(Http2Stream stream) { return stream == null ? null : (NettyClientStream.TransportState) stream.getProperty(streamKey); }nullwhenever theHttp2Streamobject is still live in the connection (sorequireHttp2Streamdoesn't throw) but itsTransportStateproperty was already cleared — e.g. a stream-cancel/RST race. That's exactly the condition hit when a HEADERS frame for that stream arrives concurrently with its own teardown.
The same unguarded clientStream(requireHttp2Stream(streamId)) → immediate dereference pattern also exists at:
NettyClientHandler.java:436-437(onDataRead)NettyClientHandler.java:448-450(onRstStreamRead)
Suggested fix
This looks like the same class of bug as #10364, which was fixed server-side in #10384 by adding a null guard right after the serverStream(requireHttp2Stream(streamId)) call:
NettyServerStream.TransportState stream = serverStream(requireHttp2Stream(streamId));
if (stream == null) {
return;
}
The same guard pattern would apply to NettyClientHandler's onHeadersRead (and likely onDataRead/onRstStreamRead, which have the identical shape).
Impact observed
Low — this was WARN-level in our application logs (not fatal), and the underlying gax retry logic (UNKNOWN status is retryable) recovered the stream automatically. Filing for visibility since the null-dereference itself is a real, reachable bug with a known-good fix pattern already established for the server-side sibling.
- Vorherrschende Sprache
- Java
- Sterne
- 12.1k
- Forks
- 4k
- Ø Merge
- 2 T. 3 Std.
- Gemergte PRs (30 T.)
- 32
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Keine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus grpc/grpc-java
-
enhancement
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 88/100
grpc/grpc-java#13063 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
grpc/grpc-java#13052 · 4 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
docs enhancement
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
grpc/grpc-java#10824 · 8 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 3/5 1-2 Tage Anfängerfreundlichkeit 65/100
grpc/grpc-java#13075 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 1 Tag
-
question
Schwierigkeit 5/5 Über eine Woche Anfängerfreundlichkeit 25/100
grpc/grpc-java#13066 · 3 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
Ähnliche Issues
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 84/100
beehive-lab/TornadoVM#1151 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 88/100
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 77/100
FasterXML/jackson-dataformats-binary#823 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 90/100
Maintainer antworten meist innerhalb von 1 Tag
-
bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
Maintainer antworten meist innerhalb von 1 Tag