HTTP/2: a received response waits until more data arrives on the connection

Open
  • Is something wrong?
#1,300 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@AdityaSreevatsaK is already working on this.

Since Nov 12, 2026.

  • #1334 by @AdityaSreevatsaK — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
backend, networking

Research direction

Start with the HTTP/2 read and event handling paths in http2.py, especially _receive_response_body, _receive_stream_event, and _receive_events, then inspect _synchronization.py and the behavior around #1163 and #1289. Run the supplied reproduction for concurrent fast and slow responses. Done means fully received responses are delivered promptly even when other streams are waiting for more connection data, with regression coverage for the reproduction.

Written by the indexing model from the issue text.

Description

Summary

With several concurrent HTTP/2 requests on one connection, a request whose response has fully arrived can stay undelivered until more data arrives on the connection. Its DataReceived and StreamEnded events are already queued for its stream, but its task is still waiting to acquire _read_lock.

In the reproduction below (six 50 ms responses and four 1500 ms responses on one connection), at least one fast request is still undelivered at 300 ms. A client that sends concurrent requests with a per-call deadline can see a request time out after its response has arrived.

Over TLS this happens in every run on httpcore2 2.16.0 and on main. In cleartext (prior-knowledge HTTP/2) it happens in nearly every run on every version tested, including encode/httpcore 1.0.9.

Reproduction

pip install "httpx2[http2]==2.16.0"
python repro.py --tls

--tls needs openssl on PATH for a throwaway certificate. The script runs an h2 server in a thread that sends each response's headers and body in separate writes. The client warms one connection, then sends ten concurrent requests on it.

Expected: all six fast requests complete within 300 ms. Observed: at least one doesn't; the script prints which and exits with status 1.

--post sends a small JSON body, and leaving out --tls switches to cleartext prior-knowledge HTTP/2. To count stalls over ten runs:

for i in $(seq 10); do python repro.py --tls >/dev/null || echo stall; done | grep -c stall
repro.py 
"""Reproduction: a fully received HTTP/2 response is not delivered until more data arrives on the connection.

Requires: httpx2[http2] (h2 is the only server dependency). --tls also needs openssl on PATH.
Run:  python repro.py [--tls] [--post]   -> exit 1 if a fast response stalled
      python repro.py --no-fast-acquire  (patches httpcore2's AsyncLock to anyio.Lock(fast_acquire=False))
"""
import asyncio
import os
import socket
import ssl
import subprocess
import tempfile
import sys
import threading
import time
from urllib.parse import parse_qs, urlsplit

import h2.config
import h2.connection
import h2.events

# ---------------------------------------------------------------- server (own thread + loop)
async def handle(reader, writer):
    conn = h2.connection.H2Connection(h2.config.H2Configuration(client_side=False))
    conn.initiate_connection()
    writer.write(conn.data_to_send())

    async def respond(stream_id, delay_ms):
        await asyncio.sleep(delay_ms / 1000)
        # Headers and body in separate writes (as a proxy streaming a response does).
        conn.send_headers(stream_id, [(":status", "200"), ("content-length", "2")])
        writer.write(conn.data_to_send())
        await writer.drain()
        conn.send_data(stream_id, b"ok", end_stream=True)
        writer.write(conn.data_to_send())

    paths = {}
    while data := await reader.read(65536):
        for ev in conn.receive_data(data):
            if isinstance(ev, h2.events.RequestReceived):
                paths[ev.stream_id] = dict(ev.headers)[b":path"].decode()
            elif isinstance(ev, h2.events.DataReceived):
                conn.acknowledge_received_data(ev.flow_controlled_length, ev.stream_id)
            elif isinstance(ev, h2.events.StreamEnded):
                q = parse_qs(urlsplit(paths[ev.stream_id]).query)
                asyncio.ensure_future(respond(ev.stream_id, int(q.get("delay_ms", ["0"])[0])))
        writer.write(conn.data_to_send())


def tls_contexts():
    """Throwaway self-signed localhost certificate (needs openssl on PATH)."""
    d = tempfile.mkdtemp()
    cert, key = os.path.join(d, "c.pem"), os.path.join(d, "k.pem")
    subprocess.run(["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-keyout", key, "-out", cert, "-days", "1",
                    "-subj", "/CN=localhost", "-addext", "subjectAltName=DNS:localhost"], check=True, capture_output=True)
    server = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    server.load_cert_chain(cert, key)
    server.set_alpn_protocols(["h2"])
    return server, ssl.create_default_context(cafile=cert)


def run_server(sock, server_ctx):
    async def main():
        server = await asyncio.start_server(handle, sock=sock, ssl=server_ctx)
        await server.serve_forever()
    asyncio.run(main())


# ---------------------------------------------------------------- client
async def client(port, client_ctx):
    try:
        import httpx2
    except ImportError:  # same script against upstream httpx/httpcore for comparison
        import httpx as httpx2
    base = f"https://localhost:{port}" if client_ctx else f"http://127.0.0.1:{port}"
    kw = {"verify": client_ctx} if client_ctx else {}
    async with httpx2.AsyncClient(http1=False, http2=True, timeout=10, **kw) as c:
        await c.get(f"{base}/")  # warm the single connection
        delays = [50] * 6 + [1500] * 4  # six fast responses arrive together; four stragglers stay open
        t0 = time.perf_counter()
        done = {}

        async def one(i):
            if "--post" in sys.argv:
                await c.post(f"{base}/?delay_ms={delays[i]}", json={"q": 1})
            else:
                await c.get(f"{base}/?delay_ms={delays[i]}")
            done[i] = (time.perf_counter() - t0) * 1000

        tasks = [asyncio.create_task(one(i)) for i in range(len(delays))]
        await asyncio.sleep(0.3)
        late = [i for i in range(6) if i not in done]
        await asyncio.gather(*tasks)
        for i in range(len(delays)):
            print(f"request {i}: server delay {delays[i]:>5} ms, completed at {done[i]:7.1f} ms")
        return late


def main():
    if "--no-fast-acquire" in sys.argv:
        import anyio
        try:
            import httpcore2._synchronization as s
        except ImportError:
            import httpcore._synchronization as s
        orig = s.AsyncLock.setup

        def setup(self):
            orig(self)
            if self._backend == "asyncio":
                self._anyio_lock = anyio.Lock(fast_acquire=False)
        s.AsyncLock.setup = setup
    sock = socket.socket()
    sock.bind(("127.0.0.1", 0))
    sock.listen()
    server_ctx, client_ctx = tls_contexts() if "--tls" in sys.argv else (None, None)
    threading.Thread(target=run_server, args=(sock, server_ctx), daemon=True).start()
    late = asyncio.run(client(sock.getsockname()[1], client_ctx))
    try:
        import httpcore2 as hc
    except ImportError:
        import httpcore as hc
    print(f"{hc.__name__} {hc.__version__}: fast responses not delivered by 300 ms: {late or 'none'}")
    sys.exit(1 if late else 0)


if __name__ == "__main__":
    main()

Results

Each cell counts the runs, out of 20, in which at least one fast response wasn't delivered within 300 ms. All rows come from one session on 2026-10-25, with a machine load average of about 8 to 9.

Client cleartext GET cleartext POST TLS GET TLS POST
httpx 0.28.1 / httpcore 1.0.9 19/20 20/20 0/20 2/20
httpx2 / httpcore2 2.16.0 19/20 19/20 20/20 20/20
httpx2 / httpcore2 main at 2fc91585813781bf0a1ebbf76d41136a0fe4a1f7 18/20 18/20 20/20 20/20

Over TLS, the baseline varied between sessions. On an idle machine (10 runs per shape), httpcore 1.0.9 and httpcore2 2.2.0 had no TLS stalls and httpcore2 2.13.1 stalled in every TLS run. In two earlier batches on a busier machine, httpcore 1.0.9 stalled in every TLS POST run (5/5 and 10/10).

The script's --no-fast-acquire flag forces fast_acquire=False on every asyncio AsyncLock, which covers the HTTP/2 connection's init, state, read and write locks but not the stream semaphore. On the idle machine it took httpcore2 2.13.1 from 10/10 to 1/10 for both TLS GET and TLS POST; cleartext was unchanged (9/10 and 10/10). #970 introduced fast_acquire. We haven't tested a read-lock-only change, the versions between 2.2.0 and 2.13.1, or this flag on main.

Where it waits

This comes from a TLS POST run on httpcore2 2.13.1. The stalled stream had [DataReceived, StreamEnded] queued (its headers were already consumed), and the slow streams had nothing queued. The task's coroutine chain ended in:

http2.py  _receive_response_body
http2.py  _receive_stream_event
http2.py  _receive_events            # async with self._read_lock
_synchronization.py  AsyncLock.__aenter__
anyio/_backends/_asyncio.py  Lock.acquire   # await fut

The task was a queued waiter. We didn't capture the lock owner or the other waiters.

Reading the source: anyio's Lock.release() hands the lock to waiters in FIFO order, and a stream's queue is only re-checked after the lock is acquired (inside _receive_events in 2.13.1, in _receive_stream_event on main). Queueing events for a stream doesn't wake its waiting task. A waiter ahead of it whose own stream has nothing queued can take the lock and block in a socket read until more data arrives on the connection, while this task's response sits in its queue.

#1163 moved that re-check into the callers and added one to _wait_for_outgoing_flow. The response path already had it, so this case is unchanged on main. We don't know why fast_acquire changes the TLS result.

Environment

  • macOS 26.6.2, arm64; CPython 3.14.3; OpenSSL 3.5.5
  • asyncio, AsyncClient(http1=False, http2=True)
  • Not tested: trio (AsyncLock uses trio.Lock there, so fast_acquire doesn't apply), the sync Client (it uses threading.Lock), Linux.
pip freeze (2.16.0 environment) 
anyio==4.13.0
h11==0.16.0
h2==4.4.1
hpack==4.2.0
httpcore2==2.16.0
httpx2==2.16.0
hyperframe==6.1.0
idna==3.20
truststore==0.10.5

Related

The HTTP/2 notes in #1289 describe the same reading model ("Whichever request holds _read_lock reads the socket for every stream") and a leader/follower reader with per-stream wakeups, which would deliver queued events without waiting for the lock.

I prepared this report with help from an AI assistant, which also ran the reproduction. I reviewed the report and the results.

Dominant language
Python
Stars
1.6k
Forks
103
Avg merge
1d 19h
Merged PRs (30d)
25

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pydantic/httpx2

All issues in pydantic/httpx2

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.