RuntimeError "aclose(): asynchronous generator is already running" from nested stream generators at event loop shutdown

Open
  • Is something wrong?
#1,280 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@Kludex is already working on this.

Since Oct 20, 2026.

  • #1281 by @Kludex — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
backend, networking

Research direction

Start with the reported cleanup paths in httpx2/_models.py (Response.aiter_bytes and Response.aiter_raw), then trace safe_async_iterate and the stream implementations mentioned in the issue. Run the reproduction with several open streams during event loop shutdown and compare against httpx 0.28.1. Done means shutdown closes the streams without aclose(): asynchronous generator is already running; an open linked pull request indicates work is already underway.

Written by the indexing model from the issue text.

Description

Drafted by Claude (AI assistant) in discussion with @Seluj78, who reviewed and is filing it.

Summary

When an event loop shuts down while streamed responses are still open, loop.shutdown_asyncgens() closes every live async generator concurrently. httpx2's streaming generators are nested with contextlib.aclosing(...) / safe_async_iterate(...), so an outer generator's aclose() tries to close an inner generator whose own aclose() is already in flight, and raises:

an error occurred during closing of asynchronous generator <async_generator object Response.aiter_bytes at 0x...>
asyncgen: <async_generator object Response.aiter_bytes at 0x...>
Traceback (most recent call last):
  File ".../httpx2/_models.py", line 996, in aiter_bytes
    yield chunk
GeneratorExit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File ".../httpx2/_models.py", line 992, in aiter_bytes
    async with contextlib.aclosing(self.aiter_raw()) as raw_stream:
  File ".../contextlib.py", line 386, in __aexit__
    await self.thing.aclose()
RuntimeError: aclose(): asynchronous generator is already running

The same error is reported for Response.aiter_raw, PoolByteStream.__aiter__ (async with safe_async_iterate(self._stream)) and HTTP11ConnectionByteStream.__aiter__.

httpx 0.28.1 with the same script reports nothing, so this looks like a regression from the added aclosing nesting. We hit it in production after upgrading openai to a release that depends on httpx2: every worker exit with in-flight streams logs a handful of these errors via asyncio's exception handler.

Reproduction
import asyncio

import httpx2


async def handle(reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
    while (await reader.readline()) not in (b"\r\n", b""):
        pass
    writer.write(b"HTTP/1.1 200 OK\r\ntransfer-encoding: chunked\r\n\r\n5\r\nhello\r\n")
    await writer.drain()
    await asyncio.sleep(3600)  # keep the body open


streams = []


async def main() -> None:
    server = await asyncio.start_server(handle, "127.0.0.1", 0)
    url = f"http://127.0.0.1:{server.sockets[0].getsockname()[1]}/"
    client = httpx2.AsyncClient()
    for _ in range(5):
        response = await client.send(client.build_request("GET", url), stream=True)
        chunks = response.aiter_bytes()
        await anext(chunks)
        streams.append(chunks)  # still open when asyncio.run() finalizes async generators


asyncio.run(main())
Environment Result
httpx2 2.14.0, CPython 3.11.13 5x RuntimeError: aclose(): asynchronous generator is already running
httpx2 2.14.0, CPython 3.13.7 5x same
httpx 0.28.1 (same script, httpx2 → httpx), CPython 3.11.13 no errors

With a single open stream there is usually no error; it needs several generators being finalized in the same gather().

Expected

Shutting down with open streams should close them quietly, as httpx 0.28 does.

Possible directions
  • Make the nested aclose() tolerant of the inner generator already being closed by someone else (e.g. skip/suppress when the inner generator is already running or finished during GeneratorExit handling).
  • Avoid nesting generator ownership (have the outer generator iterate the inner stream without aclosing, and rely on Response.aclose() / the stream's own aclose() for cleanup).
Dominant language
Python
Stars
1.6k
Forks
103
Avg merge
1d 19h
Merged PRs (30d)
25

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pydantic/httpx2

All issues in pydantic/httpx2

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.