A failed close in PoolByteStream latches _closed and keeps the pool slot

Open
  • Is something wrong?
#1,328 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@AdityaSreevatsaK is already working on this.

Since Nov 11, 2026.

  • #1330 by @AdityaSreevatsaK — open

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend, networking

Research direction

Read src/httpcore2/httpcore2/_async/connection_pool.py at PoolByteStream.aclose() and inspect the sync PoolByteStream.close() in src/httpcore2/httpcore2/_sync/connection_pool.py; follow the pattern through scripts/unasync.py. Start by reproducing the two close attempts with the supplied failing inner stream. Done when a failed close can be retried, the pool request remains until close succeeds, and the async and sync behavior agree.

Written by the indexing model from the issue text.

Description

This issue was posted by OpenCode on behalf of David.

Summary

PoolByteStream.aclose() sets self._closed = True before awaiting the inner stream's close, and removes the pool request only after that await returns. When the inner close raises, the flag stays set and the bookkeeping is skipped:

  • a second aclose() returns immediately, reporting success without retrying the inner close;
  • pool._requests keeps the pool request, so the slot it holds is never released.

The same shape is in the sync PoolByteStream.close() in src/httpcore2/httpcore2/_sync/connection_pool.py.

Cancellation does not trigger this: the inner close runs under AsyncShieldCancellation. The trigger is an ordinary exception from the inner close, such as a trace extension callback raising, or HTTP11Connection._response_closed() failing on the network stream.

Found while reviewing #1327, which makes Response.aclose() retryable after a failed close. That retry reaches this latch on the default transport, so the response reports a successful close that never happened.

Reproduction

AsyncConnectionPool with a pool request registered and an inner stream whose first aclose() raises:

from httpcore2 import AsyncConnectionPool, Request
from httpcore2._async.connection_pool import AsyncPoolRequest, PoolByteStream


class Inner:
    def __init__(self) -> None:
        self.calls = 0

    async def aclose(self) -> None:
        self.calls += 1
        if self.calls == 1:
            raise OSError('network close failed')


async def main() -> None:
    pool = AsyncConnectionPool()
    pool_request = AsyncPoolRequest(Request('GET', 'http://example.test/'))
    pool._requests.append(pool_request)
    stream = PoolByteStream(Inner(), pool_request, pool)
    for attempt in (1, 2):
        try:
            await stream.aclose()
            outcome = 'ok'
        except OSError as exc:
            outcome = repr(exc)
        print(f'attempt {attempt}: {outcome} pool_requests={len(pool._requests)}')

Output today:

attempt 1: OSError('network close failed') pool_requests=1
attempt 2: ok pool_requests=1

The second attempt reports success, the inner close was never retried, and the pool request is still there.

Suggested fix

Keep the latch, retry the inner close on a later call, and do the pool bookkeeping only once the close has returned. Run the async version through scripts/unasync.py for the sync twin.

async def aclose(self) -> None:
    if not self._closed:
        self._closed = True
        try:
            with AsyncShieldCancellation():
                if hasattr(self._stream, 'aclose'):
                    await self._stream.aclose()
        except BaseException:
            self._closed = False
            raise

        with self._pool._optional_thread_lock:
            self._pool._requests.remove(self._pool_request)
            closing = self._pool._assign_requests_to_connections()

        await self._pool._close_connections(closing)

Do not put the bookkeeping in a finally. It would remove the request on the failed attempt, and the retry would then fail with ValueError: list.remove(x): x not in list after the inner close succeeded. Verified: with the bookkeeping in a finally, attempt 1 raises OSError, attempt 2 raises that ValueError; with the version above, attempt 1 raises OSError and leaves pool_requests=1, attempt 2 retries the inner close and ends with pool_requests=0.

Dominant language
Python
Stars
1.6k
Forks
103
Avg merge
1d 19h
Merged PRs (30d)
25

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from pydantic/httpx2

All issues in pydantic/httpx2

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.