Let callers bound or skip the intermediate bodies read while following redirects and auth flows

Aperta
  • Qualcosa non va?
#1,249 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@rkuce ci sta già lavorando.

Dal 7/10/2026.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
42/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend-api-design, networking

Direzione di ricerca

Start in _client.py at _send_handling_redirects and _send_handling_auth, including the async implementations at the referenced locations, and reproduce the redirect plus DigestAuth flow with MockTransport. Compare the proposed handling of intermediate response bodies, history, and auth flows; done should establish an agreed opt-in or general limit and cover the behavior without breaking existing defaults.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Raised as an issue directly: the template points to "Ideas" and "Potential Issue" discussion categories, but this repo's discussions only have Announcements and Q&A.

When Client.send() follows redirects or runs a multi-step auth flow, it reads every intermediate response body in full before moving on. This happens even with stream=True:

  • _send_handling_redirects: response.read() on each redirect when follow_redirects=True (_client.py:1056, async :1895 in 2.12.0)
  • _send_handling_auth: response.read() on each response the auth flow answers, such as a DigestAuth 401 challenge or a token-refresh retry (_client.py:1020, async :1858)

Most callers never look at those bodies. But a client that talks to untrusted or misbehaving servers, and bounds memory by streaming and capping the final body, can't bound these reads. A server can send a huge body on a 3xx or 401 and the client buffers all of it before the caller sees anything.

Reproduction (httpx2 2.12.0):

import httpx2

pulled = 0


def huge_body():
    global pulled
    for _ in range(1024):
        pulled += 1024
        yield b"x" * 1024


def handler(request: httpx2.Request) -> httpx2.Response:
    if request.url.path == "/redirect":
        return httpx2.Response(302, headers={"location": "/digest"}, content=huge_body())
    if not request.headers.get("authorization", "").startswith("Digest "):
        challenge = 'Digest realm="api", nonce="abc", qop="auth"'
        return httpx2.Response(401, headers={"www-authenticate": challenge}, content=huge_body())
    return httpx2.Response(200, content=b"ok")


with httpx2.Client(
    transport=httpx2.MockTransport(handler),
    auth=httpx2.DigestAuth("user", "pass"),
    follow_redirects=True,
) as client:
    with client.stream("GET", "https://example.test/redirect") as response:
        print(response.status_code, f"{pulled // 1024} KiB of intermediate bodies read before the stream opened")

Output: 200 3072 KiB of intermediate bodies read before the stream opened. That is the 302, the 401, and the 302 again when DigestAuth re-sends the original request.

The only workaround today is to re-implement both loops outside the client: send with follow_redirects=False and a no-op auth, follow response.next_request, and drive auth.sync_auth_flow() by hand. httpware, a client library built on httpx2, does this to enforce its response body cap. It handles redirects in https://github.com/modern-python/httpware/pull/152 and auth flows in https://github.com/modern-python/httpware/pull/153. That works with public API, but it means copying redirect counting, history handling and the private _build_request_auth fallback to URL credentials. Those copies can drift from httpx2.

Possible directions, in increasing scope:

  1. Close intermediate bodies unread when nothing needs them. Redirect bodies are never used by httpx2. Auth-step bodies are only needed when auth.requires_response_body is set. The catch: response.history[i].content would raise ResponseNotRead, so this probably has to be opt-in.
  2. An opt-in client option for (1), keeping today's default.
  3. A general response-size limit on the client, applied to these reads and to read(). This is related to encode/httpx#1392 (#521 here), which asked for an easy way to bound body size.

Would any of these be welcome?

Lingua principale
Python
Stelle
1.6k
Fork
103
Merge medio
1g 19h
PR unite (30g)
25

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di pydantic/httpx2

Tutte le issue di pydantic/httpx2

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.