Incorrectly strips trailing dot from absolute hostnames, causing TooManyRedirects

Abierto
  • ¿Algo no está bien?
#6,131 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
38/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
python
Área
api, networking

Línea de trabajo

Comienza con requests/api.py y requests/sessions.py, especialmente con las rutas de request y redirect mostradas en el traceback, y sigue después cómo urllib3 construye el SNI de TLS y el encabezado Host. Reproduce el problema con requests.get("https://pyropus.ca./") y verifica que el encabezado Host conserve el punto final mientras la solicitud se completa sin un bucle de redirección.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

requests appears to be incorrectly stripping the trailing dot on absolute hostnames (i.e. explicitly marked as in the root DNS namespace) in URLs when constructing the Host: request header, like pyropus.ca.. This causes redirect loops when the site chooses the absolute hostname as canonical and redirects requests without the trailing dot.

The example site redirects requests with a host header of Host: pyropus.ca to the same URL with the host name changed to the canonical absolute version:

HTTP/1.1 301 Moved Permanently
[...]
Location: https://pyropus.ca./

requests then strips the dot and resends the request, resulting in a redirect loop.

The trailing dot needs to be stripped from the SNI header for https requests (required by TLS/etc spec) but it should not be stripped from the Host: header value.

When fixing the SNI trailing dot issue, curl had this bug because they changed it to also affect the Host: header value, but they've fixed it by reverting that part of the change:
https://github.com/curl/curl/issues/8290

Many other common user agents that I've tested handle this properly:

  • all GUI browers, to my knowledge, preserve the absolute domain name in the Host: header - I've tested Firefox, Chromium, Vivaldi, Konqueror, and Safari
  • wget also preserves the absolute domain name on redirect URLs
  • curl is now fixed and handles it

A few less common ones have the same buggy behaviour, or did -- I haven't re-checked:

  • lynx, links, and elinks handle it like curl, erroring out on a redirect loop

Expected Result

Successful 200 response with content.

Actual Result

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File ".../test/.venv/lib/python3.9/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File ".../test/.venv/lib/python3.9/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File ".../test/.venv/lib/python3.9/site-packages/requests/sessions.py", line 529, in request
    resp = self.send(prep, **send_kwargs)
  File ".../test/.venv/lib/python3.9/site-packages/requests/sessions.py", line 667, in send
    history = [resp for resp in gen]
  File ".../test/.venv/lib/python3.9/site-packages/requests/sessions.py", line 667, in <listcomp>
    history = [resp for resp in gen]
  File ".../test/.venv/lib/python3.9/site-packages/requests/sessions.py", line 166, in resolve_redirects
    raise TooManyRedirects('Exceeded {} redirects.'.format(self.max_redirects), response=resp)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

Reproduction Steps

import requests

requests.get("https://pyropus.ca./")

System Information

$ python -m requests.help

{
  "chardet": {
    "version": null
  },
  "charset_normalizer": {
    "version": "2.0.12"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "3.3"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.9.2"
  },
  "platform": {
    "release": "5.17.5",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.27.1"
  },
  "system_ssl": {
    "version": "101010bf"
  },
  "urllib3": {
    "version": "1.26.9"
  },
  "using_charset_normalizer": true,
  "using_pyopenssl": false
}

Edit: added a couple words to the summary to clarify.

Lenguaje dominante
Python
Estrellas
54.7k
Forks
10.8k
Merge medio
18 min
PR fusionados (30 d)
1

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de psf/requests

Todos los issues de psf/requests

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.