Request for clarification: URL parsing behavior in Requests (RFC 3986 vs WHATWG)

Open
  • Is something wrong?
#7,141 2 comments 2 reactions 0 assignees View on GitHub

@cici-czx is already working on this.

Since Jan 7, 2026.

  • #7307 by @nuglifeleoji — closed without merging

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Documentation
Clarity
Needs clarification
Activity status
Stale
Tech stack
python
Domain
backend

Research direction

Start by reviewing the cited issue #5886 and comparing the observed behavior with the RFC 3986, RFC 7230, and WHATWG URL references linked here. No source file or test is identified; the issue is done when the maintainers clarify which specification or compatibility behavior Requests intentionally follows.

Written by the indexing model from the issue text.

Description

Hello Requests maintainers,

Thank you for maintaining Requests and for the extensive discussions around URL parsing behavior over the years.

I am seeking clarification regarding which specification(s) Requests follows for URL parsing, particularly for HTTP(S) URLs, as I have observed behaviors that appear to align with different standards.

In previous discussions, Requests developers have noted that the library primarily follows RFC-based specifications. For example, in https://github.com/psf/requests/issues/5886, it is stated that:

RFC 3986 is the standard that Requests and urllib3 primarily use as a reference for their URL parsers.

This suggests that Requests' URL parsing logic is grounded in RFC 3986, with HTTP semantics further informed by RFC 7230.

However, I have observed a behavior in Requests that seems closer to the WHATWG URL Standard.

Specifically, when parsing HTTP URLs whose hostname contains a backslash (), Requests appears to treat \ as a delimiter equivalent to /, effectively separating the authority from the path.
This behavior is consistent with the WHATWG URL Standard's definition of the authority parsing state (see: https://url.spec.whatwg.org/#authority-state).

Image

By contrast: RFC 3986 treats \ as a disallowed character in URLs and does not define it as a structural delimiter. Many RFC-oriented parsers (e.g., urllib.parse.urlparse) either reject such URLs or treat \ as an ordinary character, rather than as a path/authority separator.

Image
Question

Could you please clarify:

Whether Requests intentionally references or aligns with the WHATWG URL Standard for any part of its URL parsing behavior (especially hostname handling)?

Or whether the observed behavior is still considered RFC 3986–based, but with pragmatic extensions or legacy compatibility considerations?

Understanding this would be very helpful for accurately characterizing Requests' parsing semantics and for reasoning about cross-component URL parsing consistency.

Thank you very much for your time and clarification.

Dominant language
Python
Stars
54.7k
Forks
10.8k
Avg merge
18m
Merged PRs (30d)
1

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from psf/requests

All issues in psf/requests

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.