Transport adapters not honored with proxy

Open
  • Is something wrong?
#6,205 0 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
42/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
networking

Research direction

Start by reproducing the issue with the custom HTTPAdapter, the SOCKS proxy, and the TLS 1.2 setting shown in the report. Compare the direct and proxy request paths, then verify that the configured TLS version is honored through the proxy while preserving the existing request behavior.

Written by the indexing model from the issue text.

Description

hi,

making a poor socks server at localhost on port 8001 with ssh/sshd running at localhost :

@term1$ ssh -N -D localhost:8001 localhost

making a self-signed certificate for testing and debuging negociations on server side with s_server@openssl :

@term2$ openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes 
        while :
        do
            echo -e "HTTP/1.0 200 OK\nContent-Length: 0\n\n" |
            openssl s_server -4 -cert cert.pem -key key.pem -accept 4433 -msg
        done

getting requests informations :

@term3$ python -m requests.help
{
  "chardet": {
    "version": "3.0.4"
  },
  "charset_normalizer": {
    "version": "2.1.0"
  },
  "cryptography": {
    "version": "2.8"
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.10"
  },
  "platform": {
    "release": "5.10.0-1057-oem",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "1010106f",
    "version": "19.0.0"
  },
  "requests": {
    "version": "2.28.1"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.25.8"
  },
  "using_charset_normalizer": false,
  "using_pyopenssl": true
}

running requests simply :

@term3$ python <<~~~
import requests
url = "https://localhost:4433/"
session = requests.Session()
response = session.head(url=url, verify=False)
~~~

TLS1.3 by default :

@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...

running requests with transport adapter only :

@term3$ python <<~~~
import ssl
import requests
from requests.adapters import HTTPAdapter
from urllib3.poolmanager import PoolManager
class TestHTTPAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(
            num_pools=connections,
            maxsize=maxsize,
            block=block,
            ssl_version=ssl.PROTOCOL_TLSv1_2
        )
url = "https://localhost:4433/"
session = requests.Session()
session.mount(url, TestHTTPAdapter())
response = session.head(url=url, verify=False)
~~~

TLS1.2 is OK 🙂 :

@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...

running requests with transport adapter and socks proxy :

@term3$ python <<~~~
import ssl
import requests
from requests.adapters import HTTPAdapter
from urllib3.poolmanager import PoolManager
class TestHTTPAdapter(HTTPAdapter):
    def init_poolmanager(self, connections, maxsize, block=False):
        self.poolmanager = PoolManager(
            num_pools=connections,
            maxsize=maxsize,
            block=block,
            ssl_version=ssl.PROTOCOL_TLSv1_2
        )
url = "https://localhost:4433/"
session = requests.Session()
session.mount(url, TestHTTPAdapter())
proxies={"https":"socks5://localhost:8001"}
response = session.head(url=url, proxies=proxies, verify=False)
~~~

TLS1.2 is KO 🙁 :

@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...

requests has the same behavior whether pyopenssl is present/used or not.
HTTP/HTTPS proxies not tested : sorry.

regards, lacsaP.

note that httpie works as expected :

@term4$ https --verify no https://localhost:4433
HTTP/1.0 200 OK
Content-Length: 0

@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...

@term4$ https --ssl tls1.2 --verify no https://localhost:4433
HTTP/1.0 200 OK
Content-Length: 0

@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...

@term4$ https --ssl tls1.2 --verify no --proxy https:socks5://localhost:8001 https://localhost:4433
HTTP/1.0 200 OK
Content-Length: 0

@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...

as well as urllib3 :

@term5$ python <<~~~
import urllib3
https = urllib3.PoolManager(cert_reqs='CERT_NONE')
r = https.request('GET', 'https://localhost:4433')
~~~

@term2$ ...
        <<< TLS 1.3, Handshake [length 0034], Finished
        ...
        >>> TLS 1.3, Handshake [length 00e9], NewSessionTicket
        ...

@term5$ python <<~~~
import ssl, urllib3
https = urllib3.PoolManager(cert_reqs='CERT_NONE', ssl_version=ssl.PROTOCOL_TLSv1_2)
r = https.request('GET', 'https://localhost:4433')
~~~

@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...

@term5$ python <<~~~
import ssl
from urllib3.contrib.socks import SOCKSProxyManager
socks = SOCKSProxyManager('socks5h://localhost:8001/', cert_reqs='CERT_NONE', ssl_version=ssl.PROTOCOL_TLSv1_2)
r = socks.request('GET', 'https://localhost:4433')
~~~

@term2$ ...
        >>> TLS 1.2, Handshake [length 0010], Finished
        ...
Dominant language
Python
Stars
54.7k
Forks
10.8k
Avg merge
18m
Merged PRs (30d)
1

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from psf/requests

All issues in psf/requests

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.