pypa/pipx

Allow pipx to use sytem certificates

Open

#961 opened on Mar 29, 2023

View on GitHub
 (11 comments) (4 reactions) (0 assignees)Python (7,844 stars) (363 forks)batch import
enhancementhelp wanted

Description

Describe the bug pipx install fails due to corporate network/proxy with self-signed certificate.

From the log:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:997)'))': /simple/hatch/
ERROR: Could not find a version that satisfies the requirement hatch (from versions: none)
ERROR: No matching distribution found for hatch

Note that if I create a venv, things work as expected. So the system python/pip handles the (installed) self-signed cert:

syncer on  chore/setup-hatch [!?] 
❯ python3 -m venv .venv                                                             

syncer on  chore/setup-hatch [!?] took 2s 
❯ source .venv/bin/activate

syncer on  chore/setup-hatch [!?] via 🐍 v3.10.6 (.venv) 
❯ pip install hatch    
Collecting hatch
  Downloading hatch-1.6.3-py3-none-any.whl (84 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.0/85.0 KB 2.7 MB/s eta 0:00:00
Collecting platformdirs>=2.5.0
  Downloading platformdirs-3.2.0-py3-none-any.whl (14 kB)

**snip**

Collecting pycparser
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 KB 14.3 MB/s eta 0:00:00
Using legacy 'setup.py install' for pyperclip, since package 'wheel' is not installed.
Installing collected packages: rfc3986, pyperclip, ptyprocess, distlib, zipp, tomlkit, tomli-w, tomli, sniffio, shellingham, pygments, pycparser, pluggy, platformdirs, pexpect, pathspec, packaging, more-itertools, mdurl, jeepney, idna, h11, filelock, editables, click, certifi, virtualenv, userpath, markdown-it-py, jaraco.classes, importlib-metadata, hyperlink, hatchling, cffi, anyio, rich, httpcore, cryptography, SecretStorage, httpx, keyring, hatch
  Running setup.py install for pyperclip ... done
Successfully installed SecretStorage-3.3.3 anyio-3.6.2 certifi-2022.12.7 cffi-1.15.1 click-8.1.3 cryptography-40.0.1 distlib-0.3.6 editables-0.3 filelock-3.10.7 h11-0.14.0 hatch-1.6.3 hatchling-1.13.0 httpcore-0.16.3 httpx-0.23.3 hyperlink-21.0.0 idna-3.4 importlib-metadata-6.1.0 jaraco.classes-3.2.3 jeepney-0.8.0 keyring-23.13.1 markdown-it-py-2.2.0 mdurl-0.1.2 more-itertools-9.1.0 packaging-23.0 pathspec-0.11.1 pexpect-4.8.0 platformdirs-3.2.0 pluggy-1.0.0 ptyprocess-0.7.0 pycparser-2.21 pygments-2.14.0 pyperclip-1.8.2 rfc3986-1.5.0 rich-13.3.3 shellingham-1.5.0.post1 sniffio-1.3.0 tomli-2.0.1 tomli-w-1.0.0 tomlkit-0.11.7 userpath-1.8.0 virtualenv-20.21.0 zipp-3.15.0

syncer on  chore/setup-hatch [!?] via 🐍 v3.10.6 (.venv) took 17s 
❯

I'm using the zipapp (pipx.pyz), version (1.2.0)

My environment is Ubuntu 22.04 running in WSL on Windows 10.

Python is 3.10.6

How to reproduce

syncer on  chore/setup-hatch [!?] 
❯ python3 /home/jim/dev/afa/syncer/.deps/pipx/1.2.0/pipx.pyz --verbose install hatch
usage: pipx.pyz [-h] [--version] {install,uninject,inject,upgrade,upgrade-all,uninstall,uninstall-all,reinstall,reinstall-all,list,run,runpip,ensurepath,environment,completions} ...
pipx.pyz: error: unrecognized arguments: --verbose

To reproduce I guess you have to be in an environment that has a proxy with a self-signed cert.

Expected behavior pipx successfully installs package.

cmd_2023-03-29_13.44.16.log

Contributor guide

Tech stack
python
Domain
clisecurity
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Pythonpipx internalsSSL certificates
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
The issue reports that pipx fails with self signed certificates when using the zipapp, while direct pip works. The solution likely involves configuring pip's cert option within pipx or using the system certificate store. Investigate how pipx creates virtual environments and installs packages, particularly if it uses pip's cert option or environment variables like PIP CERT. Also check comments for any suggested approaches. Files to examine: pipx's pip interface module, venv creation, and any certificate related code.