Cannot install development environment on Python 3.12

Start by reviewing the dependency information in poetry.lock and reproducing the failure with poetry install or the pip wheel command shown in the report. Done means the development environment installs successfully on Python 3.12 with a compatible PyYAML version and an updated lock file.

Bug report

Cannot install development environment on Python 3.12.

Repro steps

└➜ poetry install
Installing dependencies from lock file

Package operations: 114 installs, 0 updates, 0 removals

  - Installing smmap (5.0.0)
  - Installing gitdb (4.0.9)
  - Installing mccabe (0.6.1)
  - Installing pbr (5.8.1)
  - Installing pycodestyle (2.7.0)
  - Installing pyflakes (2.3.1)
  - Installing pyparsing (3.0.8)
  - Installing attrs (21.4.0)
  - Installing certifi (2022.12.7)
  - Installing charset-normalizer (2.0.12)
  - Installing docutils (0.19)
  - Installing flake8 (3.9.2)
  - Installing gitpython (3.1.27)
  - Installing idna (3.3)
  - Installing markupsafe (2.1.1)
  - Installing packaging (21.3)
  - Installing pytz (2022.1)
  - Installing pyyaml (6.0): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke get_requires_for_build_wheel

  running egg_info
  writing lib/PyYAML.egg-info/PKG-INFO
  writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
  writing top-level names to lib/PyYAML.egg-info/top_level.txt
  Traceback (most recent call last):
    File "/home/maxim/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
      main()
    File "/home/maxim/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
      json_out["return_val"] = hook(**hook_input["kwargs"])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/maxim/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=[])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
      self.run_setup()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 320, in run_setup
      exec(code, locals())
    File "<string>", line 288, in <module>
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 183, in setup
      return run_commands(dist)
             ^^^^^^^^^^^^^^^^^^
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
      dist.run_commands()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
      self.run_command(cmd)
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/dist.py", line 995, in run_command
      super().run_command(command)
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
      cmd_obj.run()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 313, in run
      self.find_sources()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in find_sources
      mm.run()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 544, in run
      self.add_defaults()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 582, in add_defaults
      sdist.add_defaults(self)
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/command/sdist.py", line 109, in add_defaults
      super().add_defaults()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 238, in add_defaults
      self._add_defaults_ext()
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 323, in _add_defaults_ext
      self.filelist.extend(build_ext.get_source_files())
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<string>", line 204, in get_source_files
    File "/tmp/tmps0g2qzy0/.venv/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
      raise AttributeError(attr)
  AttributeError: cython_sources


  at ~/.local/share/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/chef.py:164 in _prepare
      160│
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│
      163│             if error is not None:
    → 164│                 raise error from None
      165│
      166│             return path
      167│
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pyyaml (6.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pyyaml (==6.0)"'.

  - Installing six (1.16.0)
  - Installing snowballstemmer (2.2.0)
  - Installing stevedore (3.5.0)
  - Installing urllib3 (1.26.9)

System info

WSL2, Windows 11

└➜ uname -a
Linux 5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

└➜ python --version
Python 3.12.3

└➜ poetry --version
Poetry (version 1.8.5)

I suppose the problem is related to the fact that poetry.lock requires pyyaml == 6.0, but looks like there is a problem to build pyyaml < 6.0.1 for python 3.11 and newer (https://github.com/yaml/pyyaml/issues/736).

Proposed solution

1. Maybe bump pyyaml dependency to the latest 6.0.2?
2. Update poetry.lock for python 3.12 and newer (for pyyaml package wheels are listed for 3.11 and older)