Python: allow building the sdist without the upb C extension (pure-Python only)
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- build-system
Research direction
Start by reading python/dist/setup.py, which is copied into the source distribution and declares the google._upb._message extension. Add the proposed environment variable opt-out there, then build the sdist with PROTOBUF_PYTHON_NO_EXTENSIONS=1 and confirm it succeeds without a compiler and produces a py3-none-any wheel containing the same google/protobuf modules as the published pure-Python wheel.
Written by the indexing model from the issue text.
Description
Since 4.21, the sdist's setup.py (copied from python/dist/setup.py) always declares the google._upb._message extension, so installing protobuf from source requires a C compiler. There is no way to opt out at build time. Protobuf 3.x built pure-Python by default and only compiled the C++ backend with --cpp_implementation.
At runtime the pure-Python backend is still fully supported: api_implementation falls back to it automatically when google._upb._message is missing. Only the build is missing an opt-out.
Use case: reproducible builds that install all dependencies from sdists (pip install --no-binary :all:) where no C compiler is available. Today the only option there is the py3-none-any wheel, which forces such projects to special-case protobuf in their source-only pipeline.
Proposal: an explicit environment variable that skips the extension:
ext_modules = [
Extension(
'google._upb._message',
...
)
]
if os.environ.get('PROTOBUF_PYTHON_NO_EXTENSIONS'):
ext_modules = []
setup(
...
ext_modules=ext_modules,
...
)
With this change, PROTOBUF_PYTHON_NO_EXTENSIONS=1 pip wheel --no-binary :all: protobuf==7.36.2 builds without a compiler and produces a py3-none-any wheel with the same google/protobuf modules as the published pure-Python wheel.
An explicit opt-out is preferable to Extension(optional=True): a silent fallback would make the build output depend on whether a compiler happens to be present, which breaks reproducible builds.
- Dominant language
- C++
- Stars
- 72.2k
- Forks
- 16.4k
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 342
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from protocolbuffers/protobuf
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
protocolbuffers/protobuf#31768 ·
Maintainers usually reply within 1 day
-
google-protobuf 0.36.2-release fails to compile for ARMv7: @progbits rejected in global_asm!Possibly taken @farisbasha claimed this 1 day ago. Openbug help wanted rust
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
protocolbuffers/protobuf#31619 · 3 comments ·
Maintainers usually reply within 1 day
-
Null pointer passed to memcpy in lupb_decode() when the second argument is nilPossibly taken A pull request linked to this issue is open or already merged. Openbug help wanted lua
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
protocolbuffers/protobuf#31392 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
protocolbuffers/protobuf#31032 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
protocolbuffers/protobuf#31031 ·
Maintainers usually reply within 1 day
All issues in protocolbuffers/protobuf
Similar issues
-
Needs: Triage :mag: Platform: Android
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
react/react-native#60167 ·
Maintainers usually reply within 2 days
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 63/100
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 2 days
-
tests: ofx_render_test doesn't compile against FFmpeg 4.x (const AVCodec* to av_find_best_stream)Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NatronGitHub/openfx-io#221 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
subsurface/subsurface#5145 ·
Maintainers usually reply within 2 days