Document how to add a bytecode specialization

Open
#1,286 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
74/100
Issue type
Documentation
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
compilers, documentation

Research direction

Use PR 116385 and the nine-step outline as the source material. Start by reviewing the referenced changes and the named files—Python/bytecodes.c, Include/internal/pycore_code.h, Python/specialize.c, Lib/opcode.py, and Include/internal/pycore_magic_number.h—then choose the devguide or CPython documentation location. Done means a newcomer can follow the specialization workflow and verify it with make regen-all or build.bat --regen.

Written by the indexing model from the issue text.

Description

good first issue help wanted

How to add a new bytecode specialization is knowledge only a few of us remember right now. We should document this somewhere (devguide or CPython, I don't mind). Hopefully a new contributor can pick this up!

There reference is this PR: https://github.com/python/cpython/pull/116385/files

Assuming you found an instruction that serves as a good specialization candidate. Let's use the example of CONTAINS_OP:

  1. Set the original CONTAINS_OP to a uop by changing it from CONTAINS_OP to _CONTAINS_OP in Python/bytecodes.c. The instruction definition should also change from inst to op.
  2. Add the uop that calls a specializing function. In this case _SPECIALIZE_CONTAINS_OP.
  3. The original CONTAINS_OP is now a new macro consisting of _SPECIALIZE_CONTAINS_OP + _CONTAINS_OP.
  4. Define the cache structure in Include/internal/pycore_code.h. At the very least, a 16-bit counter is needed.
  5. Write the specializing function itself in Python/specialize.c. Refer to any other function in that file for the format.
  6. Remember to update operation stats by calling add_stat_dict in Python/specialize.c.
  7. Add the cache layout in Lib/opcode.py so that Python's dis module will know how to represent it properly.
  8. Bump magic number in Include/internal/pycore_magic_number.h.
  9. Run make regen-all on *nix or build.bat --regen on PC.
Dominant language
Python
Stars
2.1k
Forks
1k
Avg merge
2d 12h
Merged PRs (30d)
12

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from python/devguide

All issues in python/devguide

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.