Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

sre: the UCS2 and UCS4 variants of charset_loc_ignore are unreachable

Aberta
  • Algo errado?
#160,498 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
1-2 dias
Facilidade para iniciantes
42/100
Tipo de issue
Refatoração
Clareza
Razoavelmente clara
Status de atividade
Ativa
Stack de tecnologia
c, python
Domínio
backend, testing-qa

Direção de pesquisa

Start by reading Modules/_sre/sre_lib.h alongside the three character-width inclusions and the IN_LOC_IGNORE emission in Lib/re/_compiler.py; the issue provides coverage evidence and a reproduction to understand. Confirm whether the UCS2 and UCS4 functions are emitted despite being unreachable, then establish maintainer intent before making a change. Done means the project clarifies whether the duplicates are intentional or identifies a scoped change to generate only the bytes variant.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

extension-modules topic-regex type-refactor

Summary

Modules/_sre/sre_lib.h is included three times, once per character width, so
charset_loc_ignore is compiled as three functions. Only the bytes one can ever
run: the IN_LOC_IGNORE opcode that calls it is emitted only for LOCALE
patterns, and LOCALE is rejected for str patterns.

Evidence

Coverage from a --coverage build of main running the full test suite (552 test
files, 52,070 tests, ./python -m test -j4):

Function 'sre_ucs4_charset_loc_ignore'   Lines executed:0.00% of 39
Function 'sre_ucs2_charset_loc_ignore'   Lines executed:0.00% of 39
Function 'sre_ucs1_charset_loc_ignore'   Lines executed:87.18% of 39

78 lines that no input reaches.

Why they cannot be reached

IN_LOC_IGNORE is emitted in one place, Lib/re/_compiler.py:89:

if flags & SRE_FLAG_IGNORECASE and flags & SRE_FLAG_LOCALE:
    emit(IN_LOC_IGNORE)

and the parser refuses LOCALE for str patterns, Lib/re/_parser.py:1106:

raise ValueError("cannot use LOCALE flag with a str pattern")

The opcode is fixed when the pattern is compiled, so adding the flag later does
not help. Constructing a pattern directly through _sre.compile() with the
LOCALE flag set keeps whatever opcode the code array already carries, which for
a str pattern is IN_UNI_IGNORE, not IN_LOC_IGNORE:

import re, _sre
from re import _parser, _compiler
p = _parser.parse('[a-z]', re.IGNORECASE)
code = _compiler._code(p, re.IGNORECASE)
groupindex = p.state.groupdict
indexgroup = [None] * p.state.groups
for k, i in groupindex.items():
    indexgroup[i] = k
pat = _sre.compile('[a-z]', re.IGNORECASE | re.LOCALE, code,
                   p.state.groups - 1, groupindex, tuple(indexgroup))
pat.search('\U0001d11eA')   # matches, and still does not call the UCS4 variant

Matching a str against a bytes pattern is rejected, so there is no route from
any public or private API to the UCS2/UCS4 copies.

Question

Is the triplication intentional (the template is included and every function is
emitted regardless of reachability, and the duplicates are simply dropped by the
linker or the optimiser), or would it be worth guarding the emission so only the
bytes variant exists?

I have not prepared a patch: this is a question about intent first. If the answer
is that they should not exist, the fix is to wrap the function definition in
sre_lib.h so it is only generated for the bytes build, and the coverage build
then confirms it.

CPython versions tested on

CPython main branch (3.16.0a1+)

Operating systems tested on

Linux

Linguagem predominante
Python
Estrelas
77.7k
Forks
38.3k
Merge médio
12h 27min
PRs com merge (30d)
710

Preparar o ambiente

Abrir no Codespaces

Inicia o contêiner de desenvolvimento do projeto no navegador, com a sua própria conta do GitHub.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de python/cpython

Todas as issues de python/cpython

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.