Unconsumed lookahead token used for ErrorRecoveryNode causes unsigned location length underflow

Aberta
#4,358 0 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
52/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
c, ruby
Domínio
backend, compilers

Direção de pesquisa

Start in src/prism.c and inspect the eight listed pm_error_recovery_node_create call sites, especially the parse_not recovery path described in the reproduction. Trace how each call handles parser->current and parser->previous, then check the parser tests or fuzzing coverage for {not %. Done means the reproducer no longer creates a node with a wrapped location length and the affected recovery paths are covered.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Fuzzing find on bdbd482dc.

When whitespace or skipped invalid bytes separate parser->previous.end from parser->current.start, the resulting ErrorRecoveryNode is given a start offset at parser->current.start while parser->previous.end remains at an earlier offset. When an enclosing construct then synthesizes a missing token via expect1 (which places the missing token at parser->previous.end), the parent node's end (parser->previous.end) is smaller than its child's start (parser->current.start), underflowing uint32_t location.length to ~4,294,967,295.

Reproduction

Simulation here: https://stevenjohnstone.github.io/prism-loupe/?version=v2.1.0&code=e25vdCAl.

✗ ./bin/parse -e "{not %"
Warnings:
[#<Prism::ParseWarning @type=:void_statement @message="possibly useless use of % in void context" @location=#<Prism::Location @start_offset=0 @length=6 start_line=1> @level=:verbose>]

Errors:
[#<Prism::ParseError @type=:invalid_percent_eof @message="unterminated quoted string meets end of file" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
 #<Prism::ParseError @type=:expect_lparen_after_not_other @message="expected a `(` after `not`" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
 #<Prism::ParseError @type=:hash_rocket @message="expected a `=>` between the hash key and value" @location=#<Prism::Location @start_offset=4 @length=0 start_line=1> @level=:syntax>,
 #<Prism::ParseError @type=:hash_value @message="unexpected '%'; expected a value in the hash literal" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
 #<Prism::ParseError @type=:hash_term @message="expected a `}` to close the hash literal" @location=#<Prism::Location @start_offset=0 @length=1 start_line=1> @level=:syntax>,
 #<Prism::ParseError @type=:expect_expression_after_operator @message="unexpected end-of-input; expected an expression after the operator" @location=#<Prism::Location @start_offset=6 @length=0 start_line=1> @level=:syntax>,
 #<Prism::ParseError @type=:unexpected_token_close_context @message="unexpected end-of-input, assuming it is closing the parent top level context" @location=#<Prism::Location @start_offset=6 @length=0 start_line=1> @level=:syntax>]

AST:
@ ProgramNode (location: (1,0)-(1,6))
├── flags: ∅
├── locals: []
└── statements:
    @ StatementsNode (location: (1,0)-(1,6))
    ├── flags: ∅
    └── body: (length: 1)
        └── @ CallNode (location: (1,0)-(1,6))
            ├── flags: newline
            ├── receiver:
            │   @ HashNode (location: (1,0)-(1,4))
            │   ├── flags: ∅
            │   ├── opening_loc: (1,0)-(1,1) = "{"
            │   ├── elements: (length: 1)
            │   │   └── @ AssocNode (location: (1,5)-(1,4294967300))
            │   │       ├── flags: ∅
            │   │       ├── key:
            │   │       │   @ ErrorRecoveryNode (location: (1,5)-(1,6))
            │   │       │   ├── flags: ∅
            │   │       │   └── unexpected: ∅
            │   │       ├── value:
            │   │       │   @ ErrorRecoveryNode (location: (1,4)-(1,4))
            │   │       │   ├── flags: ∅
            │   │       │   └── unexpected: ∅
            │   │       └── operator_loc: ∅
            │   └── closing_loc: (1,4)-(1,4) = ""
            ├── call_operator_loc: ∅
            ├── name: :%
            ├── message_loc: (1,5)-(1,6) = "%"
            ├── opening_loc: ∅
            ├── arguments:
            │   @ ArgumentsNode (location: (1,5)-(1,6))
            │   ├── flags: ∅
            │   └── arguments: (length: 1)
            │       └── @ ErrorRecoveryNode (location: (1,5)-(1,6))
            │           ├── flags: ∅
            │           └── unexpected: ∅
            ├── closing_loc: ∅
            ├── equal_loc: ∅
            └── block: ∅

Byte offsets in "{not %" (6 bytes):

  • {: 0..1
  • not: 1..4 (parser->previous after consuming not, so parser->previous.end == 4)
  • ' ': 4..5
  • %: 5..6 (invalid string delimiter at EOF; skipped to offset 6, so parser->current is PM_TOKEN_EOF at 6..6)

In Prism.parse("{not %"), the resulting AssocNode has location.start = 6 and end = 4, so location.length (U32(end - PM_NODE_START(key))) wraps modulo $2^{32}$ to 4294967294 (0xFFFFFFFE).

Root Cause

  1. In "{not %", { opens a hash literal (parse_hash).
  2. not is parsed as the start of a hash key expression (parse_not), setting parser->previous to not (offsets 1..4).
  3. The lookahead token after the space (offset 4..5) and invalid % (offset 5..6) is PM_TOKEN_EOF at 6..6.
  4. parse_not calls parse_expression for its operand, which hits context_recoverable(parser, context):
    if (context_recoverable(parser, context) || depth == PRISM_DEPTH_MAXIMUM) {
        ...
        return UP(pm_error_recovery_node_create(parser, PM_TOKEN_START(parser, &parser->current), PM_TOKEN_LENGTH(&parser->current)));
    }
    
    Because &parser->current (6..6) is passed without consuming parser->current, the returned node has start = 6 while parser->previous is still not (1..4, end = 4).
  5. When parse_hash then calls expect1 for => (which synthesises the missing operator at parser->previous.end == 4) and constructs an AssocNode whose start is 6 and whose end is 4, U32(end - PM_NODE_START(key)) underflows uint32_t.

The same pattern occurs in 8 places in src/prism.c where pm_error_recovery_node_create is called without consuming parser->current:

  1. parse_string_part (default: branch of PM_TOKEN_EMBVAR after expect1(parser, PM_TOKEN_IDENTIFIER, PM_ERR_EMBVAR_INVALID))
  2. parse_undef_argument (default: branch)
  3. parse_alias_argument (default: branch)
  4. parse_pattern_primitive (default: branch)
  5. parse_pattern_primitives (default: branch)
  6. parse_pattern (context_recoverable / depth == PRISM_DEPTH_MAXIMUM branch)
  7. parse_not (context_recoverable / depth == PRISM_DEPTH_MAXIMUM branch)
  8. parse_expression (context_recoverable / depth == PRISM_DEPTH_MAXIMUM branch)
Linguagem predominante
C
Estrelas
1k
Forks
202
Merge médio
5h 55min
PRs com merge (30d)
85

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de ruby/prism

Todas as issues de ruby/prism

Issues semelhantes

Mais issues de C

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.