Unconsumed lookahead token used for ErrorRecoveryNode causes unsigned location length underflow
Mantenedores costumam responder em até 1 dia
Ninguém assumiu esta issue ainda.
Avaliação
- Dificuldade
- 4/5
- Tempo estimado
- 3-5 dias
- Facilidade para iniciantes
- 52/100
Direção de pesquisa
Start in src/prism.c and inspect the eight listed pm_error_recovery_node_create call sites, especially the parse_not recovery path described in the reproduction. Trace how each call handles parser->current and parser->previous, then check the parser tests or fuzzing coverage for {not %. Done means the reproducer no longer creates a node with a wrapped location length and the affected recovery paths are covered.
Escrita pelo modelo de indexação a partir do texto da issue.
Descrição
Fuzzing find on bdbd482dc.
When whitespace or skipped invalid bytes separate parser->previous.end from parser->current.start, the resulting ErrorRecoveryNode is given a start offset at parser->current.start while parser->previous.end remains at an earlier offset. When an enclosing construct then synthesizes a missing token via expect1 (which places the missing token at parser->previous.end), the parent node's end (parser->previous.end) is smaller than its child's start (parser->current.start), underflowing uint32_t location.length to ~4,294,967,295.
Reproduction
Simulation here: https://stevenjohnstone.github.io/prism-loupe/?version=v2.1.0&code=e25vdCAl.
✗ ./bin/parse -e "{not %"
Warnings:
[#<Prism::ParseWarning @type=:void_statement @message="possibly useless use of % in void context" @location=#<Prism::Location @start_offset=0 @length=6 start_line=1> @level=:verbose>]
Errors:
[#<Prism::ParseError @type=:invalid_percent_eof @message="unterminated quoted string meets end of file" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:expect_lparen_after_not_other @message="expected a `(` after `not`" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:hash_rocket @message="expected a `=>` between the hash key and value" @location=#<Prism::Location @start_offset=4 @length=0 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:hash_value @message="unexpected '%'; expected a value in the hash literal" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:hash_term @message="expected a `}` to close the hash literal" @location=#<Prism::Location @start_offset=0 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:expect_expression_after_operator @message="unexpected end-of-input; expected an expression after the operator" @location=#<Prism::Location @start_offset=6 @length=0 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:unexpected_token_close_context @message="unexpected end-of-input, assuming it is closing the parent top level context" @location=#<Prism::Location @start_offset=6 @length=0 start_line=1> @level=:syntax>]
AST:
@ ProgramNode (location: (1,0)-(1,6))
├── flags: ∅
├── locals: []
└── statements:
@ StatementsNode (location: (1,0)-(1,6))
├── flags: ∅
└── body: (length: 1)
└── @ CallNode (location: (1,0)-(1,6))
├── flags: newline
├── receiver:
│ @ HashNode (location: (1,0)-(1,4))
│ ├── flags: ∅
│ ├── opening_loc: (1,0)-(1,1) = "{"
│ ├── elements: (length: 1)
│ │ └── @ AssocNode (location: (1,5)-(1,4294967300))
│ │ ├── flags: ∅
│ │ ├── key:
│ │ │ @ ErrorRecoveryNode (location: (1,5)-(1,6))
│ │ │ ├── flags: ∅
│ │ │ └── unexpected: ∅
│ │ ├── value:
│ │ │ @ ErrorRecoveryNode (location: (1,4)-(1,4))
│ │ │ ├── flags: ∅
│ │ │ └── unexpected: ∅
│ │ └── operator_loc: ∅
│ └── closing_loc: (1,4)-(1,4) = ""
├── call_operator_loc: ∅
├── name: :%
├── message_loc: (1,5)-(1,6) = "%"
├── opening_loc: ∅
├── arguments:
│ @ ArgumentsNode (location: (1,5)-(1,6))
│ ├── flags: ∅
│ └── arguments: (length: 1)
│ └── @ ErrorRecoveryNode (location: (1,5)-(1,6))
│ ├── flags: ∅
│ └── unexpected: ∅
├── closing_loc: ∅
├── equal_loc: ∅
└── block: ∅
Byte offsets in "{not %" (6 bytes):
{:0..1not:1..4(parser->previousafter consumingnot, soparser->previous.end == 4)' ':4..5%:5..6(invalid string delimiter at EOF; skipped to offset6, soparser->currentisPM_TOKEN_EOFat6..6)
In Prism.parse("{not %"), the resulting AssocNode has location.start = 6 and end = 4, so location.length (U32(end - PM_NODE_START(key))) wraps modulo $2^{32}$ to 4294967294 (0xFFFFFFFE).
Root Cause
- In
"{not %",{opens a hash literal (parse_hash). notis parsed as the start of a hash key expression (parse_not), settingparser->previoustonot(offsets1..4).- The lookahead token after the space (
offset 4..5) and invalid%(offset 5..6) isPM_TOKEN_EOFat6..6. parse_notcallsparse_expressionfor its operand, which hitscontext_recoverable(parser, context):
Becauseif (context_recoverable(parser, context) || depth == PRISM_DEPTH_MAXIMUM) { ... return UP(pm_error_recovery_node_create(parser, PM_TOKEN_START(parser, &parser->current), PM_TOKEN_LENGTH(&parser->current))); }&parser->current(6..6) is passed without consumingparser->current, the returned node hasstart = 6whileparser->previousis stillnot(1..4,end = 4).- When
parse_hashthen callsexpect1for=>(which synthesises the missing operator atparser->previous.end == 4) and constructs anAssocNodewhosestartis6and whoseendis4,U32(end - PM_NODE_START(key))underflowsuint32_t.
The same pattern occurs in 8 places in src/prism.c where pm_error_recovery_node_create is called without consuming parser->current:
parse_string_part(default:branch ofPM_TOKEN_EMBVARafterexpect1(parser, PM_TOKEN_IDENTIFIER, PM_ERR_EMBVAR_INVALID))parse_undef_argument(default:branch)parse_alias_argument(default:branch)parse_pattern_primitive(default:branch)parse_pattern_primitives(default:branch)parse_pattern(context_recoverable/depth == PRISM_DEPTH_MAXIMUMbranch)parse_not(context_recoverable/depth == PRISM_DEPTH_MAXIMUMbranch)parse_expression(context_recoverable/depth == PRISM_DEPTH_MAXIMUMbranch)
- Linguagem predominante
- C
- Estrelas
- 1k
- Forks
- 202
- Merge médio
- 5h 55min
- PRs com merge (30d)
- 85
Preparar o ambiente
- Sem Dockerfile nem arquivo Docker Compose
- Sem modelo de pull request
- Ler o guia de contribuição
Primeiros passos
- Leia a issue inteira e depois o guia de contribuição do projeto.
- Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
- Faça um fork do repositório e trabalhe em uma branch.
- Abra um pull request que referencie o número da issue.
Mais de ruby/prism
-
Prism::Translation::Parser raises NoMethodError for a method with a &nil parameter and a bodyTalvez já em andamento @bbatsov assumiu hoje. Aberta
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 82/100
Mantenedores costumam responder em até 1 dia
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 74/100
Mantenedores costumam responder em até 1 dia
-
Heredoc and outer-context recovery at recursion depth / stack budget limit causes location underflowAberta
Dificuldade 4/5 3-5 dias Facilidade para iniciantes 43/100
Mantenedores costumam responder em até 1 dia
-
Dificuldade 4/5 3-5 dias Facilidade para iniciantes 45/100
Mantenedores costumam responder em até 1 dia
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 1/100
Mantenedores costumam responder em até 1 dia
Issues semelhantes
-
runtime: lf_f128_mod is documented as fmodq (truncating) but implements floor-based modulo; unusedAbertainternal correctness low priority rt_lib
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 72/100
Mantenedores costumam responder em até 1 dia
-
Unreachable code warningAbertabug
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 70/100
SanderMertens/flecs#2395 ·
Mantenedores costumam responder em até 1 dia
-
Memory leak: "Add to Playlist" (`CMD_EVENT_ADD_TO_PLAYLIST`) never frees the playlist it loadsAberta
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 76/100
Mantenedores costumam responder em até 1 dia
-
Signed/unsigned mix in the EOCD backward scan rejects valid archives with a 65535-byte commentAberta
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 78/100
-
Dificuldade 2/5 1-3 horas Facilidade para iniciantes 78/100
OpenXiangShan/NEMU#1287 ·
Mantenedores costumam responder em até 1 dia