Unconsumed lookahead token used for ErrorRecoveryNode causes unsigned location length underflow
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 52/100
調査の方向性
Start in src/prism.c and inspect the eight listed pm_error_recovery_node_create call sites, especially the parse_not recovery path described in the reproduction. Trace how each call handles parser->current and parser->previous, then check the parser tests or fuzzing coverage for {not %. Done means the reproducer no longer creates a node with a wrapped location length and the affected recovery paths are covered.
索引モデルが issue の本文から書いたものです。
説明
Fuzzing find on bdbd482dc.
When whitespace or skipped invalid bytes separate parser->previous.end from parser->current.start, the resulting ErrorRecoveryNode is given a start offset at parser->current.start while parser->previous.end remains at an earlier offset. When an enclosing construct then synthesizes a missing token via expect1 (which places the missing token at parser->previous.end), the parent node's end (parser->previous.end) is smaller than its child's start (parser->current.start), underflowing uint32_t location.length to ~4,294,967,295.
Reproduction
Simulation here: https://stevenjohnstone.github.io/prism-loupe/?version=v2.1.0&code=e25vdCAl.
✗ ./bin/parse -e "{not %"
Warnings:
[#<Prism::ParseWarning @type=:void_statement @message="possibly useless use of % in void context" @location=#<Prism::Location @start_offset=0 @length=6 start_line=1> @level=:verbose>]
Errors:
[#<Prism::ParseError @type=:invalid_percent_eof @message="unterminated quoted string meets end of file" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:expect_lparen_after_not_other @message="expected a `(` after `not`" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:hash_rocket @message="expected a `=>` between the hash key and value" @location=#<Prism::Location @start_offset=4 @length=0 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:hash_value @message="unexpected '%'; expected a value in the hash literal" @location=#<Prism::Location @start_offset=5 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:hash_term @message="expected a `}` to close the hash literal" @location=#<Prism::Location @start_offset=0 @length=1 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:expect_expression_after_operator @message="unexpected end-of-input; expected an expression after the operator" @location=#<Prism::Location @start_offset=6 @length=0 start_line=1> @level=:syntax>,
#<Prism::ParseError @type=:unexpected_token_close_context @message="unexpected end-of-input, assuming it is closing the parent top level context" @location=#<Prism::Location @start_offset=6 @length=0 start_line=1> @level=:syntax>]
AST:
@ ProgramNode (location: (1,0)-(1,6))
├── flags: ∅
├── locals: []
└── statements:
@ StatementsNode (location: (1,0)-(1,6))
├── flags: ∅
└── body: (length: 1)
└── @ CallNode (location: (1,0)-(1,6))
├── flags: newline
├── receiver:
│ @ HashNode (location: (1,0)-(1,4))
│ ├── flags: ∅
│ ├── opening_loc: (1,0)-(1,1) = "{"
│ ├── elements: (length: 1)
│ │ └── @ AssocNode (location: (1,5)-(1,4294967300))
│ │ ├── flags: ∅
│ │ ├── key:
│ │ │ @ ErrorRecoveryNode (location: (1,5)-(1,6))
│ │ │ ├── flags: ∅
│ │ │ └── unexpected: ∅
│ │ ├── value:
│ │ │ @ ErrorRecoveryNode (location: (1,4)-(1,4))
│ │ │ ├── flags: ∅
│ │ │ └── unexpected: ∅
│ │ └── operator_loc: ∅
│ └── closing_loc: (1,4)-(1,4) = ""
├── call_operator_loc: ∅
├── name: :%
├── message_loc: (1,5)-(1,6) = "%"
├── opening_loc: ∅
├── arguments:
│ @ ArgumentsNode (location: (1,5)-(1,6))
│ ├── flags: ∅
│ └── arguments: (length: 1)
│ └── @ ErrorRecoveryNode (location: (1,5)-(1,6))
│ ├── flags: ∅
│ └── unexpected: ∅
├── closing_loc: ∅
├── equal_loc: ∅
└── block: ∅
Byte offsets in "{not %" (6 bytes):
{:0..1not:1..4(parser->previousafter consumingnot, soparser->previous.end == 4)' ':4..5%:5..6(invalid string delimiter at EOF; skipped to offset6, soparser->currentisPM_TOKEN_EOFat6..6)
In Prism.parse("{not %"), the resulting AssocNode has location.start = 6 and end = 4, so location.length (U32(end - PM_NODE_START(key))) wraps modulo $2^{32}$ to 4294967294 (0xFFFFFFFE).
Root Cause
- In
"{not %",{opens a hash literal (parse_hash). notis parsed as the start of a hash key expression (parse_not), settingparser->previoustonot(offsets1..4).- The lookahead token after the space (
offset 4..5) and invalid%(offset 5..6) isPM_TOKEN_EOFat6..6. parse_notcallsparse_expressionfor its operand, which hitscontext_recoverable(parser, context):
Becauseif (context_recoverable(parser, context) || depth == PRISM_DEPTH_MAXIMUM) { ... return UP(pm_error_recovery_node_create(parser, PM_TOKEN_START(parser, &parser->current), PM_TOKEN_LENGTH(&parser->current))); }&parser->current(6..6) is passed without consumingparser->current, the returned node hasstart = 6whileparser->previousis stillnot(1..4,end = 4).- When
parse_hashthen callsexpect1for=>(which synthesises the missing operator atparser->previous.end == 4) and constructs anAssocNodewhosestartis6and whoseendis4,U32(end - PM_NODE_START(key))underflowsuint32_t.
The same pattern occurs in 8 places in src/prism.c where pm_error_recovery_node_create is called without consuming parser->current:
parse_string_part(default:branch ofPM_TOKEN_EMBVARafterexpect1(parser, PM_TOKEN_IDENTIFIER, PM_ERR_EMBVAR_INVALID))parse_undef_argument(default:branch)parse_alias_argument(default:branch)parse_pattern_primitive(default:branch)parse_pattern_primitives(default:branch)parse_pattern(context_recoverable/depth == PRISM_DEPTH_MAXIMUMbranch)parse_not(context_recoverable/depth == PRISM_DEPTH_MAXIMUMbranch)parse_expression(context_recoverable/depth == PRISM_DEPTH_MAXIMUMbranch)
- 主要言語
- C
- スター
- 1k
- フォーク
- 202
- 平均マージ
- 5時間 55分
- マージ済み PR(30日)
- 85
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
ruby/prism のほかの issue
-
Prism::Translation::Parser raises NoMethodError for a method with a &nil parameter and a body対応中かも @bbatsov が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
メンテナーはふだん 1 日以内に返信
-
Heredoc and outer-context recovery at recursion depth / stack budget limit causes location underflowオープン
難易度 4/5 3〜5日 初心者へのやさしさ 43/100
メンテナーはふだん 1 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 1/100
メンテナーはふだん 1 日以内に返信
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
OpenXiangShan/NEMU#1287 ·
メンテナーはふだん 1 日以内に返信
-
area: recompiler area: runtime kind: bug preview-effort: small priority: low
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
メンテナーはふだん 1 日以内に返信