JSONPath parser raises unhandled ValueError for exponent-form integer indexes

Open Beginner friendly
  • Is something wrong?
#8,867 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, sql
Domain
compilers

Research direction

The failure is in _parse_literal() in sqlglot/jsonpath.py, where int(number) runs on a NUMBER token that can contain exponent notation such as 2E or 1e5. Reproduce with jsonpath.parse("[2E]", dialect="snowflake") and check that Dialect.to_json_path() also lets the ValueError escape. Done means the invalid index raises sqlglot.errors.ParseError with a message naming the bad index, and existing integer-index tests still pass.

Written by the indexing model from the issue text.

Description

  • Read dialect: snowflake (using the direct sqlglot.jsonpath.parse() API with dialect="snowflake").
  • Write dialect: N/A (this is a parsing issue; no SQL generation is involved).
  • Checked on main: Yes, reproduced on commit 3b1fc58.

Fully reproducible code snippet

from sqlglot import jsonpath

jsonpath.parse("[2E]", dialect="snowflake")

Actual output:

ValueError: invalid literal for int() with base 10: '2E'

The same failure can be reproduced with:

from sqlglot import jsonpath

jsonpath.parse("[1e5]", dialect="snowflake")
jsonpath.parse("[-2E]", dialect="snowflake")

Expected output:

A controlled sqlglot.errors.ParseError indicating that the JSONPath index is invalid, rather than an uncaught Python ValueError.

For example:

ParseError: Invalid integer index: 2E at index 2: [2E]

Notes

The issue originates in sqlglot/jsonpath.py, inside _parse_literal():

number = "-" if _match(TokenType.DASH) else ""

token = _match(TokenType.NUMBER)
if token:
    number += token.text

if number:
    return int(number)

The JSONPath tokenizer can produce a numeric token containing exponent notation. However, JSONPath array indexes must be decimal integers.

Consequently, int(number) raises ValueError when the token contains exponent notation, such as 2E or 1e5.

The exception is not converted into SQLGlot's expected ParseError.

This also affects the dialect-level JSONPath conversion path. Dialect.to_json_path() catches ParseError and TokenError, but not ValueError. Consequently, this exception bypasses the existing error-handling mechanism.

This is distinct from the previously addressed JSONPath TokenError handling in [PR #8260](https://github.com/tobymao/sqlglot/pull/8260), which does not address this integer-conversion failure.

Suggested fix

In sqlglot/jsonpath.py:

if number:
    try:
        return int(number)
    except ValueError as e:
        raise ParseError(
            _error(f"Invalid integer index: {number}")
        ) from e

This preserves the existing integer-index behavior while converting invalid numeric indexes into the parser's expected exception type.

Official Documentation

LLM assistance: An LLM was used to assist with drafting this report. The reproducer and relevant implementation path were independently examined.

Dominant language
Python
Stars
9.7k
Forks
1.4k
Avg merge
13h 20m
Merged PRs (30d)
217

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from tobymao/sqlglot

All issues in tobymao/sqlglot

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.