[Bug]: Python SDKs lowercase output expressions, silently corrupting string literals

Open Beginner friendly
#3,479 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
Half a day
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
api

Research direction

Inspect output() in python/infinity_sdk/infinity/remote_thrift/query_builder.py and python/infinity_embedded/local_infinity/query_builder.py, then read test_output_preserves_string_literal_case in python/test_pysdk/test_condition.py. Verify that special tokens remain case-insensitive while string literals retain their original case, identifiers still normalize, and the existing regression test passes in both builders.

Written by the indexing model from the issue text.

Description

Problem

Both Python SDKs (thrift and embedded) lowercase every output() column string before parsing it:

table.output(["'ACTIVE'"])   # silently selects 'active'

The lowering is meant to make the special output tokens (*, _row_id, _score, _distance, ...) case-insensitive, but it rewrites the whole expression string, including string literals inside it. Any literal with uppercase characters is corrupted before the server ever sees it - and once json_extract(...) parses through the SDKs (currently blocked by #3463), a call like json_extract(data, '$.UserName') would silently query the key $.username and return NULL.

The HTTP API is not affected: it forwards output strings to the server, which parses them case-correctly. So the same query returns different data depending on which client you use.

Root cause

python/infinity_sdk/infinity/remote_thrift/query_builder.py and python/infinity_embedded/local_infinity/query_builder.py, output():

for column in columns:
    if isinstance(column, str):
        column = column.lower()   # rewrites the expression itself
    match column:
        ...
        case _:
            ... maybe_parse(column) ...   # parses the lowered string

Fix

Match the special tokens against a lowercased copy and parse the original string:

key = column.lower() if isinstance(column, str) else column
match key:
    ...
    case _:
        ... maybe_parse(column) ...   # original case preserved

Plain identifiers are still lowercased later by the exp.Column arm (alias_or_name.lower()), so column-name handling is unchanged; only string literals keep their case.

Testing

  • Reproduced on current main (eca7266), sqlglot 30.18.0: before the fix output(["'ACTIVE'"]) built a constant expression holding 'active' in both SDKs; after the fix it holds 'ACTIVE'. _SCORE still maps to the score function, MyCol still normalizes to mycol, and * still selects all.
  • Added regression test test_output_preserves_string_literal_case in python/test_pysdk/test_condition.py covering the thrift builder directly and the embedded builder in embedded mode.
  • Full pysdk suite not run locally; it needs a running server / built embedded engine.
Dominant language
C++
Stars
4.7k
Forks
445
Avg merge
2d 9h
Merged PRs (30d)
8

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from infiniflow/infinity

All issues in infiniflow/infinity

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.