bug: SQL array literals take the type of their first element, so [0, 0.5, 1], [NULL, 1] and [] fail

Aperta
#11,114 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

@aimanmalib ci sta già lavorando.

Dal 15/11/2026.

  • #11115 di @aimanmalib — aperta

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python, rust, sql
Ambito
databases

Direzione di ricerca

The fix lives in the SQLExpr::Array branch of Planner::parse_sql_expr in rust/lance-datafusion/src/planner.rs, where the first element currently fixes the element type and the rest are concatenated. Read that branch first, then compare with DataFusion's make_array behaviour for numeric widening, null elements and the empty-list case. Reproduce with the Python ds.update snippet; done means [0, 0.5, 1], [NULL, 1], [1, NULL] and [] all plan to a sensible list type while [1, 'a'] still errors. Note a pull request (#11115) is already open against this, so coordinate before starting.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Description

An SQL array literal takes the type of its first element and coerces every other element to that type. So [0, 0.5, 1] fails with Array expressions must have a consistent datatype. Expected: Int64, got: Float64, while [0.5, 0, 1] works. A NULL element fails in any position: [NULL, 1] gives Expected: Null, got: Int64, and [1, NULL] gives It is not possible to concatenate arrays of different data types. An empty literal [] fails with concat requires input of at least one array.

This shows up in update on list and vector columns, for example from the LanceDB JS client, which writes the whole number 0 as 0. The code is the SQLExpr::Array branch of Planner::parse_sql_expr in rust/lance-datafusion/src/planner.rs.

Steps to reproduce
import tempfile
import lance
import pyarrow as pa

ds = lance.write_dataset(
    pa.table({"id": [1, 2], "f": pa.array([[1.0], [3.5]], pa.list_(pa.float64()))}),
    tempfile.mkdtemp(),
)
ds.update({"f": "[0.5, 0, 1]"}, where="id = 1")  # works
ds.update({"f": "[0, 0.5, 1]"}, where="id = 1")
# ValueError: Invalid user input: Array expressions must have a consistent datatype. Expected: Int64, got: Float64
ds.update({"f": "[1, NULL]"}, where="id = 1")
# ValueError: ... It is not possible to concatenate arrays of different data types
ds.update({"f": "[]"}, where="id = 1")
# ValueError: ... concat requires input of at least one array

Vector (FixedSizeList<float32>) columns fail the same way with [0, 0.5, 1] and [NULL, 1].

Expected behavior

Element order should not matter. The literal should use the type all elements share, as DataFusion's make_array does: [0, 0.5, 1] becomes [0.0, 0.5, 1.0], NULL becomes a null element, and [] becomes an empty list. Elements with no shared type, such as [1, 'a'], should still be rejected.

Lance version

pylance 15.0.0-beta.22 (49e29d7be). The planner code is unchanged on main 8888a674d.

Lingua principale
Rust
Stelle
7.3k
Fork
929
Merge medio
2g 22h
PR unite (30g)
572

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di lance-format/lance

Tutte le issue di lance-format/lance

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.