polars 1.35.2 (RAPIDS 26.02 image) rust-panics on grouped min/max over Categorical — a pyo3 PanicException no GFQL wrapper can catch

Aperta
  • Qualcosa non va?
#1,822 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
backend, testing

Direzione di ricerca

Start with the categorical aggregation workaround and its test at graphistry/tests/compute/gfql/test_aggregate_type_contract.py::test_categorical_is_decategorized_before_aggregating_on_polars; then survey other Polars expressions emitted by the repository that could send categorical values into kernels. Also compare the Polars versions in the RAPIDS image and local development. Done means the survey and version-drift decision are documented, with any identified risk covered by tests.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Surfaced by the GPU lane of #1819 on the dgx GB10
(graphistry/test-rapids-official:26.02-gfql-polars, polars 1.35.2). Worked around there, but
the underlying hazard is general and worth its own record.

What happens

A grouped min()/max() over a Categorical column makes polars 1.35.2 panic in its Rust core:

thread 'polars-2' panicked at crates/polars-core/src/series/implementations/categorical.rs:350:1:
not implemented
...
E   pyo3_runtime.PanicException: not implemented
    polars/lazyframe/frame.py:2422: pyo3_runtime.PanicException

Local polars 1.42 handles the same expression fine, so this is invisible outside the RAPIDS
image — 10 test failures on the GPU box, 0 locally, on identical code.

Why it is a hazard beyond one polars version

pyo3_runtime.PanicException is not a polars.exceptions.PolarsError. #1819 added a choke point
in the native polars chain that rewraps polars errors as GFQLTypeError(E303) so a third-party class
never reaches the caller — and that wrapper cannot catch this. A Rust panic escaping through the
GFQL surface is uncatchable by construction on the python side, so the only defence is not to emit
the expression that panics.

Workaround already in place (#1819)

Categorical/Enum columns are cast to String before any aggregate on both engines — which is
independently correct (Cypher treats a categorical as a string column, and it also fixed pandas
raising and cuDF answering count(DISTINCT <categorical>) with a category label). Pinned by a test
that asserts the result dtype is String, because the failure it prevents is version-dependent
and a value assertion on polars 1.42 cannot see it:

graphistry/tests/compute/gfql/test_aggregate_type_contract.py::test_categorical_is_decategorized_before_aggregating_on_polars

What is left to decide

  1. Whether other polars expressions this repo emits can panic the same way on the pinned image
    version — a survey, not a guess. Anything Categorical-typed reaching a polars kernel is suspect.
  2. Whether the polars version drift between the RAPIDS image (1.35.2) and local dev (1.42) should be
    closed or at least asserted, since it silently changes which defects are observable. Related
    standing note: validate in-container before measuring anything polars-version-sensitive.
Lingua principale
Python
Stelle
2.6k
Fork
229
Merge medio
1g 5h
PR unite (30g)
99

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di graphistry/pygraphistry

Tutte le issue di graphistry/pygraphistry

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.