keys() and to_arrays(attr) fail on an aggregation with dj.Top ordered by an aggregate

Aperta
#1,670 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 6 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
65/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
mysql, postgresql, python, sql
Ambito
databases

Direzione di ricerca

Start by running the reproduction against a local backend to see keys() and to_arrays("t_id") fail while to_dicts() succeeds. The cause points at Aggregation.make_sql(fields) around expression.py:1355: read how the select list is built versus how the ORDER BY and HAVING clauses are carried through, including the PostgreSQL HAVING wrapping path. Done means the ORDER BY attributes and HAVING aliases are always selected inside the aggregation and the requested fields are projected by an outer SELECT.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

An aggregation restricted by dj.Top and ordered by a computed attribute works with to_dicts(), to_arrays() and len(). It fails with keys() and to_arrays("attr") whenever the ordering attribute is not among the selected fields.

Reproduced on master @ 3c912c46 against MySQL 8.0 and PostgreSQL 16. Connection via DJ_HOST/DJ_USER/DJ_PASS/DJ_BACKEND. Found while benchmarking subquery pruning (#1665).

Reproducible example
import datajoint as dj

schema = dj.Schema("repro_top")


@schema
class T(dj.Manual):
    definition = """
    t_id : int32
    ---
    x : int32
    y : int32
    """


@schema
class S(dj.Manual):
    definition = """
    -> T
    s_id : int32
    """


T.insert([(i, i, 100 - i) for i in range(10)], skip_duplicates=True)
S.insert([(i, j) for i in range(10) for j in range(2)], skip_duplicates=True)

q = T.aggr(S, n="count(*)") & dj.Top(3, "n DESC, t_id")
print(q.to_dicts())          # works
print(len(q))                # works
print(q.keys())              # fails
print(q.to_arrays("t_id"))   # fails
Actual

MySQL:

[{'t_id': 0, 'n': 2}, {'t_id': 1, 'n': 2}, {'t_id': 2, 'n': 2}]
3
datajoint.errors.UnknownAttributeError: Unknown column 'n' in 'order clause'

PostgreSQL:

datajoint.errors.UnknownAttributeError: column "n" does not exist
LINE 1: ..._top"."s" USING ("t_id") GROUP BY "t_id" ORDER BY "n" DESC, ...

SQL generated for q.proj():

SELECT DISTINCT `t_id` FROM `repro_top`.`t` LEFT JOIN `repro_top`.`s` USING (`t_id`) GROUP BY `t_id` ORDER BY `n` DESC, `t_id` LIMIT 3
Expected

keys() returns the 3 keys, and to_arrays("t_id") returns an array of 3 ids.

Cause

Fetching specific fields calls Aggregation.make_sql(fields) (expression.py:1355), which selects only those fields. Its ORDER BY, however, still names the aggregate alias n. Once n is not in the select list, the alias no longer exists. The PostgreSQL path wraps the aggregation for HAVING, but the same problem applies there.

Fix: always select the ORDER BY attributes (and HAVING aliases) inside the aggregation. When the requested fields differ, wrap the aggregation in an outer SELECT <fields>.

Lingua principale
Python
Stelle
198
Fork
103
Merge medio
6g 1h
PR unite (30g)
10

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di datajoint/datajoint-python

Tutte le issue di datajoint/datajoint-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.