Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Mapping restriction naming a hidden attribute silently drops the predicate

Abierto
  • ¿Algo no está bien?
#1,561 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
75/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
mysql, postgresql, python
Área
backend, databases

Línea de trabajo

Read condition.py around lines 347 and 409, then compare the hidden-attribute lookup with the fallback in heading.py:398-404. Verify restriction tests for hidden and absent attributes on MySQL and PostgreSQL, while confirming visible JSON-path restrictions remain unchanged; done means a mapping restriction on an existing hidden column produces its predicate and an absent column is still ignored.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug

A mapping restriction naming an attribute that exists on the table but is hidden has its predicate dropped, and every row is returned.

Nothing about JSON is involved. Earlier revisions of this issue demonstrated it with _prov, whose JSON type made it look like a JSON-path problem; it is not.

Repro — no JSON

@schema
class Comp(dj.Computed):          # with config.jobs.add_job_metadata on
    definition = """
    -> Src
    ---
    val : int32
    """

Two populated rows. _job_version is a varchar(64):

restriction MySQL 8.0 postgres:15
Comp & {"_job_version": "nonexistent"} 2 2
Comp & "_job_version IS NOT NULL" 2 2

The first should match nothing. The emitted SQL carries no predicate at all. Same on both backends, so the defect is backend-agnostic, and it applies to _job_start_time, _job_duration and _prov equally.

What is not broken

Worth stating, because the earlier framing of this issue implied otherwise: JSON path restriction works correctly and portably on a visible attribute.

Doc & {"data.system": "PyRat"}     # `data : json`

returns 1 of 2 rows on MySQL and on PostgreSQL alike. translate_attribute (condition.py:59-61) hands the path to adapter.json_path_expr, which emits json_value() on one and jsonb_extract_path_text() on the other. That machinery is fine and is not what this issue is about.

The reason a JSON-typed hidden attribute looks worse is only that the portable spelling runs through the mapping form, which is the form being dropped — so for _prov the workaround has to be backend-specific SQL. That is a consequence of this bug, not a separate JSON defect.

Cause

condition.py:409:

common_attributes = set(c.split(".", 1)[0] for c in condition).intersection(query_expression.heading.names)
if not common_attributes:
    return not negate  # no matching attributes -> evaluates to True

heading.names is visible-only, so a hidden name lands in the same bucket as an attribute the table does not have.

Suggested fix

Match against the names present in _attributes rather than names, so a hidden attribute resolves instead of being discarded.

The two cases are different facts and are distinguishable:

  • scan_id on Session — absent from _attributes entirely. Ignoring it is deliberate and must stay: Session & key has to work when key carries attributes from a more detailed downstream table, and make() depends on that.
  • _job_version on a Computed table — present in _attributes, filtered only out of names. The column exists; the caller named a real thing.

So the key-passing idiom is untouched. A dict reaching & can only acquire a hidden name by hand: every key DataJoint produces — keys(), to_dicts(), fetch1(), key_source — is built from the visible heading, so no generated query changes.

Nothing hidden becomes readable by this. Restriction on these columns already works through the string form; #1562 is the separate question of reading values back.

One obstacle in the same change: condition.py:347 does query_expression.heading[key_match["attr"]].uuid, and Heading.__getitem__ resolves through attributes, so it raises KeyError on a hidden name. It needs the _attributes fallback Heading.as_sql already uses (heading.py:398-404).

Noticed in passing, not filed

Doc & {"data": {"system": "PyRat", "n": 7}} — restricting by a whole JSON object rather than a path — raises QuerySyntaxError on both backends. prep_value only special-cases a dict value when the key is a JSON path (condition.py:343-344), so a whole-object comparison falls through and emits invalid SQL. It errors rather than answering wrongly, so it is a poor message rather than a correctness problem. Mentioned only so the next person who hits it knows it is understood.

Documentation

reference/specs/boundary-provenance.md, how-to/record-data-origin.md, the data-entry tutorial and reference/specs/job-metadata.md advertised the mapping form on _prov. Corrected in datajoint/datajoint-docs#288.

Lenguaje dominante
Python
Estrellas
197
Forks
98
Merge medio
1 d 23 h
PR fusionados (30 d)
6

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de datajoint/datajoint-python

Todos los issues de datajoint/datajoint-python

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.