Hacktoberfest 2026 : les issues que les mainteneurs ont marquées pour octobre, ouvertes et accessibles aux débutants. Parcourir les issues Hacktoberfest

Mapping restriction naming a hidden attribute silently drops the predicate

Ouverte
  • Quelque chose ne va pas ?
#1,561 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
3/5
Temps estimé
1-2 jours
Accessibilité débutants
75/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
mysql, postgresql, python
Domaine
backend, databases

Piste de recherche

Read condition.py around lines 347 and 409, then compare the hidden-attribute lookup with the fallback in heading.py:398-404. Verify restriction tests for hidden and absent attributes on MySQL and PostgreSQL, while confirming visible JSON-path restrictions remain unchanged; done means a mapping restriction on an existing hidden column produces its predicate and an absent column is still ignored.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

bug

A mapping restriction naming an attribute that exists on the table but is hidden has its predicate dropped, and every row is returned.

Nothing about JSON is involved. Earlier revisions of this issue demonstrated it with _prov, whose JSON type made it look like a JSON-path problem; it is not.

Repro — no JSON

@schema
class Comp(dj.Computed):          # with config.jobs.add_job_metadata on
    definition = """
    -> Src
    ---
    val : int32
    """

Two populated rows. _job_version is a varchar(64):

restriction MySQL 8.0 postgres:15
Comp & {"_job_version": "nonexistent"} 2 2
Comp & "_job_version IS NOT NULL" 2 2

The first should match nothing. The emitted SQL carries no predicate at all. Same on both backends, so the defect is backend-agnostic, and it applies to _job_start_time, _job_duration and _prov equally.

What is not broken

Worth stating, because the earlier framing of this issue implied otherwise: JSON path restriction works correctly and portably on a visible attribute.

Doc & {"data.system": "PyRat"}     # `data : json`

returns 1 of 2 rows on MySQL and on PostgreSQL alike. translate_attribute (condition.py:59-61) hands the path to adapter.json_path_expr, which emits json_value() on one and jsonb_extract_path_text() on the other. That machinery is fine and is not what this issue is about.

The reason a JSON-typed hidden attribute looks worse is only that the portable spelling runs through the mapping form, which is the form being dropped — so for _prov the workaround has to be backend-specific SQL. That is a consequence of this bug, not a separate JSON defect.

Cause

condition.py:409:

common_attributes = set(c.split(".", 1)[0] for c in condition).intersection(query_expression.heading.names)
if not common_attributes:
    return not negate  # no matching attributes -> evaluates to True

heading.names is visible-only, so a hidden name lands in the same bucket as an attribute the table does not have.

Suggested fix

Match against the names present in _attributes rather than names, so a hidden attribute resolves instead of being discarded.

The two cases are different facts and are distinguishable:

  • scan_id on Session — absent from _attributes entirely. Ignoring it is deliberate and must stay: Session & key has to work when key carries attributes from a more detailed downstream table, and make() depends on that.
  • _job_version on a Computed table — present in _attributes, filtered only out of names. The column exists; the caller named a real thing.

So the key-passing idiom is untouched. A dict reaching & can only acquire a hidden name by hand: every key DataJoint produces — keys(), to_dicts(), fetch1(), key_source — is built from the visible heading, so no generated query changes.

Nothing hidden becomes readable by this. Restriction on these columns already works through the string form; #1562 is the separate question of reading values back.

One obstacle in the same change: condition.py:347 does query_expression.heading[key_match["attr"]].uuid, and Heading.__getitem__ resolves through attributes, so it raises KeyError on a hidden name. It needs the _attributes fallback Heading.as_sql already uses (heading.py:398-404).

Noticed in passing, not filed

Doc & {"data": {"system": "PyRat", "n": 7}} — restricting by a whole JSON object rather than a path — raises QuerySyntaxError on both backends. prep_value only special-cases a dict value when the key is a JSON path (condition.py:343-344), so a whole-object comparison falls through and emits invalid SQL. It errors rather than answering wrongly, so it is a poor message rather than a correctness problem. Mentioned only so the next person who hits it knows it is understood.

Documentation

reference/specs/boundary-provenance.md, how-to/record-data-origin.md, the data-entry tutorial and reference/specs/job-metadata.md advertised the mapping form on _prov. Corrected in datajoint/datajoint-docs#288.

Langage dominant
Python
Étoiles
197
Forks
98
Merge moyen
1 j 23 h
PR mergées (30 j)
6

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de datajoint/datajoint-python

Toutes les issues de datajoint/datajoint-python

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.