Diagram.restrict: apply Part conditions through the Master to preserve master-part compositional integrity
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- python
- Domain
- data-engineering, databases
Research direction
Start in src/datajoint/diagram.py with restrict and _propagate_restrictions(mode="restrict"), then compare _propagate_part_to_master for cascade’s upward walk. Verify the existing reproduction and trace how Part conditions are propagated. Done means Part restrictions select qualifying Masters while every selected Master retains all of its Parts, without changing cascade deletion behavior.
Written by the indexing model from the issue text.
Description
Summary
dj.Diagram.restrict is logically conjunctive (AND — it removes rows that fail the criteria).
Unlike the disjunctive trace/cascade (which include everything touched and keep/pull whole
compositional units), a conjunctive filter can cut into a master-part group, leaving partial
groups — a Master exported with only a subset of its Parts (or none). This preserves referential
integrity but breaks compositional integrity: the exported entity misrepresents the original.
Reproduction (structural)
dj.Diagram(schema).restrict(Master & 'm=1').restrict(Master.Part & 'p>3')
Master→ all ofm=1.Master.Part→ (Parts ofm=1, from the master-keyed downstream restriction) ANDp>3→ only
thep>3Parts.- Result:
Master m=1whole, but only a subset of its Parts. A Master can even survive with zero
Parts, sincerestrictdoes not walk up from Part to Master.
Why it matters
- Referential integrity — fine. A kept Part's FK to its Master is satisfied; a Master needs no Parts
to be valid. The export re-imports without dangling foreign keys. - Compositional integrity — broken. DataJoint's master-part contract treats a Master's Parts as one
atomic entity. A recording exported with only some of its channels loads cleanly but is a
corrupted/partial entity.
Root cause / model
In DataJoint's model the Master is the entity; Parts have no independent existence. The unit of
export is therefore the Master-with-all-its-Parts. A Part-level filter is either meaningless for export,
or should mean "Masters that have a matching Part."
Decision (v2.4)
In Diagram restriction, conditions on Part tables are applied through their Master:
- A restriction that selects Part rows lifts to an existential restriction on the Master — Masters
that have a matching Part are selected. - A selected Master brings in all of its Parts.
- Net: compositional units are never split; Parts are never filtered within a unit.
This is the compositional-integrity analogue, for read-only restriction/export, of what cascade's
part_integrity="cascade" does for delete.
Scope / relationships
- Not for 2.3.1 — behavior change deferred to v2.4.
- Distinct from #1500 (which established that
restrictdoes not performcascade's deletion
part→master walk-up). This issue is about filtering granularity — keeping units whole under a
conjunctive filter — a different mechanism; the two are compatible. - Optional 2.3.x stopgap: document the current behavior (restrict can yield partial Part groups; lift
Part criteria to the Master viaMaster & (Part & cond)).
References
- Surfaced in the 2.3.1
dj.Diagramreview (finding F12). - Code:
src/datajoint/diagram.py—restrict,_propagate_restrictions(mode="restrict"); contrast
_propagate_part_to_master(cascade's upward walk).
- Dominant language
- Python
- Stars
- 197
- Forks
- 98
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 1
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from datajoint/datajoint-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
datajoint/datajoint-python#1539 · 3 comments ·
-
dj.Diagram SVG output is not byte-reproducible: set iteration order leaks into node emission order Openbug
Difficulty 3/5 1-2 days Newbie friendliness 78/100
datajoint/datajoint-python#1551 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
datajoint/datajoint-python#1550 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
datajoint/datajoint-python#1547 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 52/100
datajoint/datajoint-python#1546 · 1 comment ·
All issues in datajoint/datajoint-python
Similar issues
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
canonical/paas-charm#368 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
tech debt
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
StevenBlack/hosts#3256 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
qualcomm/qai-appbuilder#275 ·