Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Diagram.trace: descend through master-part boundaries (Merge-table upstream tracing)

Open
#1,481 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python

Research direction

Start with reference/specs/trace.md §2, Diagram.trace(), and the existing test test_trace_stops_at_master_no_part_down_collection. Read _propagate_part_to_master and discussion #1232 to understand the proposed upward analog, then resolve the default-versus-opt-in behavior and add coverage showing stable OR-merged tracing through Master.Part, including strict_provenance effects.

Written by the indexing model from the issue text.

Description

enhancement

Summary

Diagram.trace() (and therefore self.upstream) walks ancestor FK edges only. It does not descend from an ancestor Master into that Master's Parts. In the merge-table shape used by Spyglass —

Parent → Master.Part → Master → Child

trace(Child & key) reaches Master, but trace[Master.Part] and trace[Parent] raise DataJointError. The true upstream source (Parent) is unreachable through the merge point.

This matches the shipped spec (the Upstream Trace spec, reference/specs/trace.md §2 "Allowed table set"; formerly provenance.md, renamed in 2.3: an ancestor's Part is included only when the Part itself lies on an FK path to the seed) and is now pinned by test_trace_stops_at_master_no_part_down_collection. However, the design comment on discussion #1232 described a symmetric down-collection ("when a Master is reached, add its Parts to the trace, and continue upward from the Parts' FK parents") that was never implemented — a correction has been posted there. This issue tracks actually building that capability.

Proposed behavior (opt-in or default — design question)

When the upward walk reaches a Master, additionally:

  1. Restrict the Master's Parts downward from the Master's restriction (the existing forward rules).
  2. Continue upward from the Parts' other FK parents (the upward rules), OR-merging into the trace.
  3. Multi-pass until stable (mirror of the downstream part_integrity="cascade" mechanics — this is its upward analog, cf. _propagate_part_to_master).

Design questions

  • Default-on (a Part is semantically an extension of its Master, so "what contributed" arguably includes contributions through Parts) vs. opt-in flag (trace(expr, through_parts=True)) to preserve current semantics.
  • OR-merge and termination through alias nodes; interaction with the (part, master)-pair dedup introduced for the downstream analog.
  • Effect on self.upstream's allowed-table set under strict_provenance — the merge pattern is exactly where downstream make()s need to read through the merge point.

Motivation

  • Spyglass Merge tables (discussion #1232, @CBroz1): versioned pipelines route provenance through Master/Part merge points; upstream tracing that stops at the merge master cannot answer "which Parent produced this result."
  • Without it, strict-provenance mode makes such reads impossible rather than merely unergonomic (the true parent is not in the allowed set).
Dominant language
Python
Stars
197
Forks
98
Avg merge
6d 7h
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from datajoint/datajoint-python

All issues in datajoint/datajoint-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.