Deprecate Diagram.restrict() in 2.4 in favor of Diagram.expand(); remove it in 3.0

Offen
#1,693 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 6 Tagen

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
48/100
Issue-Typ
Refactoring
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
python
Bereich
databases

Rechercherichtung

Start in the diagram module where _propagate_restrictions, _apply_propagation_rule, _restrict_conditions, _restricted_table and the _cascade_restrictions flag live; the warning goes at the top of Diagram.restrict(). Check out the #1637 branch first, since expand() must be merged before this lands. Freeze current behavior by wrapping test_prune_after_restrict, test_prune_idempotent, test_prune_then_restrict and test_diagram_restrict_by_limited_expression in pytest.warns, then add the warning-source test in tests/integration/test_diagram.py. Done means the acceptance checklist passes, including the MySQL/PostgreSQL equivalence tests.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

enhancement

Summary

Diagram.expand(seed, direction) (#1637) replaces Diagram.restrict() for traversal. In 2.4, restrict() emits a DeprecationWarning that names the expand call to use instead, and its behavior stays as it is. It is removed in 3.0.

This issue tracks that work. #1637 lists "the deprecation of restrict()" under "Next". #1501, which proposed changing what restrict() does with part tables, was closed as not planned: expand keeps master-part instances whole, and restrict() will not change. The specification text is the "Retiring restrict" section of diagram.md in datajoint/datajoint-docs#249.

What restrict() does today, and what expand does instead

diag.restrict(T & cond) (2.2 and later) dj.Diagram.expand(T & cond, direction) (2.4)
Called on An existing diagram (instance method). The seed's table must be a node of the diagram. The class. The seed is one table with any restriction.
Direction Downstream only. "down": D(S). "up": U(S). "both": U(D(S)).
Where paths meet AND: a row is kept only if it matches every restricted parent, including two paths from the same seed. OR: a row is kept if any path reaches it.
Master-part groups Ignored. A condition on a part never reaches its master, and a master can be kept with only some of its parts. A part row reached brings in its master and all of the master's parts. Instances are never split.
Chaining Can be called again from another seed; conditions are AND-ed per table. One seed per call. Diagrams of the same direction will combine by union (d1 + d2) once the expanded-diagram contract lands.
Nodes The whole diagram stays. Tables the restriction never reached are iterated unrestricted. counts() counts only the reached tables. Only the tables the seed reaches. Every node is restricted, and counts() counts all of them.
Schemas Only those already loaded. Loads the schemas reachable in the direction of travel.
prune() Drops reached tables with no rows. Allowed on "up" and "both"; raises on "down", where an empty table can gain rows before a delete runs.
On a cascade() or trace() diagram Raises DataJointError. Not applicable.

For one seed, the two return the same rows when every reached table has a single path from the seed and none of them is a part table. With several paths or part tables, expand can return more rows, depending on the data.

Replacements

Delete previews. restrict() was never an exact preview of a delete. Where two paths from the seed meet, it keeps a row only if both paths reach it, while the delete removes the row if either one does. It also ignores part_integrity. Use the preview that Table.delete() itself uses:

# before
dj.Diagram(schema).restrict(Session & cond).counts()
# after: the rows (Session & cond).delete(part_integrity=...) removes
dj.Diagram.cascade(Session & cond, part_integrity="enforce").counts()
# after: everything that depends on the seed, with whole master-part instances
dj.Diagram.expand(Session & cond, "down").counts()

Traces. restrict() only goes downstream, so a trace built with it answers "which rows depend on these?":

# before: d[T] on a restricted diagram is a graph lookup, so the table is found by iterating
trials = next(t for t in dj.Diagram(schema).restrict(Session & cond)
              if t.full_table_name == Trial.full_table_name)
# after
trials = dj.Diagram.expand(Session & cond, "down")[Trial]

To find the rows a selection depends on (its lineage), use dj.Diagram.expand(Trial & key, "up")[Session]. In 2.4, dj.Diagram.trace(Trial & key) is the same call.

Export. Use dj.Diagram.expand(Session & cond, "both"), the smallest set of rows that includes the seed and is consistent on its own.

Several conditions. When the conditions share a descendant, seed that descendant with all of them: dj.Diagram.expand(Session & (Subject & cond_a) & cond_b, "both"). To intersect several seeds table by table, use the recipe in "Retiring restrict". Its result can split an instance, so it is for inspection, not for export.

The change in 2.4

On next, once #1637 is merged:

  • Diagram.restrict() calls warnings.warn(..., DeprecationWarning, stacklevel=2) once per call, before any other work. The message:

    Diagram.restrict() is deprecated and will be removed in DataJoint 3.0. Use dj.Diagram.expand(seed, direction): "down" for the rows that depend on the seed, "both" for an export. For a delete preview, use dj.Diagram.cascade(seed). expand combines paths by union and keeps master-part instances whole, so it can return more rows than restrict().

  • restrict() behaves as it does today: AND at convergence, chaining, the whole node set, prune(), and raising on a cascade() diagram.

  • restrict() on a diagram returned by expand(), cascade() or trace() raises DataJointError. It already does for cascade() and trace(), which set _cascade_restrictions; the check should rest on how the diagram was made, not on that attribute. On the #1637 branch, such a call silently drops the expansion: the copy constructor does not carry it over, and the result holds only the new seed's restriction.

  • The counts() error for a diagram with no restriction names expand() and cascade(), not restrict(). The docstrings of restrict() and prune() carry a deprecation note and point to expand.

  • No library code calls restrict(). Today nothing does, and expand, cascade, trace, Table.delete() and Table.drop() must not start.

In 3.0, restrict() is removed, along with the code only it uses: the mode="restrict" branches of _propagate_restrictions and _apply_propagation_rule, _restrict_conditions, and the AND branch of _restricted_table.

Acceptance criteria

  • Diagram.restrict() emits one DeprecationWarning per call, whose message names Diagram.expand and 3.0 and is attributed to the caller's line.
  • Apart from the warning, restrict() returns the same rows and node sets as on master at the time of the merge (including the dj.Top and subquery seeds handled there).
  • restrict() on a diagram returned by expand(), cascade() or trace() raises DataJointError.
  • expand, cascade, trace, counts, prune, Table.delete() and Table.drop() emit no DeprecationWarning from restrict().
  • The counts() error message and the restrict() and prune() docstrings point to expand.
  • The docs agree with the code: the deprecation admonition and "Retiring restrict" in diagram.md (datajoint/datajoint-docs#249, which also drops restrict from the comparison table in trace.md), and the 2.4 release notes.
  • A 3.0 issue for the removal is opened and linked here.

Tests

  • tests/integration/test_diagram.py: a new test asserts the warning with pytest.warns(DeprecationWarning, match="Diagram.expand") and checks that the warning's filename is the test file, so stacklevel points at the caller.
  • The existing restrict() tests (test_prune_after_restrict, test_prune_idempotent, test_prune_then_restrict, and test_diagram_restrict_by_limited_expression in test_top_in_queries.py, which reaches next when master is next merged into it) are wrapped in pytest.warns(DeprecationWarning) and otherwise unchanged. This shows the behavior is frozen.
  • A test that runs expand, cascade, trace, counts() and Table.delete() under warnings.simplefilter("error", DeprecationWarning) and expects no error.
  • A test that dj.Diagram.expand(A & cond, "down").restrict(A & cond) raises DataJointError.
  • Two migration tests on MySQL and PostgreSQL:
    • On a chain with no part tables, restrict(A & cond).counts() equals expand(A & cond, "down").counts() for every table that restrict reaches.
    • On a diamond (A -> B, A -> C, B -> D, C -> D) and on a master with a part, the rows of restrict() are a subset of the rows of expand(..., "down"), and the test checks the rows where they differ. This documents why the warning says expand can return more rows.
Vorherrschende Sprache
Python
Sterne
198
Forks
103
Ø Merge
6 T. 1 Std.
Gemergte PRs (30 T.)
10

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus datajoint/datajoint-python

Alle Issues in datajoint/datajoint-python

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.