JuliaPluto/Pluto.jl

Preserve cell order in Safe preview

Open

#2691 opened on Oct 30, 2023

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Julia (5,295 stars) (329 forks)batch import
backendenhancementgood first issuereactivitywide audience

Description

This is a good first issue, and requires no JavaScript knowledge!

We recently launched Safe preview (#2563, take a look), but it has one missing feature!

When you open a notebook without running it, and you make a small change (eg fix a typo in a cell), Pluto will save the file, but the order of cells in the file can be drastically different, and you get a big git diff. Try this with one of your more complex notebooks to see what I mean, and take a look at which cells get moved and why.

This is also an issue currently in some more niche areas:

  • Pluto.activate_notebook_environment and Pluto.update_notebook_environment will modify the embedded pkg info (yay) but also mess up the cell order (boo).
  • If you open a notebook and shut it down before it finished, you have the same problem.

The cause

Pluto saves cells in the .jl file in an order that allows the cells to be executed as a standalone script. It essentially takes the visual cell order, and then we do a stable sort based on cell dependencies.

The reason why the issue happens is that Pluto often needs to evaluate your code to fully understand cell dependencies. The main examples are

  • using Plots defines the symbol scatter
  • a macro can create or remove variable definitions and references, e.g. @gensym x looks like it references x, but it actually defines x

You cannot derive this info statically, so we run code.

When you open a notebook without running it, we only have information that can be derived statically. When we save your notebook, we do the best we can using that information, but this can lead to a very different file order than what was previously derived (based on complete information), leading to a large diff.

Solution idea

My idea for solving this is: when we load a notebook file, also remember the original order in which cells appear in the file. Whenever you save a notebook, check if we have enough information to fully derive the order. If so, remove the memorized order and do as usual. If not, try to stick to the original file order as much as possible (i.e. use the original file order as a way to disambiguate the unknown cell deps).

Relevant code is here:

Pluto's reactivity algorithm. The core is this function that sorts a list of cells into search order https://github.com/fonsp/Pluto.jl/blob/v0.19.29-src/src/analysis/topological_order.jl#L10-L137

The output of this function is: https://github.com/fonsp/Pluto.jl/blob/v0.19.29-src/src/analysis/TopologicalOrder.jl

One of the inputs of this function is: https://github.com/fonsp/Pluto.jl/blob/v0.19.29-src/src/analysis/Topology.jl#L28-L36 which is a snapshot of the cells, and their symbol definitions etc that are known at one point in time.

The Notebook struct: https://github.com/fonsp/Pluto.jl/blob/v0.19.29-src/src/notebook/Notebook.jl#L24-L65

Reading and writing notebook files: https://github.com/fonsp/Pluto.jl/blob/v0.19.29-src/src/notebook/saving%20and%20loading.jl

For tests, search for the word "order" in this file: https://github.com/fonsp/Pluto.jl/blob/v0.19.29-src/test/React.jl Also search for only_versions_or_lineorder_differ in the source code.

Contributor guide