Cell-level caching
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start by reviewing ipykernel's notebook execution path and the referenced akernel approach, including its use of Python's ast module to compare cached cell inputs and outputs. Compare that design with knitr's caching model and clarify how caching should serve Quarto and JupyterBook. Done requires an agreed cell-level caching design for ipykernel, including when execution may be skipped and how validity is established.
Written by the indexing model from the issue text.
Description
Often we want to visualize the content of the notebook for example as an HTML and also recompute it to check its validity. This ensures that the notebook is runnable and also shows the desired output. However, running a notebook is very time-consuming. It would be great if we had some cell-level cache that would speed up repeated executing of the notebook.
@davidbrochart in his project akernel (asynchronous kernel) solved a similar issue. He approached it by parsing inputs and outputs of the cached cell using ast module. He could then skip the execution of the particular time-consuming cell, if the outputs were cached and the inputs hasn't changed from the previous execution. I hope I didn't misinterpret it. @davidbrochart please comment on any of my mistake.
An inspiration could also be taken from knitr from R, who has this implementation available.
Could we get something like this for ipykernel? Many popular projects would benefit from this improvement, such as Quarto or JupyterBook.
- Dominant language
- Python
- Stars
- 734
- Forks
- 411
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 9
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ipython/ipykernel
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 4/5 3-5 days Newbie friendliness 68/100
-
ipython/ipykernel#1550 · 1 comment · 1 reaction · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 66/100
All issues in ipython/ipykernel
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
syfoud/Simulated_Scepter#172 ·
-
A cancelled tests run makes the coverage comment workflow fail and reports it as a red check on main Openarea: ci bug perceived difficulty: 3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Nitjsefnie-Harness-Commons/daedalus#921 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
EleutherAI/lm-evaluation-harness#4207 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ClickHouse/clickhouse-connect#1057 ·