nteract/papermill

Support for partial execution

Open

#797 opened on May 7, 2024

View on GitHub
 (0 comments) (0 reactions) (0 assignees)Python (5,381 stars) (402 forks)batch import
enhancementhelp wanted

Description

🚀 Feature

I do not know of this is already possible somehow.

Is it possible to launch papermill in a way that I can execute a notebook,

  • up to a specified cell
  • setting a range using cell numbers
  • setting a range using Markdown headers

Motivation

I have some notebooks that, even if relatively short, perform operations that might benefit to be split such as defining a list of files to download and actually try to download them. I am trying to avoid having to split the notebooks further.

Contributor guide

Tech stack
python
Domain
datatooling
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
3-5 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Pythonpapermill basicsJupyter notebook structure
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
To implement partial execution, inspect papermill's execution module (e.g., papermill/execute.py) to understand how notebooks are executed cell by cell. Consider adding a parameter like 'stop at' accepting a cell index or header, and modify the execution loop to break after that cell. Review how parameters are passed and ensure compatibility. Since no PRs or comments exist, this is a fresh feature requiring exploration of the codebase.