nteract/papermill

Question: Documentation for supported data types of parameters? (E.g. What is proper way to use tuples with Papermill?)

Open

#586 opened on Apr 6, 2021

View on GitHub
 (5 comments) (0 reactions) (0 assignees)Python (402 forks)batch import
enhancementhelp wantednew-contributor-friendly

Repository metrics

Stars
 (5,381 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

MWE:

input_notebook.ipynb has two cells, a parameters cell:

 numbers = None

and a code execution cell

 small, big = numbers

papermill_notebook.ipynb has one cell with the code:

import papermill as pm

pm.execute_notebook(
   'input_notebook.ipynb',
   'output_notebook.ipynb',
   parameters = {'numbers':(-1,1)}
)

Tuples are a basic, immutable datatype. The parameter is a tuple whose entries are also immutable and even more basic.

However, attempting to run papermill_notebook.ipynb leads to:

PapermillExecutionError: 
---------------------------------------------------------------------------
Exception encountered at "In [2]":
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-3-4b9474dadad0> in <module>
----> 1 small, big = numbers

ValueError: too many values to unpack (expected 2)

This is because papermill converted the length two tuple, (-1,1), to the length 7 string "(-1,1)".

This definitely wasn't the behavior I was expecting, which seems to suggest I don't sufficiently understand the "philosophy" of Papermill. Question/Request: Would it be possible to point to a relevant portion of the documentation explaining to users which data types they should and should not expect to be able to pass as parameters?

I haven't actually looked at the papermill source code in detail yet, but I'm guessing it has something to do with JSON serialization, and the fact that there is no notion of 'tuple' in JSON?

(E.g. when I tried to pass a lambda as a parameter once I got a '... not JSON serializable error', so maybe papermill is converting to JSON to avoid security risks associated with pickle or whatever.)

For example the above example works using parameters = {'numbers':[-1,1]} (i.e. a list) instead, presumably because lists correspond to the JSON array objects so are serializable without first converting to string?

Contributor guide