Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Pipeline parameters used with DataPath and DataPathComputeBinding to specify side inputs of Parallel pipeline

Open
#1,801 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
azure, python

Research direction

Start with the PipelineParameter API page and the linked AzureML-Docset source file, then reproduce the example using azureml-core==1.40.0.post2 and azureml-pipeline==1.40.0. Compare the documented DataPath/DataPathComputeBinding usage with the reported ParallelRunStep error; done means the documentation or supported-version guidance accurately reflects the behavior.

Written by the indexing model from the issue text.

Description

[Enter feedback here]
I'm following this example to create a PipelineParameters for my Parallel pipeline

from azureml.core.datastore import Datastore
from azureml.data.datapath import DataPath, DataPathComputeBinding
from azureml.pipeline.steps import PythonScriptStep
from azureml.pipeline.core import PipelineParameter

datastore = Datastore(workspace=workspace, name="workspaceblobstore")
datapath = DataPath(datastore=datastore, path_on_datastore='input_data')
data_path_pipeline_param = (PipelineParameter(name="input_data", default_value=datapath),
                           DataPathComputeBinding(mode='mount'))

train_step = PythonScriptStep(script_name="train.py",
                             arguments=["--input", data_path_pipeline_param],
                             inputs=[data_path_pipeline_param],
                             compute_target=compute_target,
                             source_directory=project_folder)

This is my code to create the pipeline with the parameters

path = DataPath(datastore=default_store, path_on_datastore='path')
input_param= (PipelineParameter(name="param_name", default_value=path), DataPathComputeBinding(mode='mount'))

parallel_run_config = ParallelRunConfig(
    source_directory=script_dir,
    entry_script='script.py',  # the user script to run against each input
    partition_keys=['key'],
    error_threshold=50,
    output_action='append_row',
    environment=environment,
    compute_target=compute_target, 
    node_count=2,
    run_invocation_timeout=1200
)

parallel_run_step = ParallelRunStep(
    name='test-batch-inference',
    inputs=[partition_input],
    side_inputs=[input1, input2, input_param],
    output=output_dir,
    parallel_run_config=parallel_run_config,
    arguments=['--input_param', input_param],
    allow_reuse=False
)

And it raised this error:

Exception: Step input must be of any type: (<class 'azureml.data.dataset_consumption_config.DatasetConsumptionConfig'>, <class 'azureml.pipeline.core.pipeline_output_dataset.PipelineOutputFileDataset'>, <class 'azureml.pipeline.core.pipeline_output_dataset.PipelineOutputTabularDataset'>, <class 'azureml.data.output_dataset_config.OutputFileDatasetConfig'>, <class 'azureml.data.output_dataset_config.OutputTabularDatasetConfig'>, <class 'azureml.data.output_dataset_config.LinkFileOutputDatasetConfig'>, <class 'azureml.data.output_dataset_config.LinkTabularOutputDatasetConfig'>), found <class 'tuple'>

I'm using azureml-core==1.40.0.post2, azureml-pipeline==1.40.0
It's seems like the sample code is not supported with these version? Before trying this datapath as pipeline parameter, I tried int type input and its just work fine


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
2.6k
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Azure/MachineLearningNotebooks

All issues in Azure/MachineLearningNotebooks

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.