Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

[azureml python sdk v2] access files in URI_FOLDER output after job has finished?

Open
#1,891 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
azure, jupyter-notebook, python
Domain
data, machine-learning

Research direction

Start with the Azure ML v2 SDK job and NodeOutput APIs shown in the issue, then compare them with the MlflowClient access path. Done means documenting or demonstrating a programmatic way to list, retrieve, and download the URI_FOLDER files after job ivory_octopus_yd6by49kxf finishes.

Written by the indexing model from the issue text.

Description

I have a training job that persists some files in an URI_FOLDER output.
How can I access those through the v2 SDK API after the job has finished?

1. job setup

The output is set up like this in the command:

job = command(
    # ...
    outputs=dict(
        outputs=Output(type=AssetTypes.URI_FOLDER, mode='rw_mount'),
    ),
    command="python training_script.py " + 
            "--outputs_dir ${{outputs.outputs}} " +
            # ...other arguments...
)

This seems to work fine, the corresponding folder is mounted correctly and accessible in the training script.

2. training script

In the training script, I persist a dataframe like this:

parser.add_argument("--outputs_dir", dest="outputs_dir", default=DEFAULT_MODEL_DIR)
# ...
some_dataframe.to_csv(os.path.join(args.outputs_dir, 'some_dataframe.csv'), index=True)

This works fine.

3. resulting dataset

After the job has finished, the outputs are available as a dataset.
This is what is shown in Azure ML Studio in the "Overview" tab for job ivory_octopus_yd6by49kxf:
image

The dataset is successfully stored in the workspaceblobstore datastore. I checked it in the Azure ML Studio and it looks fine.

4. accessing the persisted data

After the job has finished, I access the run using a MlflowClient()

MLFLOW_TRACKING_URI = ml_client.workspaces.get(name=ml_client.workspace_name).mlflow_tracking_uri
mlflow.set_tracking_uri(MLFLOW_TRACKING_URI)
mlflow_client = MlflowClient()

mlflow_run = mlflow_client.get_run("ivory_octopus_yd6by49kxf")

or

run = ml_client.jobs.get('ivory_octopus_yd6by49kxf')
# returns NodeOutput class

How can I programmatically list / get / download the outputs connected to the job?

Thanks!

Dominant language
Jupyter Notebook
Stars
4.4k
Forks
2.6k
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Azure/MachineLearningNotebooks

All issues in Azure/MachineLearningNotebooks

Similar issues

More Data Engineering issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.