Azure ML Pipeline with V2 SDK
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
- Issue type
- Documentation
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- azure, jupyter-notebook, kubernetes
- Domain
- cloud, data-engineering, devops, machine-learning
Research direction
No specific file, test, or entry point is identified. Review the linked Azure ML training material and compare it with the described Blob, pipeline, DevOps, ACI, and AKS workflow. Done would be complete SDK V2 guidance for all listed steps, including the gaps the issue reports in existing examples.
Written by the indexing model from the issue text.
Description
Previously microsoft suggested us to learn Azure ML Pipeline (using SDK V1) & then use it for our model creation. Link for the training is https://learn.microsoft.com/en-us/training/paths/build-ai-solutions-with-azure-ml-service/.
The process that we followed:
- Azure Data Factory generates a new file every month for all the models & publishes in Blob storage.
- We have created and published Azure ML Pipeline for models which gets data from Blob, preprocess the data, train model, register the model, finds feature importance, finds data drift.
- Then we have used the Azure ML Pipeline ID in DevOps build pipeline, so that the ML Pipeline can be triggered through build Pipeline (The build pipeline gets triggered through Azure Logic App, whenever a new file is published to the Blob container).
- Post DevOps pipeline's completion, release pipeline deploys the model into ACI & AKS.
Now we have been suggested to use SDK V2 for all our model training & other processes.
Please suggest, how can we perform all the above steps using SDK V2. All the microsoft documents are incomplete to answer this.
As SDK V1 is legacy now, we are bound to move our code to V2. But SDK V2 examples are incomplete to address our issues.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 2.6k
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from Azure/MachineLearningNotebooks
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Azure/MachineLearningNotebooks#1975 · 1 comment ·
-
duplicates Open
Difficulty 1/5 Under an hour Newbie friendliness 68/100
Azure/MachineLearningNotebooks#1960 ·
-
machine Open
Difficulty 5/5 Over a week Newbie friendliness 10/100
Azure/MachineLearningNotebooks#1987 ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
Azure/MachineLearningNotebooks#1985 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
Azure/MachineLearningNotebooks#1981 · 1 reaction ·
All issues in Azure/MachineLearningNotebooks
Similar issues
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
palladius/rails8-app-on-gcp#141 · 1 comment ·
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
documentation improve or update documentation priority/low triage
Difficulty 2/5 Half a day Newbie friendliness 86/100
warpdotdev/docs#782 · 1 comment ·
-
customer-reported question
Difficulty 1/5 Under an hour Newbie friendliness 80/100
Azure/awesome-azd#1017 ·