Netflix/metaflow

Package code in more cases

Open

#124 opened on Feb 12, 2020

View on GitHub
 (10 comments) (6 reactions) (0 assignees)Python (7,202 stars) (695 forks)batch import
help wantedupforgrabs

Description

Currently, the code is packaged for Batch runs only. To be able to fully replicate and version things, it would be nicer if code was always stored as an artifact of each step.

Contributor guide

Tech stack
python
Domain
datamachine learning
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
Pythonmetaflow basics
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
Investigate how Metaflow currently packages code for Batch runs (e.g., in the 'step' or 'run' modules). Look for existing code in the repository that handles artifact packaging, and generalize it to be triggered for all runs, not just Batch. Check the issue comments for any suggested approaches or concerns.