Netflix/metaflow

Improve visibility of Metaflow Code object

Open

#1,009 opened on Apr 13, 2022

View on GitHub
 (1 comment) (0 reactions) (0 assignees)Python (7,202 stars) (695 forks)batch import
enhancementhelp wanted

Description

The code tarball saved by Metaflow is accessible with the client but it is not very easy to work with. We may want to make it more visible possibly even in the Metaflow UI.

Another idea would be to link it to the source control that the Metaflow code is in (if there is one). In particular, we could store the branch/commit information as well as a "dirty" patch file if needed. We can keep the same tarball to work internally but also linking it to the user's development environment may make it easier to work with.

Contributor guide

Tech stack
python
Domain
backenddata
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
Basic familiarity with MetaflowPython programmingUnderstanding of version control (git)
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30
Research direction
Investigate the current Metaflow client API for Code objects, specifically in the `metaflow/client.py` or similar files. Review the Metaflow UI code to understand how code visibility is currently handled. Check if there are any existing methods or properties for linking to source control. The single comment on issue #1009 may provide additional context. Consider exploring how to store branch/commit info and dirty patch in the tarball metadata, and propose a method on the Code object to expose this information.