Artificact upload for remote instances
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- julia
- Domain
- api, machine-learning
Research direction
Start in src/loggers.jl around lines 69-72 and inspect logartifact and getrun to understand how artifact_uri is currently handled. Decide with maintainers how local and remote artifact access should be distinguished, then verify the chosen implementation against a local MLflow instance and the GitLab endpoints; done means remote artifact uploads work without breaking local access.
Written by the indexing model from the issue text.
Description
As per https://github.com/JuliaAI/MLFlowClient.jl/blob/a31b41a91dc32107fa7dcd6ad5a581c116a4ff9c/src/loggers.jl#L69-72 we currently can't upload artifacts to remote servers
Assumes that artifact_uri is mapped to a local directory.
At the moment, this only works if both MLFlow and the client are running on the same host or they map a directory that leads to the same location over NFS, for example.
Some (untested) code along the lines of:
function logartifact(mlf::GitLabMLFlow, run_id::AbstractString, basefilename::AbstractString, data)
mlflowrun = getrun(mlf, run_id)
artifact_uri = mlflowrun.info.artifact_uri
filepath = joinpath(artifact_uri, basefilename)
try
HTTP.post(url, push!(copy(mlf.headers), "Content-Type" => "multipart/form-data" ),
HTTP.Form(Dict(:file => data)))
catch e
error("Unable to upload $(filepath): $e")
end
filepath
end
Would probably work for this purpose and not introduce new dependencies.
The question remains how to differentiate between local and remote file access, as they happen through different interfaces (the headers are irrelevant for file access for example).
I see three options to approach this and would be interested in making a pull request implementing one of these and testing them against a local mlflow instance and the gitlab end points to confirm they are working (for me).
- Use a different subtype of
AbstractMLFlowto denote remote instances. (Probably not preferred) - Keep using a String and just have an if checking for prefixes of "https://" and use HTTP then.
- Switch to using URIs and use file:// to denote file access and check the prefix in an if statement.
This is simply a question of taste i would leave to (some of) the maintainers.
- Dominant language
- Julia
- Stars
- 59
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from JuliaAI/MLFlowClient.jl
-
tagbot-manual
Difficulty 1/5 Under an hour Newbie friendliness 58/100
JuliaAI/MLFlowClient.jl#83 ·
-
Migrate backend from raw REST API to PythonCall.jl + CondaPkg.jlMay be free again @pebeto claimed this 242 days ago, and no pull request is open. Opendocumentation enhancement help wanted
JuliaAI/MLFlowClient.jl#79 · 5 comments · 1 assignee ·
-
need a tutorial for using logger with dagshub and mlflowMay be free again @pebeto claimed this 1049 days ago, and no pull request is open. Opengood first issue
JuliaAI/MLFlowClient.jl#84 · 9 comments · 1 assignee ·
-
enhancement help wanted
Difficulty 4/5 3-5 days Newbie friendliness 35/100
JuliaAI/MLFlowClient.jl#31 · 8 comments ·
All issues in JuliaAI/MLFlowClient.jl
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
Sienna-Platform/PowerSimulations.jl#1737 ·
Maintainers usually reply within 4 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
FluxML/Optimisers.jl#254 ·
-
Minor: floodplain warning percentage, scalar output error message and a never-true identity checkOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
JuliaAI/MLJTransforms.jl#101 ·
Maintainers usually reply within 1 day
-
@contents blocks fail to render on Julia 1.14Possibly taken @sbfnk claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
LuxDL/DocumenterVitepress.jl#410 ·