mlflow/mlflow

[FR] Block registered models from being updated

Aperta

#3175 aperta il 27 lug 2020

 (10 commenti) (4 reazioni) (0 assegnatari)Python (3904 fork)batch import
area/model-registryarea/trackingenhancementhelp wantedneeds design

Metriche repository

Star
 (17.127 stelle)
Metriche merge PR
 (Merge medio 2g 9h) (246 PR mergiate in 30 g)

Descrizione

Willingness to contribute

The MLflow Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature (either as an MLflow Plugin or an enhancement to the MLflow code base)?

  • Yes. I can contribute this feature independently.
  • Yes. I would be willing to contribute this feature with guidance from the MLflow community.
  • No. I cannot contribute this feature at this time.

Proposal Summary

Editing/updating of an experiment run should not be possible if the run has been registered in the model registry.

Motivation

  • What is the use case for this feature? Currently you can edit/update an existing experiment run after it has been registered as a model in the model registry. This makes it possible to break the traceability of registered models, making models possibly less reliable.
  • Why is this use case valuable to support for MLflow users in general? Once a model is registered MLflow users should be able to assume that it is "frozen" in place, and cannot be changed anymore.
  • Why is this use case valuable to support for your project(s) or organization?
  • Why is it currently difficult to achieve this use case? (please be as specific as possible about why related MLflow features and components are insufficient)

What component(s), interfaces, languages, and integrations does this feature affect?

Components

  • area/artifacts: Artifact stores and artifact logging
  • area/build: Build and test infrastructure for MLflow
  • area/docs: MLflow documentation pages
  • area/examples: Example code
  • area/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registry
  • area/models: MLmodel format, model serialization/deserialization, flavors
  • area/projects: MLproject format, project running backends
  • area/scoring: Local serving, model deployment tools, spark UDFs
  • area/tracking: Tracking Service, tracking client APIs, autologging

Interfaces

  • area/uiux: Front-end, user experience, JavaScript, plotting
  • area/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
  • area/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registry
  • area/windows: Windows support

Languages

  • language/r: R APIs and clients
  • language/java: Java APIs and clients

Integrations

  • integrations/azure: Azure and Azure ML integrations
  • integrations/sagemaker: SageMaker integrations

Details

My 2cts on what the solution to this should be: once an experiment run is registered as a model, it should be marked as "frozen". This would allow non-registered experiment runs to still be updated, while protecting registered models from changing/breaking. This would probably mean a change to the tracking API as it should reject changes to an experiment run that has a model registered to it. I think this issue is related to #3160. I'm happy to help out implementing this, if this is deemed a valid approach to resolving the problem.

Guida contributor