AutoML experiment: get model and metrics for any algorithm (not only for the best one)

issue に示されている MLflow クライアントの使用方法から始めます。get_run、automl_best_child_run_id タグ、pipeline_id_000 が含まれます。AutoML の子 run がどのように識別されるか、またアルゴリズム名が内部の run 名とどのように対応するかを追跡します。実験内の最良の run だけでなく、任意のアルゴリズムについてモデルとそのメトリクスを取得する方法が文書化されるか、取得できるようになれば完了です。

What I'm trying to do

For an AutoML Forecasting experiment, I'd like to compare the performance of the best model with the performance of another model from the same experiment.

For an AutoML run, I understand how to get the best performing model and its metrics like this:

# ...initialize MLFlow client...
mlflow_parent_run = mlflow_client.get_run('upbeat_square_abs3942')
best_child_run_id = mlflow_parent_run.data.tags["automl_best_child_run_id"]
best_run = mlflow_client.get_run(best_child_run_id)
best_run.data.metrics
# etc...

But how can I fetch the job for any model based on the algorithm name?
Something like:

# pseudocode:
mlflow_client.get_automl_run_by_algorithm('XGBoostRegressor')

So far, I managed to figure out the following:

1. list of algorithms used in the AutoML experiment

mlflow_parent_run.data.tags['pipeline_id_000']
#  '__AutoML_Naive__;__AutoML_SeasonalNaive__;__AutoML_Average__;__AutoML_SeasonalAverage__;__AutoML_Ensemble__'

However, this list seems to be in an arbitrary order and I struggle to get the corresponding job names for the algorithms.

2. "internal" job names for the child runs

The child runs seem to have different names than the names shown in Azure ML Studio.
They are named for instance upbeat_square_abs3942_2 - i.e. the name of the parent run upbeat_square_abs3942 followed by underscore plus a number (_2in this example).

But Azure ML Studio displays names like (no upbeat_square_abs3942_2 to be found):

So this code works:

child_run = mlflow_client.get_run('upbeat_square_abs3942_2')

but using a name shown in the screenshot above throws an exception, e.g.

child_run = mlflow_client.get_run('green_floor_0ln3tlpv')

Question

How can I obtain the model and metrics for any algorithm used in the experiment?

Thanks!