kubeflow/sdk

[SDK] Get the correct TrainJob components using `get_job()` API

Open

#25 geöffnet am 11. Dez. 2024

Auf GitHub ansehen
 (26 Kommentare) (1 Reaktion) (3 zugewiesene Personen)Python (196 Forks)auto 404
good first issuehelp wantedkind/feature

Repository-Metriken

Stars
 (124 Stars)
PR-Merge-Metriken
 (PR-Metriken ausstehend)

Beschreibung

What you would like to be added?

As we discussed, currently get_job() API can return multiple Pods for every TrainJob component, like initializer or trainer-node-0: https://github.com/kubeflow/training-operator/pull/2324#discussion_r1860773473. That can happen when Pods are re-created based on Batch/Job restart policies. Therefore, users can see unexpected logs while using the Kubeflow Training SDK.

We should improve this API to show the correct TrainJob components to users. For example, when we list all of the Pods, we can select the most recently created Pod with the same role (e.g. dataset-initializer).

Love this feature?

Give it a 👍 We prioritize the features with most 👍

Contributor Guide