Ensure Control Plane Availability During Kubeflow SDK Client Creation

What you would like to be added?

As we discussed in this Slack thread, we want to expose public ConfigMap in the system namespace that tells which version of Kubeflow control plane components are installed (Trainer, Pipelines, Spark Operator, etc.)

This ConfigMaps should be accessible by any user with system:authenticated RBAC, and have these names:

kubeflow-trainer-public
kubeflow-pipelines-public
kubeflow-spark-public
kubeflow-hub-public

Before user tries to use corresponding clients in Kubeflow SDK, we should verify that the appropriate version of control plane is deployed (e.g. kubeflow_trainer_api == Trainer version).

We can start with Kubeflow Trainer, and introduce similar functionality for other clients.

• Kubeflow Trainer
  • https://github.com/kubeflow/trainer/pull/3083
• Kubeflow Pipelines
  • https://github.com/kubeflow/pipelines/pull/12717
• Kubeflow Spark Operator
• Kubeflow Hub

cc @kubeflow/wg-pipeline-leads @kubeflow/kubeflow-trainer-team @kubeflow/kubeflow-sdk-team @juliusvonkohout @kubeflow/wg-data-leads @Shekharrajak

Why is this needed?

Verifies that Kubeflow SDK clients can be used.

Love this feature?

Give it a 👍 We prioritize the features with most 👍