Multi Cluster Support

Bắt đầu bằng việc xem xét issue #1307 và PR #2499 được liên kết để hiểu cách tiếp cận trước đây. Không có tệp hoặc bài kiểm thử nào được nêu tên; được coi là hoàn thành khi hỗ trợ các chú thích ngữ cảnh theo từng controller và thêm một probe khởi động để xác minh rằng tất cả các ngữ cảnh mong muốn đều khả dụng.

As discussed in #1307 it would be nice to have cluster a context setting for each controller.

Why?
Currently a operator can only a act on a single context. Traditionally that's fine as an operator often orchestrates only resources in the very cluster its deployed it self. In this scenario the operator gets its identity natively from the kubernetes context and is statically configured for its own cluster.
In contrast to that we have a use case where we manage multiple clusters with on operator which is deployed to a meta/central cluster. The operator still manages resources within this meta cluster but to instantiate the desired application it will also manage child/runtime clusters.

Example:

The control cluster manages a CRD which has been instantiated twice. The operator picks them up and deploys resources to fulfil the desired resource. To do that the operator has to deploy the application into multiple clusters which are e.g. physically separated and therefore can't have a single identity or are separated due to availability needs or to test new versions of kuberentes etc. Therefore a full replication of the resources is created in each cluster and the operator watches all of them.

Control Cluster
┌──────────────────────────────┐  Cluster A
│                              │ ┌──────────────────────────────────────────┐
│    Namespace-crds            │ │  Namespace-A            Namespace-B      │
│    ┌──────────────┐      ┌───┼─┤►┌─────────────┐       ┌─────────────┐    │
│    │              │      │   │ │ │             │       │             │    │
│    │ CRD-A        │      │   │ │ │ Deployment  │       │  Deployment │    │
│    │ CRD-B        │      │   │ │ │             │       │             │    │
│    │              │      │   │ │ └─────────────┘       └─────────────┘    │
│    └─────▲────────┘      │   │ │                                          │
│          │               │   │ └──────────────────────────────────────────┘
│          │ Reconcile     │   │
│          │               │   │  Cluster B
│     ┌────┴────────┬──────┘   │  ┌──────────────────────────────────────────┐
│     │             │          │  │  Namespace-A            Namespace-B      │
│     │  Operator   ├──────────┼──┤►┌─────────────┐       ┌─────────────┐    │
│     │             │  Create/ │  │ │             │       │             │    │
│     └─────────────┘  Reconcile  │ │ Deployment  │       │  Deployment │    │
│                              │  │ │             │       │             │    │
│                              │  │ └─────────────┘       └─────────────┘    │
│                              │  │                                          │
└──────────────────────────────┘  └──────────────────────────────────────────┘

Goals:

• add a context setting for the controller annotation so context can be in addition to the namespace for each reconciler individually
• add startup probe to ensure all desired context are available