Mark daprd as a sidecar container using the SideCar type in the container's lifecycle spec

In what area(s)?

/area operator

Describe the feature

I noticed that the application container might be ready before the daprd one, especially because of the initial delay added to the daprd container. Since K8s 1.18, it is possible to mark a container as a SideCar so that K8s makes sure the SideCar container starts before others. By reverse-engineering an injected Pod, I could see that this is not used today by Dapr. It would certainly be a breaking change but since you're still in Alpha mode, that might be the right time.

To illustrate this, if I write the follwowing busybox:

apiVersion: v1
kind: Pod
metadata:
  name: busyboxwithdapr
  namespace: azug
  labels:
    app: busyboxwithdapr
    aadpodidbinding: busyboxidentity
  annotations:        
        dapr.io/enabled: "true"        
        dapr.io/log-level: "debug"
spec:
  containers:
  - image: yauritux/busybox-curl
    command: [ "/bin/sh", "-c", "--" ]
    args: [ "while true; do echo busybox;curl --request GET http://127.0.0.1:3500/v1.0/secrets/mysecretstore/storageKey; sleep 31; done;" ]
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always

that immediately calls a secret store, I initially get errors back from the curl command because daprd isn't ready yet.

busybox
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed connect to 127.0.0.1:3500; Connection refused
busybox
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   105  100   105    0     0    172      0 --:--:-- --:--:-- --:--:--   172
{"storageKey":"o2q620...=="}busybox
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

This could cause the application container (not in this case) not to restart properly in case of poor exception handling and with a restartPolicy of Never.