kubernetes/minikube

registry-creds addon: secrets stored with different name to defaults

Open

#2,805 opened on May 11, 2018

View on GitHub
 (14 comments) (0 reactions) (1 assignee)Go (31,799 stars) (5,222 forks)batch import
addon/registry-credsgood first issuehelp wantedkind/buglifecycle/frozenpriority/backlog

Description

Environment:

Minikube version (use minikube version): v0.26.1

  • OS (e.g. from /etc/os-release): MacOS 10.13.4 High Sierra
  • VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): hyperkit
  • ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): v0.26.0
  • Install tools: homebrew

What happened: Using private ECR registry images with registry-creds addon.

What you expected to happen: Installing and configuring registry-creds with valid credentials would allow ECR images to be retrieved by minikube while creating pods.

How to reproduce it (as minimally and precisely as possible):

  1. minikube addons configure registry-creds
  2. Configure the credentials with a valid private AWS keypair and ECR registry ID/region.
  3. Configure a deployment/replicaset/replicationcontroller/etc with an image located in that private registry.
  4. Image fails to download.

Output of minikube logs (if applicable):

2018-05-11 16:53:17 +0200 CEST   2018-05-11 16:51:41 +0200 CEST   4         user-interface-9789bc6d8-wwjns.152d9ea9f11e04d0    Pod          spec.containers{user-interface}   Warning   Failed                    kubelet, minikube       Failed to pull image "XXXX.dkr.ecr.us-east-1.amazonaws.com/XXXX/XXXX": rpc error: code = Unknown desc = Error response from daemon: Get https://XXXX.dkr.ecr.us-east-1.amazonaws.com/v2/XXXX/XXXX/manifests/latest: no basic auth credentials

Anything else do we need to know:

  • Installing the registry-creds addon causes credentials to be stored in secrets named as follows:
$ k get secrets --namespace=kube-system | grep registry-creds                                               17:27:36
registry-creds-dpr                               Opaque                                3         35m
registry-creds-ecr                               Opaque                                6         35m
registry-creds-gcr                               Opaque                                2         35m
  • According to the source of registry-creds, the default secret name for AWS credentials in fact awsecr-cred
  • No configuration seems to be provided to registry-creds to override this value, so it is left looking for secrets that don't exist:
time="2018-05-11T15:08:27Z" level=error msg="Error getting secret: secrets "awsecr-cred" not found"
2018/05/11 15:08:27 Finished processing secret for namespace default, secret awsecr-cred
time="2018-05-11T15:08:27Z" level=error msg="Error getting secret: secrets "dpr-secret" not found"
2018/05/11 15:08:27 Finished processing secret for namespace default, secret dpr-secret
2018/05/11 15:08:27 Refreshing credentials...
time="2018-05-11T15:08:27Z" level=info msg="------------------ [gcr-secret] ----------------------
"
time="2018-05-11T15:08:27Z" level=info msg="Error getting secret for provider gcr-secret. Skipping secret provider! [Err: google: error getting credentials using well-known file (/root/.config/gcloud/application_default_credentials.json): invalid character 'c' looking for beginning of value]"
  • Re-storing the secret at the default path appears to work.

Contributor guide