GoogleContainerTools/skaffold
View on GitHubSupport configuring remote cluster for pulling images
Open
#4,132 opened on May 8, 2020
area/confighelp wantedkind/feature-requestkind/frictionpriority/p3
Description
I'm trying to build a container on GCB, and then deploy it on my local Docker for Desktop Kubernetes cluster, but I seem to be hitting auth issues. The container builds fine, but the Kubernetes pod gets stuck in an error state. I went through https://cloud.google.com/container-registry/docs/advanced-authentication#gcloud-helper and set gcloud as the credential helper via gcloud auth configure-docker, but the problem still seems to persist.
Using this example: https://github.com/GoogleCloudPlatform/cloud-code-samples/tree/master/java/java-hello-world
$ pwd
/Users/michihara/Code/cloud-code-samples/java/java-hello-world
$ skaffold version
v1.8.0
$ gcloud version
Google Cloud SDK 289.0.0
alpha 2020.02.25
app-engine-java 1.9.79
app-engine-python 1.9.90
beta 2020.02.25
bq 2.0.56
cloud-datastore-emulator 2.1.0
core 2020.04.10
gsutil 4.49
minikube
skaffold
$ gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
* michihara@google.com
To set the active account, run:
$ gcloud config set account `ACCOUNT`
$ gcloud auth configure-docker
WARNING: Your config file at [/Users/michihara/.docker/config.json] contains these credential helper entries:
{
"credHelpers": {
"us.gcr.io": "gcloud",
"asia.gcr.io": "gcloud",
"marketplace.gcr.io": "gcloud",
"gcr.io": "gcloud",
"eu.gcr.io": "gcloud",
"staging-k8s.gcr.io": "gcloud"
}
}
Adding credentials for all GCR repositories.
WARNING: A long list of credential helpers may cause delays running 'docker build'. We recommend passing the registry name to configure only the registry you are using.
gcloud credential helpers already registered correctly.
$ skaffold run -p cloudbuild --default-repo gcr.io/chelseamarket/p
Generating tags...
- java-hello-world -> gcr.io/chelseamarket/p/java-hello-world:latest
Checking cache...
- java-hello-world: Found Remotely
Tags used in deployment:
- java-hello-world -> gcr.io/chelseamarket/p/java-hello-world:latest@sha256:446df95cdad401e4aadf5ccc137648a725ba8d7f25aa0a2b26017f4f262f1997
Starting deploy...
- deployment.apps/java-hello-world configured
- service/java-hello-world-external configured
Waiting for deployments to stabilize...
- deployment/java-hello-world: waiting for rollout to finish: 1 old replicas are pending termination...
- deployment/java-hello-world failed. Error: could not stabilize within 2m0s: context deadline exceeded.
FATA[0124] 1/1 deployment(s) failed
$ kubectl config current-context
docker-desktop
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/java-hello-world-6d47c78bf9-xpm7f 0/1 ImagePullBackOff 0 36s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/java-hello-world-external LoadBalancer 10.101.0.92 localhost 80:30743/TCP 37s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 45s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/java-hello-world 0/1 1 0 37s
NAME DESIRED CURRENT READY AGE
replicaset.apps/java-hello-world-6d47c78bf9 1 1 0 37s
$ kubectl describe pod/java-hello-world-6d47c78bf9-xpm7f
Name: java-hello-world-6d47c78bf9-xpm7f
Namespace: default
Priority: 0
Node: docker-desktop/192.168.65.3
Start Time: Fri, 08 May 2020 11:27:08 -0400
Labels: app=java-hello-world
app.kubernetes.io/managed-by=skaffold-v1.8.0
pod-template-hash=6d47c78bf9
skaffold.dev/builder=google-cloud-build
skaffold.dev/cleanup=true
skaffold.dev/deployer=kubectl
skaffold.dev/profile.0=cloudbuild
skaffold.dev/run-id=3cffaa33-5716-4371-92ed-43be9f3a1d70
skaffold.dev/tag-policy=sha256
skaffold.dev/tail=true
Annotations: <none>
Status: Pending
IP: 10.1.1.133
IPs: <none>
Controlled By: ReplicaSet/java-hello-world-6d47c78bf9
Containers:
server:
Container ID:
Image: gcr.io/chelseamarket/p/java-hello-world:latest@sha256:446df95cdad401e4aadf5ccc137648a725ba8d7f25aa0a2b26017f4f262f1997
Image ID:
Port: 8080/TCP
Host Port: 0/TCP
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment:
PORT: 8080
JAVA_TOOL_OPTIONS: -agentlib:jdwp=transport=dt_socket,server=y,address=5005,suspend=n,quiet=y
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-q4ngt (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-q4ngt:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-q4ngt
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 49s default-scheduler Successfully assigned default/java-hello-world-6d47c78bf9-xpm7f to docker-desktop
Normal BackOff 24s (x3 over 47s) kubelet, docker-desktop Back-off pulling image "gcr.io/chelseamarket/p/java-hello-world:latest@sha256:446df95cdad401e4aadf5ccc137648a725ba8d7f25aa0a2b26017f4f262f1997"
Warning Failed 24s (x3 over 47s) kubelet, docker-desktop Error: ImagePullBackOff
Normal Pulling 13s (x3 over 48s) kubelet, docker-desktop Pulling image "gcr.io/chelseamarket/p/java-hello-world:latest@sha256:446df95cdad401e4aadf5ccc137648a725ba8d7f25aa0a2b26017f4f262f1997"
Warning Failed 12s (x3 over 47s) kubelet, docker-desktop Failed to pull image "gcr.io/chelseamarket/p/java-hello-world:latest@sha256:446df95cdad401e4aadf5ccc137648a725ba8d7f25aa0a2b26017f4f262f1997": rpc error: code = Unknown desc = Error response from daemon: unauthorized: You don't have the needed permissions to perform this operation, and you may have invalid credentials. To authenticate your request, follow the steps in: https://cloud.google.com/container-registry/docs/advanced-authentication
Warning Failed 12s (x3 over 47s) kubelet, docker-desktop Error: ErrImagePull