kubernetes/minikube

Add solution message when VPN is causing "ssh: handshake failed"

Open

#9,914 opened on Dec 10, 2020

View on GitHub
 (13 comments) (2 reactions) (0 assignees)Go (31,799 stars) (5,222 forks)batch import
help wantedkind/buglifecycle/frozenneeds-solution-messagepriority/backlog

Description

minikube start --extra-config=apiserver.Authorization.Mode=RBAC --cpus 5 --memory 16000 -p minikube

Steps to reproduce the issue:

  1. Use the above command to start minikube

Full output of failed command:

I1210 10:54:07.651091   36599 out.go:185] Setting OutFile to fd 1 ...
I1210 10:54:07.651256   36599 out.go:237] isatty.IsTerminal(1) = true
I1210 10:54:07.651266   36599 out.go:198] Setting ErrFile to fd 2...
I1210 10:54:07.651276   36599 out.go:237] isatty.IsTerminal(2) = true
I1210 10:54:07.651387   36599 root.go:279] Updating PATH: /home/erahbaw/.minikube/bin
I1210 10:54:07.651631   36599 out.go:192] Setting JSON to false
I1210 10:54:07.673951   36599 start.go:103] hostinfo: {"hostname":"elx79232bqz","uptime":2384,"bootTime":1607575463,"procs":442,"os":"linux","platform":"ubuntu","platformFamily":"debian","platformVersion":"18.04","kernelVersion":"4.15.0-122-generic","virtualizationSystem":"kvm","virtualizationRole":"host","hostid":"5f39a65d-1138-2183-3921-94185c82cc75"}
I1210 10:54:07.674472   36599 start.go:113] virtualization: kvm host
I1210 10:54:07.684853   36599 out.go:110] 😄  minikube v1.15.1 on Ubuntu 18.04
😄  minikube v1.15.1 on Ubuntu 18.04
I1210 10:54:07.685237   36599 notify.go:126] Checking for updates...
I1210 10:54:07.685890   36599 driver.go:302] Setting default libvirt URI to qemu:///system
I1210 10:54:07.739340   36599 docker.go:117] docker version: linux-19.03.2
I1210 10:54:07.739415   36599 cli_runner.go:110] Run: docker system info --format "{{json .}}"
I1210 10:54:07.815245   36599 info.go:253] docker info: {ID:CEOT:UV3A:CES6:TU44:BEYI:EIMU:OJQC:FYV2:BLCY:CENC:3BXL:URGJ Containers:1 ContainersRunning:0 ContainersPaused:0 ContainersStopped:1 Images:4608 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:false KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:24 OomKillDisable:true NGoroutines:38 SystemTime:2020-12-10 10:54:07.773605988 +0530 IST LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:4.15.0-122-generic OperatingSystem:Ubuntu 18.04.5 LTS OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33568440320 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:elx79232bqz Labels:[] ExperimentalBuild:false ServerVersion:19.03.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:894b81a4b802e4eb2a91d1ce216b8817763c29fb Expected:894b81a4b802e4eb2a91d1ce216b8817763c29fb} RuncCommit:{ID:425e105d5a03fabd737a126ad93d62a9eeede87f Expected:425e105d5a03fabd737a126ad93d62a9eeede87f} InitCommit:{ID:fec3683 Expected:fec3683} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:[WARNING: No swap limit support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker Application Vendor:Docker Inc. Version:v0.8.0] map[Experimental:true Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.3.0-5-g5b97415-tp-docker]] Warnings:<nil>}}
I1210 10:54:07.815329   36599 docker.go:147] overlay module found
I1210 10:54:07.816118   36599 out.go:110] ✨  Using the docker driver based on existing profile
✨  Using the docker driver based on existing profile
I1210 10:54:07.816137   36599 start.go:272] selected driver: docker
I1210 10:54:07.816144   36599 start.go:686] validating driver "docker" against &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.14@sha256:2bd97b482faf5b6a403ac39dd5e7c6fe2006425c6663a12f94f64f5f81a7787e Memory:16000 CPUs:5 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio KubernetesConfig:{KubernetesVersion:v1.19.4 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: ExtraOptions:[{Component:apiserver Key:Authorization.Mode Value:RBAC}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.19.4 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[]}
I1210 10:54:07.816204   36599 start.go:697] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Fix: Doc:}
I1210 10:54:07.816862   36599 cli_runner.go:110] Run: docker system info --format "{{json .}}"
I1210 10:54:07.890864   36599 info.go:253] docker info: {ID:CEOT:UV3A:CES6:TU44:BEYI:EIMU:OJQC:FYV2:BLCY:CENC:3BXL:URGJ Containers:1 ContainersRunning:0 ContainersPaused:0 ContainersStopped:1 Images:4608 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff true]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:false KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:24 OomKillDisable:true NGoroutines:38 SystemTime:2020-12-10 10:54:07.849652341 +0530 IST LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:4.15.0-122-generic OperatingSystem:Ubuntu 18.04.5 LTS OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:8 MemTotal:33568440320 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:elx79232bqz Labels:[] ExperimentalBuild:false ServerVersion:19.03.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:894b81a4b802e4eb2a91d1ce216b8817763c29fb Expected:894b81a4b802e4eb2a91d1ce216b8817763c29fb} RuncCommit:{ID:425e105d5a03fabd737a126ad93d62a9eeede87f Expected:425e105d5a03fabd737a126ad93d62a9eeede87f} InitCommit:{ID:fec3683 Expected:fec3683} SecurityOptions:[name=apparmor name=seccomp,profile=default] ProductLicense: Warnings:[WARNING: No swap limit support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/libexec/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker Application Vendor:Docker Inc. Version:v0.8.0] map[Experimental:true Name:buildx Path:/usr/libexec/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.3.0-5-g5b97415-tp-docker]] Warnings:<nil>}}
I1210 10:54:07.890992   36599 start_flags.go:364] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.14@sha256:2bd97b482faf5b6a403ac39dd5e7c6fe2006425c6663a12f94f64f5f81a7787e Memory:16000 CPUs:5 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio KubernetesConfig:{KubernetesVersion:v1.19.4 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: ExtraOptions:[{Component:apiserver Key:Authorization.Mode Value:RBAC}] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:192.168.49.2 Port:8443 KubernetesVersion:v1.19.4 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[]}
I1210 10:54:07.892052   36599 out.go:110] 👍  Starting control plane node minikube in cluster minikube
👍  Starting control plane node minikube in cluster minikube
I1210 10:54:08.665001   36599 image.go:92] Found gcr.io/k8s-minikube/kicbase:v0.0.14@sha256:2bd97b482faf5b6a403ac39dd5e7c6fe2006425c6663a12f94f64f5f81a7787e in local docker daemon, skipping pull
I1210 10:54:08.665018   36599 cache.go:116] gcr.io/k8s-minikube/kicbase:v0.0.14@sha256:2bd97b482faf5b6a403ac39dd5e7c6fe2006425c6663a12f94f64f5f81a7787e exists in daemon, skipping pull
I1210 10:54:08.665027   36599 preload.go:97] Checking if preload exists for k8s version v1.19.4 and runtime docker
I1210 10:54:08.665052   36599 preload.go:105] Found local preload: /home/erahbaw/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v6-v1.19.4-docker-overlay2-amd64.tar.lz4
I1210 10:54:08.665058   36599 cache.go:54] Caching tarball of preloaded images
I1210 10:54:08.665066   36599 preload.go:131] Found /home/erahbaw/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v6-v1.19.4-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I1210 10:54:08.665070   36599 cache.go:57] Finished verifying existence of preloaded tar for  v1.19.4 on docker
I1210 10:54:08.665133   36599 profile.go:150] Saving config to /home/erahbaw/.minikube/profiles/minikube/config.json ...
I1210 10:54:08.665256   36599 cache.go:184] Successfully downloaded all kic artifacts
I1210 10:54:08.665276   36599 start.go:314] acquiring machines lock for minikube: {Name:mkd369b7090fdde0461aa3667dfb6af4ccd001fc Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I1210 10:54:08.665348   36599 start.go:318] acquired machines lock for "minikube" in 56.897µs
I1210 10:54:08.665360   36599 start.go:94] Skipping create...Using existing machine configuration
I1210 10:54:08.665364   36599 fix.go:54] fixHost starting: 
I1210 10:54:08.665525   36599 cli_runner.go:110] Run: docker container inspect minikube --format={{.State.Status}}
I1210 10:54:08.697395   36599 fix.go:107] recreateIfNeeded on minikube: state=Stopped err=<nil>
W1210 10:54:08.697420   36599 fix.go:133] unexpected machine state, will restart: <nil>
I1210 10:54:08.699849   36599 out.go:110] 🔄  Restarting existing docker container for "minikube" ...
🔄  Restarting existing docker container for "minikube" ...
I1210 10:54:08.699909   36599 cli_runner.go:110] Run: docker start minikube
I1210 10:54:09.874933   36599 cli_runner.go:154] Completed: docker start minikube: (1.175001154s)
I1210 10:54:09.875019   36599 cli_runner.go:110] Run: docker container inspect minikube --format={{.State.Status}}
I1210 10:54:09.916693   36599 kic.go:356] container "minikube" state is running.
I1210 10:54:09.917372   36599 cli_runner.go:110] Run: docker container inspect -f "{{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}}" minikube
I1210 10:54:09.957760   36599 profile.go:150] Saving config to /home/erahbaw/.minikube/profiles/minikube/config.json ...
I1210 10:54:09.957930   36599 machine.go:88] provisioning docker machine ...
I1210 10:54:09.957946   36599 ubuntu.go:166] provisioning hostname "minikube"
I1210 10:54:09.957994   36599 cli_runner.go:110] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
I1210 10:54:09.997855   36599 main.go:119] libmachine: Using SSH client type: native
I1210 10:54:09.998047   36599 main.go:119] libmachine: &{{{<nil> 0 [] [] []} docker [0x808c20] 0x808be0 <nil>  [] 0s} 127.0.0.1 32779 <nil> <nil>}
I1210 10:54:09.998065   36599 main.go:119] libmachine: About to run SSH command:
sudo hostname minikube && echo "minikube" | sudo tee /etc/hostname





I1210 10:56:19.368881   36599 main.go:119] libmachine: Error dialing TCP: ssh: handshake failed: read tcp 127.0.0.1:60486->127.0.0.1:32779: read: connection reset by peer

Full output of minikube start command used, if not already included:

😄 minikube v1.15.1 on Ubuntu 18.04 ✨ Using the docker driver based on existing profile 👍 Starting control plane node minikube in cluster minikube 🔄 Restarting existing docker container for "minikube" ...

FROZE AFTER THE ABOVE LINE

Optional: Full output of minikube logs command:

minikube logs isn't really working since it froze during the start command but here are logs I fetched from the minikube docker container.

Welcome to Ubuntu 20.04.1 LTS!

Set hostname to . Failed to attach 1 to compat systemd cgroup /docker/b003b01e9897d76981100c4a9839936c8877694ff56b532fe74ee45e9328d0fd/init.scope: No such file or directory /lib/systemd/system/docker.socket:6: ListenStream= references a path below legacy directory /var/run/, updating /var/run/docker.sock → /run/docker.sock; please update the unit file accordingly. [ OK ] Started Dispatch Password …ts to Console Directory Watch. [ OK ] Set up automount Arbitrary…s File System Automount Point. [ OK ] Reached target Local Encrypted Volumes. [ OK ] Reached target Network is Online. [ OK ] Reached target Paths. [ OK ] Reached target Slices. [ OK ] Reached target Swap. [ OK ] Listening on Journal Audit Socket. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Listening on Journal Socket. Mounting Huge Pages File System... Mounting Kernel Debug File System... Mounting Kernel Trace File System... Starting Journal Service... Starting Create list of st…odes for the current kernel... Mounting FUSE Control File System... Starting Remount Root and Kernel File Systems... Starting Apply Kernel Variables... [ OK ] Mounted Huge Pages File System. [ OK ] Mounted Kernel Debug File System. [ OK ] Mounted Kernel Trace File System. [ OK ] Finished Create list of st… nodes for the current kernel. [ OK ] Mounted FUSE Control File System. [ OK ] Finished Remount Root and Kernel File Systems. [ OK ] Finished Apply Kernel Variables. Starting Create System Users... Starting Update UTMP about System Boot/Shutdown... [ OK ] Finished Create System Users. Starting Create Static Device Nodes in /dev... [ OK ] Finished Update UTMP about System Boot/Shutdown. [ OK ] Finished Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). [ OK ] Reached target Local File Systems. [ OK ] Started Journal Service. [ OK ] Reached target System Initialization. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Reached target Timers. Starting Docker Socket for the API. Starting Flush Journal to Persistent Storage... [ OK ] Listening on Docker Socket for the API. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting containerd container runtime... Starting minikube automount... Starting OpenBSD Secure Shell server... [ OK ] Finished Flush Journal to Persistent Storage. [ OK ] Finished minikube automount. [ OK ] Started OpenBSD Secure Shell server. [ OK ] Started containerd container runtime. Starting Docker Application Container Engine... [ OK ] Started Docker Application Container Engine. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Finished Update UTMP about System Runlevel Changes.

Contributor guide