`Handshake status 401 Unauthorized` during connect_get_namespaced_pod_exec when using service account

Comienza con kubernetes/stream/ws_client.py, especialmente con create_websocket y websocket_call, y luego sigue la llamada stream(...) hasta connect_get_namespaced_pod_exec en el script proporcionado. Reproduce el caso de service-account y compáralo con la llamada REST exitosa y la impersonación de kubectl; se considera terminado cuando exec se autentica correctamente mediante el WebSocket.

What happened (please include outputs or screenshots):

I would like to use kubernetes-client to exec into a pod container. This script is to run from within another pod, so I need to work with a service account. For testing I run the script from my workstation command line, using my standard kubernetes (admin) user. So the idea is to switch to the service account from within the script (unconditionally for test purposes). This seems to work. But when I actually exec into a container, I receive an exception:

Traceback (most recent call last):
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/stream/ws_client.py", line 528, in websocket_call
    client = WSClient(configuration, url, headers, capture_all, binary=binary)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/stream/ws_client.py", line 68, in __init__
    self.sock = create_websocket(configuration, url, headers)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/stream/ws_client.py", line 494, in create_websocket
    websocket.connect(url, **connect_opt)
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/websocket/_core.py", line 261, in connect
    self.handshake_response = handshake(self.sock, url, *addrs, **options)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/websocket/_handshake.py", line 65, in handshake
    status, resp = _get_resp_headers(sock)
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/websocket/_handshake.py", line 150, in _get_resp_headers
    raise WebSocketBadStatusException(
websocket._exceptions.WebSocketBadStatusException: Handshake status 401 Unauthorized -+-+- {'audit-id': 'cd85f343-bdb9-4e94-808c-26dd49a34d1f', 'cache-control': 'no-cache, private', 'content-type': 'application/json', 'date': 'Wed, 09 Oct 2024 13:38:29 GMT', 'content-length': '129'} -+-+- b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}\n'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspaces/misc/python-client-test/python-client-test.py", line 87, in <module>
    main()
  File "/workspaces/misc/python-client-test/python-client-test.py", line 75, in main
    resp = stream(core_api.connect_get_namespaced_pod_exec,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/stream/stream.py", line 36, in _websocket_request
    out = api_method(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/client/api/core_v1_api.py", line 994, in connect_get_namespaced_pod_exec
    return self.connect_get_namespaced_pod_exec_with_http_info(name, namespace, **kwargs)  # noqa: E501
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/client/api/core_v1_api.py", line 1101, in connect_get_namespaced_pod_exec_with_http_info
    return self.api_client.call_api(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/client/api_client.py", line 348, in call_api
    return self.__call_api(resource_path, method,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/client/api_client.py", line 180, in __call_api
    response_data = self.request(
                    ^^^^^^^^^^^^^
  File "/workspaces/misc/python-client-test/.venv/lib/python3.12/site-packages/kubernetes/stream/ws_client.py", line 538, in websocket_call
    raise ApiException(status=0, reason=str(e))
kubernetes.client.exceptions.ApiException: (0)
Reason: Handshake status 401 Unauthorized -+-+- {'audit-id': 'cd85f343-bdb9-4e94-808c-26dd49a34d1f', 'cache-control': 'no-cache, private', 'content-type': 'application/json', 'date': 'Wed, 09 Oct 2024 13:38:29 GMT', 'content-length': '129'} -+-+- b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Unauthorized","reason":"Unauthorized","code":401}\ 

I've executed some more tests and figured out:

• I can exec into the container with the standard user, without changing to the service account first
• Using kubectl with impersonation of the service account also works -- so RBAC seems to be correct
• Switching to the service user and running other API calls than exec also works fine

I conclude: there seems to be an issue during protocol handover from REST to Web Socket (this is also suggested by the call stack).

Did I do something wrong? Is this a bug? Is there a workaround?

What you expected to happen:

I expected to be able to successfully exec into a container.

How to reproduce it (as minimally and precisely as possible):

This is my kubernetes test environment (use kubectl apply -f):

apiVersion: v1
kind: Namespace
metadata:
  name: test-ns
---
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
  namespace: test-ns
spec:
  containers:
    - name: test-container
      image: busybox
      command: ["sleep"]
      args: ["infinity"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: test-sa
  namespace: test-ns
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: test-role
  namespace: test-ns
rules:
  - apiGroups: [""] # core API group
    resources: ["pods"]
    verbs: ["get", "list"]
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["get", "create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: test-rb
  namespace: test-ns
subjects:
  - kind: ServiceAccount
    name: test-sa
    namespace: test-ns
roleRef:
  kind: Role
  name: test-role
  apiGroup: rbac.authorization.k8s.io

and this is the script, I'm trying to execute:

from kubernetes import client, config
from kubernetes.stream import stream

# Usage example
NAMESPACE = "test-ns"
POD_NAME = "test-pod"
SERVICE_ACCOUNT = "test-sa"
COMMAND = ["ls"]

def main():
    
    # read $KUBECONFIG and instantiate core_api
    config.load_kube_config()
    api_client = client.ApiClient()
    core_api = client.CoreV1Api(api_client)

    #######################################
    # Test case 1: works
    #######################################
    resp = stream(core_api.connect_get_namespaced_pod_exec,
                  POD_NAME,
                  NAMESPACE,
                  command=COMMAND,
                  stderr=True,
                  stdin=False,
                  stdout=True,
                  tty=False)
    print(resp)

    # switch to service account
    tokenRequest = client.AuthenticationV1TokenRequest(
        spec=client.V1TokenRequestSpec(audiences=[""])
    )
    token = core_api.create_namespaced_service_account_token(
        SERVICE_ACCOUNT, NAMESPACE, tokenRequest)
    api_client.configuration.api_key['authorization'] = token.status.token
    api_client.configuration.api_key_prefix['authorization'] = 'Bearer'
    api_client.configuration.key_file = None
    api_client.configuration.cert_file = None

    # re-instantiate core_api
    core_api = client.CoreV1Api(api_client)

    #######################################
    # Test case 2: works
    #######################################
    pods = core_api.list_namespaced_pod(NAMESPACE)
    print(pods)

    #######################################
    # Test case 3: triggers an exception, even though the corresponding kubectl call works:
    # kubectl exec -it -n test-ns test-pod --as system:serviceaccount:test-ns:test-sa -- ls
    #######################################
    resp = stream(core_api.connect_get_namespaced_pod_exec,
                  POD_NAME,
                  NAMESPACE,
                  command=COMMAND,
                  stderr=True,
                  stdin=False,
                  stdout=True,
                  tty=False)

    print(resp)

if __name__ == '__main__':
    main()

Environment:

• Kubernetes version (kubectl version):

Client Version: v1.29.1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.6

• OS: Microsoft vscode dev container image base:1.0.10-bullseye on Docker Desktop 4.21.1 on WSL2 on Windows 10 Version 22H2, Build 19045.4780
• Python version: 3.12.6
• Python client version: 31.0.0