Authentication using python Kubernetes api in GCP is not working

I would like to be able to access GKE (kubernetes) cluster in GCP from python kubernetes client. I cant authenticate and connect to my cluster and i dont find the reason. Here is what i tried so far.

from google.auth import compute_engine
from google.cloud.container_v1 import ClusterManagerClient
from kubernetes import client


def test_gke(request):
    project_id = "myproject"
    zone = "myzone"
    cluster_id = "mycluster"

    credentials = compute_engine.Credentials()

    cluster_manager_client = ClusterManagerClient(credentials=credentials)
    cluster = cluster_manager_client.get_cluster(name=f'projects/{project_id}/locations/{zone}/clusters/{cluster_id}')

    configuration = client.Configuration()
    configuration.host = f"https://{cluster.endpoint}:443"
    configuration.verify_ssl = False
    configuration.api_key = {"authorization": "Bearer " + credentials.token}
    client.Configuration.set_default(configuration)

    v1 = client.CoreV1Api()
    print("Listing pods with their IPs:")
    pods = v1.list_pod_for_all_namespaces(watch=False)
    for i in pods.items:
        print("%st%st%s" % (i.status.pod_ip, i.metadata.namespace, i.metadata.name))

Answer

I’d like to get the configuration working I have it work where, the code is running off cluster and it produces the kubectl config file for itself. (see update at end)

Original

The first solution assumes (!) you’ve the cluster configured in your local (~/.kube/config and probably adjusted by KUBE_CONFIG) config.

from google.cloud.container_v1 import ClusterManagerClient
from kubernetes import client,config

config.load_kube_config()
api_instance = client.CoreV1Api()

resp = api_instance.list_pod_for_all_namespaces()
for i in resp.items:
    print(f"{i.status.pod_ip}t{i.metadata.namespace}t{i.metadata.name}")

NOTE

1. Assumes you’ve run gcloud containers clusters get-credentials to set the ~/.kube/config file for the current cluster (and has a current-context set.
2. Uses your user credentials in the ~/.kube/config file so no additional credentials are needed.

Update

Okay, I have it working. Here’s the code that will generate a kubectl config and connect to the cluster. This code uses Application Default Credentials to provide a Service Account key to the code (usually export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json)

import os
import google.auth
import base64

from google.cloud.container_v1 import ClusterManagerClient
from kubernetes import client,config
from ruamel import yaml

PROJECT = os.getenv("PROJECT")
ZONE = os.getenv("ZONE")
CLUSTER = os.getenv("CLUSTER")

# Get Application Default Credentials
# `project_id` is the Service Account's
# This may differ to the cluster's `PROJECT`
credentials, project_id = google.auth.default()

# Get the cluster config from GCP
cluster_manager_client = ClusterManagerClient(credentials=credentials)

name=f"projects/{PROJECT}/locations/{ZONE}/clusters/{CLUSTER}"
cluster = cluster_manager_client.get_cluster(name=name)

SERVER = cluster.endpoint
CERT = cluster.master_auth.cluster_ca_certificate

configuration = client.Configuration()

# Create's a `kubectl` config
NAME="freddie" # arbitrary
CONFIG=f"""
apiVersion: v1
kind: Config
clusters:
- name: {NAME}
  cluster:
    certificate-authority-data: {CERT}
    server: https://{SERVER}
contexts:
- name: {NAME}
  context:
    cluster: {NAME}
    user: {NAME}
current-context: {NAME}
users:
- name: {NAME}
  user:
    auth-provider:
      name: gcp
      config:
        scopes: https://www.googleapis.com/auth/cloud-platform
"""

# The Python SDK doesn't directly support providing a dict
# See: https://github.com/kubernetes-client/python/issues/870
kubeconfig = yaml.safe_load(CONFIG)

loader = config.kube_config.KubeConfigLoader(kubeconfig)
loader.load_and_set(configuration)

api_client= client.ApiClient(configuration)
api_instance = client.CoreV1Api(api_client)

# Enumerate e.g. Pods
resp = api_instance.list_pod_for_all_namespaces()
for i in resp.items:
     print(f"{i.status.pod_ip}t{i.metadata.namespace}t{i.metadata.name}")