Mastering the KUBECONFIG file

Mastering the KUBECONFIG fileAhmet Alp BalkanBlockedUnblockFollowFollowingFeb 12There is a kubeconfig file behind every working kubectl command.

This file typically lives at $HOME/.

kube/config.

Having written kubectx, I’ve interacted with kubeconfigs long enough to write some tips about how to deal with them.

If you’re not familiar with kubeconfig files, read the documentation first.

Tip 1: Know the kubeconfig precedenceIf you’re using kubectl, here’s the preference that takes effect while determining which kubeconfig file is used.

use –kubeconfig flag, if specifieduse KUBECONFIG environment variable, if specifieduse $HOME/.

kube/config fileWith this, you can easily override kubeconfig file you use per-kubectl command:kubectl get pods –kubeconfig=file1kubectl get pods –kubeconfig=file2# OR #KUBECONFIG=file1 kubectl get podsKUBECONFIG=file2 kubectl get podsAlthough this precedence list not officially specified in the documentation it is codified here.

If you’re developing client tools for Kubernetes, you should consider using cli-runtime library which will bring the standard –kubeconfig flag and $KUBECONFIG detection to your program.

Tip 2: Using multiple kubeconfigs at onceSometimes you have a bunch of small kubeconfig files (e.

g.

one per cluster) but you want to use them all at once, with tools like kubectl or kubectx that work with multiple contexts at once.

To do that, you need a “merged” kubeconfig file.

Tip #3 explains how you can merge the kubeconfigs into a single file, but you can also merge them in-memory.

By specifying multiple files in KUBECONFIG environment variable, you can temporarily stitch kubeconfig files together and use them all in kubectl.

export KUBECONFIG=file1:file2kubectl get pods –context=cluster-1kubectl get pods –context=cluster-2Tip 3: Merging kubeconfig filesSince kubeconfig files are structured YAML files, you can’t just append them to get one big kubeconfig file, but kubectl can help you merge these files:KUBECONFIG=file1:file2:file3 kubectl config view –merge –flatten > out.

txtTip 4: Extracting a context from a kubeconfig fileLet’s say you followed Tip #4 and have a merged kubeconfig file.

Now you want to extract a cluster’s information to a portable kubeconfig file that only has the parts you need to connect to that cluster.

Run:KUBECONFIG=in.

txt kubectl config view –minify –flatten –context=context-1 > out.

txtIn this command, we extract data about context-1 from in.

txt to out.

txt.

The –minify flag allows us to extract only info about that context, and the –flatten flag allows us to keep the credentials unredacted.

Tip 5: Use kubectl without a kubeconfigkubectl command offers a bunch of command line flags (run kubectl options to see) that allow you to override pretty much every piece of information it reads from a kubeconfig file.

For example, if I wanted to use my local Docker for Mac cluster without a kubeconfig file, I would first look at kubectl config view –context=docker-for-desktop and extract the information to the following flags:KUBECONFIG= kubectl get nodes –server https://localhost:6443 –user docker-for-desktop –client-certificate my.

cert –client-key my.

key –insecure-skip-tls-verifyIt gets tricky (and impossible) to use as your kubeconfig gets complicated, like when you have an auth plugin with various fields you can’t configure via a CLI.

Don’t forget to set your $KUBECONFIG to empty (as seen above) to prevent accidentally picking up some settings from the ~/.

kube/config file.

(Bonus) Tip 6: Auto-$KUBECONFIG based on directoryMany people complain accidentally executing commands on the wrong cluster.

In most cases, this happens because you’re in the directory containing manifests for cluster-1, but you apply it to cluster-2 as that was the active context.

To prevent this scenario, you can use direnv tool which lets you automatically set environment variables based on the directory tree you’re in.

This way, when navigate to the directory of cluster-1 manifests, direnv will set $KUBECONFIG to cluster-1 and prevent the disaster.

(Bonus) Tip 7: Know which context you’re pointing atThis is not directly about munging KUBECONFIG files, but kube-ps1 (which I proudly advised on its early development) that lets you see the current namespace/context you’re on in your bash/zsh prompt:$ kubeon{⎈ |N/A:N/A} $ export KUBECONFIG=f2{⎈ |docker-for-desktop:default} $ export KUBECONFIG=f1{⎈ |gke_ahmetb_us-central1-b_mycluster:kube-system} $ kubens defaultActive namespace is "default".

{⎈ |gke_ahmetb_us-central1-b_mycluster:kube-system} $ kubeoff$You can turn it on/off per-shell, or globally with -g flag to kubeon/kubeoff.

(Bonus) Tip 8: Save GKE contexts to separate filesWhen you create a GKE cluster (or retrieve its credentials) through the gcloud command, it normally modifies your default ~/.

kube/config file.

However, you can set $KUBECONFIG for gcloud to save cluster credentials to a file:KUBECONFIG=c1.

yaml gcloud container clusters get-credentials "cluster-1"This article was originally published on my personal blog at ahmet.

im.

Do you know more tips like these?.If so, hit me up on twitter: Ahmet Balat.

. More details

Leave a Reply