This blog post focuses on why I built the "Kubernetes resources hierarchy parser plugin". This plugin prints the hierarchy resources in one-liners & saves a ton of time for the end-users while writing or editing Kubernetes object configuration files.
Github link: https://github.com/rewanthtammana/kubectl-fields
Problem Statement
For example, you want to add capabilities/securitycontext to your pod configuration. The only way to achieve this is by recursively expanding the kubectl explain and using the grep command. This default method to identify the hierarchy of specific fields for any resource in Kubernetes is cumbersome & tedious.
Introduction to Kubernetes
Kubernetes is an open-source container orchestration system for automating application deployment, scaling, and management. kubectl provides a CLI interface to manage Kubernetes clusters. Kubectl enables the users to run different operations like describe, edit, exec, explain, logs, run, etc on Kubernetes clusters.
Kubernetes objects can be created, updated, and deleted by writing object configuration files either in declarative/imperative method. Kubernetes object configuration files need to follow a pre-defined parental hierarchy structure. All the configuration files need to be addressed in the same pre-defined sequential/parental order to get processed by Kubernetes.
Kubectl CLI
The kubectl CLI has an extended feature called kubectl plugins - this advanced feature allows the users to develop plugins to customize kubectl functionality. I leverage this feature & built this plugin to solve the inception problem.
Default approach
Let' say you want to add capabilities to your pod configuration.
To achieve this, the first thing is to know the hierarchy of capabilities for chosen resources.
The current/default methodology to find the hierarchical order for any field is to use grep or similar commands for the specific field in the terminal.
$ kubectl explain --recursive po.spec | grep capabilities
capabilities <Object>
capabilities <Object>
The above result shows only the matched patterns but it doesn’t show the parental hierarchy. Alternatively, the search can be extended with grep advanced functionalities.
$ kubectl explain --recursive po.spec | grep capabilities -C 5
resources <Object>
limits <map[string]string>
requests <map[string]string>
securityContext <Object>
allowPrivilegeEscalation <boolean>
capabilities <Object>
add <[]string>
drop <[]string>
privileged <boolean>
procMount <string>
readOnlyRootFilesystem <boolean>
--
resources <Object>
limits <map[string]string>
requests <map[string]string>
securityContext <Object>
allowPrivilegeEscalation <boolean>
capabilities <Object>
add <[]string>
drop <[]string>
privileged <boolean>
procMount <string>
readOnlyRootFilesystem <boolean>
Though the above grep command gives us a rough idea of the hierarchy, it doesn’t show the complete sequence and we have to print the entire output & scroll through the terminal to find the right order.
kubectl explain –recursive po.spec (click to expand 624 lines output)
bash
$ kubectl explain --recursive po.spec
KIND: Pod
VERSION: v1
RESOURCE: spec <Object>
DESCRIPTION:
Specification of the desired behavior of the pod. More info:
https://git.k8s.io/community/contributors/devel/api-conventions.md#spec-and-status
PodSpec is a description of a pod.
FIELDS:
activeDeadlineSeconds <integer>
affinity <Object>
nodeAffinity <Object>
preferredDuringSchedulingIgnoredDuringExecution <[]Object>
preference <Object>
matchExpressions <[]Object>
key <string>
operator <string>
values <[]string>
matchFields <[]Object>
key <string>
operator <string>
values <[]string>
weight <integer>
requiredDuringSchedulingIgnoredDuringExecution <Object>
nodeSelectorTerms <[]Object>
matchExpressions <[]Object>
key <string>
operator <string>
values <[]string>
matchFields <[]Object>
key <string>
operator <string>
values <[]string>
podAffinity <Object>
preferredDuringSchedulingIgnoredDuringExecution <[]Object>
podAffinityTerm <Object>
labelSelector <Object>
matchExpressions <[]Object>
key <string>
operator <string>
values <[]string>
matchLabels <map[string]string>
namespaces <[]string>
topologyKey <string>
weight <integer>
requiredDuringSchedulingIgnoredDuringExecution <[]Object>
labelSelector <Object>
matchExpressions <[]Object>
key <string>
operator <string>
values <[]string>
matchLabels <map[string]string>
namespaces <[]string>
topologyKey <string>
podAntiAffinity <Object>
preferredDuringSchedulingIgnoredDuringExecution <[]Object>
podAffinityTerm <Object>
labelSelector <Object>
matchExpressions <[]Object>
key <string>
operator <string>
values <[]string>
matchLabels <map[string]string>
namespaces <[]string>
topologyKey <string>
weight <integer>
requiredDuringSchedulingIgnoredDuringExecution <[]Object>
labelSelector <Object>
matchExpressions <[]Object>
key <string>
operator <string>
values <[]string>
matchLabels <map[string]string>
namespaces <[]string>
topologyKey <string>
automountServiceAccountToken <boolean>
containers <[]Object>
args <[]string>
command <[]string>
env <[]Object>
name <string>
value <string>
valueFrom <Object>
configMapKeyRef <Object>
key <string>
name <string>
optional <boolean>
fieldRef <Object>
apiVersion <string>
fieldPath <string>
resourceFieldRef <Object>
containerName <string>
divisor <string>
resource <string>
secretKeyRef <Object>
key <string>
name <string>
optional <boolean>
envFrom <[]Object>
configMapRef <Object>
name <string>
optional <boolean>
prefix <string>
secretRef <Object>
name <string>
optional <boolean>
image <string>
imagePullPolicy <string>
lifecycle <Object>
postStart <Object>
exec <Object>
command <[]string>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
tcpSocket <Object>
host <string>
port <string>
preStop <Object>
exec <Object>
command <[]string>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
tcpSocket <Object>
host <string>
port <string>
livenessProbe <Object>
exec <Object>
command <[]string>
failureThreshold <integer>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
initialDelaySeconds <integer>
periodSeconds <integer>
successThreshold <integer>
tcpSocket <Object>
host <string>
port <string>
timeoutSeconds <integer>
name <string>
ports <[]Object>
containerPort <integer>
hostIP <string>
hostPort <integer>
name <string>
protocol <string>
readinessProbe <Object>
exec <Object>
command <[]string>
failureThreshold <integer>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
initialDelaySeconds <integer>
periodSeconds <integer>
successThreshold <integer>
tcpSocket <Object>
host <string>
port <string>
timeoutSeconds <integer>
resources <Object>
limits <map[string]string>
requests <map[string]string>
securityContext <Object>
allowPrivilegeEscalation <boolean>
capabilities <Object>
add <[]string>
drop <[]string>
privileged <boolean>
procMount <string>
readOnlyRootFilesystem <boolean>
runAsGroup <integer>
runAsNonRoot <boolean>
runAsUser <integer>
seLinuxOptions <Object>
level <string>
role <string>
type <string>
user <string>
windowsOptions <Object>
gmsaCredentialSpec <string>
gmsaCredentialSpecName <string>
stdin <boolean>
stdinOnce <boolean>
terminationMessagePath <string>
terminationMessagePolicy <string>
tty <boolean>
volumeDevices <[]Object>
devicePath <string>
name <string>
volumeMounts <[]Object>
mountPath <string>
mountPropagation <string>
name <string>
readOnly <boolean>
subPath <string>
subPathExpr <string>
workingDir <string>
dnsConfig <Object>
nameservers <[]string>
options <[]Object>
name <string>
value <string>
searches <[]string>
dnsPolicy <string>
enableServiceLinks <boolean>
hostAliases <[]Object>
hostnames <[]string>
ip <string>
hostIPC <boolean>
hostNetwork <boolean>
hostPID <boolean>
hostname <string>
imagePullSecrets <[]Object>
name <string>
initContainers <[]Object>
args <[]string>
command <[]string>
env <[]Object>
name <string>
value <string>
valueFrom <Object>
configMapKeyRef <Object>
key <string>
name <string>
optional <boolean>
fieldRef <Object>
apiVersion <string>
fieldPath <string>
resourceFieldRef <Object>
containerName <string>
divisor <string>
resource <string>
secretKeyRef <Object>
key <string>
name <string>
optional <boolean>
envFrom <[]Object>
configMapRef <Object>
name <string>
optional <boolean>
prefix <string>
secretRef <Object>
name <string>
optional <boolean>
image <string>
imagePullPolicy <string>
lifecycle <Object>
postStart <Object>
exec <Object>
command <[]string>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
tcpSocket <Object>
host <string>
port <string>
preStop <Object>
exec <Object>
command <[]string>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
tcpSocket <Object>
host <string>
port <string>
livenessProbe <Object>
exec <Object>
command <[]string>
failureThreshold <integer>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
initialDelaySeconds <integer>
periodSeconds <integer>
successThreshold <integer>
tcpSocket <Object>
host <string>
port <string>
timeoutSeconds <integer>
name <string>
ports <[]Object>
containerPort <integer>
hostIP <string>
hostPort <integer>
name <string>
protocol <string>
readinessProbe <Object>
exec <Object>
command <[]string>
failureThreshold <integer>
httpGet <Object>
host <string>
httpHeaders <[]Object>
name <string>
value <string>
path <string>
port <string>
scheme <string>
initialDelaySeconds <integer>
periodSeconds <integer>
successThreshold <integer>
tcpSocket <Object>
host <string>
port <string>
timeoutSeconds <integer>
resources <Object>
limits <map[string]string>
requests <map[string]string>
securityContext <Object>
allowPrivilegeEscalation <boolean>
capabilities <Object>
add <[]string>
drop <[]string>
privileged <boolean>
procMount <string>
readOnlyRootFilesystem <boolean>
runAsGroup <integer>
runAsNonRoot <boolean>
runAsUser <integer>
seLinuxOptions <Object>
level <string>
role <string>
type <string>
user <string>
windowsOptions <Object>
gmsaCredentialSpec <string>
gmsaCredentialSpecName <string>
stdin <boolean>
stdinOnce <boolean>
terminationMessagePath <string>
terminationMessagePolicy <string>
tty <boolean>
volumeDevices <[]Object>
devicePath <string>
name <string>
volumeMounts <[]Object>
mountPath <string>
mountPropagation <string>
name <string>
readOnly <boolean>
subPath <string>
subPathExpr <string>
workingDir <string>
nodeName <string>
nodeSelector <map[string]string>
preemptionPolicy <string>
priority <integer>
priorityClassName <string>
readinessGates <[]Object>
conditionType <string>
restartPolicy <string>
runtimeClassName <string>
schedulerName <string>
securityContext <Object>
fsGroup <integer>
runAsGroup <integer>
runAsNonRoot <boolean>
runAsUser <integer>
seLinuxOptions <Object>
level <string>
role <string>
type <string>
user <string>
supplementalGroups <[]integer>
sysctls <[]Object>
name <string>
value <string>
windowsOptions <Object>
gmsaCredentialSpec <string>
gmsaCredentialSpecName <string>
serviceAccount <string>
serviceAccountName <string>
shareProcessNamespace <boolean>
subdomain <string>
terminationGracePeriodSeconds <integer>
tolerations <[]Object>
effect <string>
key <string>
operator <string>
tolerationSeconds <integer>
value <string>
volumes <[]Object>
awsElasticBlockStore <Object>
fsType <string>
partition <integer>
readOnly <boolean>
volumeID <string>
azureDisk <Object>
cachingMode <string>
diskName <string>
diskURI <string>
fsType <string>
kind <string>
readOnly <boolean>
azureFile <Object>
readOnly <boolean>
secretName <string>
shareName <string>
cephfs <Object>
monitors <[]string>
path <string>
readOnly <boolean>
secretFile <string>
secretRef <Object>
name <string>
user <string>
cinder <Object>
fsType <string>
readOnly <boolean>
secretRef <Object>
name <string>
volumeID <string>
configMap <Object>
defaultMode <integer>
items <[]Object>
key <string>
mode <integer>
path <string>
name <string>
optional <boolean>
csi <Object>
driver <string>
fsType <string>
nodePublishSecretRef <Object>
name <string>
readOnly <boolean>
volumeAttributes <map[string]string>
downwardAPI <Object>
defaultMode <integer>
items <[]Object>
fieldRef <Object>
apiVersion <string>
fieldPath <string>
mode <integer>
path <string>
resourceFieldRef <Object>
containerName <string>
divisor <string>
resource <string>
emptyDir <Object>
medium <string>
sizeLimit <string>
fc <Object>
fsType <string>
lun <integer>
readOnly <boolean>
targetWWNs <[]string>
wwids <[]string>
flexVolume <Object>
driver <string>
fsType <string>
options <map[string]string>
readOnly <boolean>
secretRef <Object>
name <string>
flocker <Object>
datasetName <string>
datasetUUID <string>
gcePersistentDisk <Object>
fsType <string>
partition <integer>
pdName <string>
readOnly <boolean>
gitRepo <Object>
directory <string>
repository <string>
revision <string>
glusterfs <Object>
endpoints <string>
path <string>
readOnly <boolean>
hostPath <Object>
path <string>
type <string>
iscsi <Object>
chapAuthDiscovery <boolean>
chapAuthSession <boolean>
fsType <string>
initiatorName <string>
iqn <string>
iscsiInterface <string>
lun <integer>
portals <[]string>
readOnly <boolean>
secretRef <Object>
name <string>
targetPortal <string>
name <string>
nfs <Object>
path <string>
readOnly <boolean>
server <string>
persistentVolumeClaim <Object>
claimName <string>
readOnly <boolean>
photonPersistentDisk <Object>
fsType <string>
pdID <string>
portworxVolume <Object>
fsType <string>
readOnly <boolean>
volumeID <string>
projected <Object>
defaultMode <integer>
sources <[]Object>
configMap <Object>
items <[]Object>
key <string>
mode <integer>
path <string>
name <string>
optional <boolean>
downwardAPI <Object>
items <[]Object>
fieldRef <Object>
apiVersion <string>
fieldPath <string>
mode <integer>
path <string>
resourceFieldRef <Object>
containerName <string>
divisor <string>
resource <string>
secret <Object>
items <[]Object>
key <string>
mode <integer>
path <string>
name <string>
optional <boolean>
serviceAccountToken <Object>
audience <string>
expirationSeconds <integer>
path <string>
quobyte <Object>
group <string>
readOnly <boolean>
registry <string>
tenant <string>
user <string>
volume <string>
rbd <Object>
fsType <string>
image <string>
keyring <string>
monitors <[]string>
pool <string>
readOnly <boolean>
secretRef <Object>
name <string>
user <string>
scaleIO <Object>
fsType <string>
gateway <string>
protectionDomain <string>
readOnly <boolean>
secretRef <Object>
name <string>
sslEnabled <boolean>
storageMode <string>
storagePool <string>
system <string>
volumeName <string>
secret <Object>
defaultMode <integer>
items <[]Object>
key <string>
mode <integer>
path <string>
optional <boolean>
secretName <string>
storageos <Object>
fsType <string>
readOnly <boolean>
secretRef <Object>
name <string>
volumeName <string>
volumeNamespace <string>
vsphereVolume <Object>
fsType <string>
storagePolicyID <string>
storagePolicyName <string>
This is a tedious job and consumes a lot of time. If there are multiple matching fields in different objects, that only makes the situation worse.
Proposed solution: kubectl-fields
I leveraged the kubectl plugins feature & built a plugin to demonstrate an alternative solution & approach to this problem.
kubectl explain --recursive | grep doesn’t show the exact hierarchy of matched fields, but this plugin does! kubectl fields solves this problem by printing a one-liner parental hierarchy of any field in any selected resource.
$ kubectl fields po capabilities
spec.containers.securityContext.capabilities
spec.ephemeralContainers.securityContext.capabilities
spec.initContainers.securityContext.capabilities
$ kubectl fields po.spec securitycontext
containers.securityContext
ephemeralContainers.securityContext
initContainers.securityContext
securityContext
Bonus
kubectl fields is now integrated with krew, a kubectl plugin manager. This plugin integration works on all platforms. So, this plugin can be installed directly with krew. It’s as simple as,
kubectl krew install fields
Huge thanks to ahmetb for the krew integration suggestion.
References
https://github.com/rewanthtammana/kubectl-fields
