Want to level up your Kubernetes knowledge? Explore our tutorials and step-by-step guides written by Kubernetes experts. Learn Kubernetes best practices and solutions to the most difficult problems.

Kubernetes Guides and Tutorials | Spacelift Blog

Author title

OpenTofu is now part of the Linux Foundation 🎉

Title category

Get all the latest content on Infrastructure as Code, DevOps, Spacelift features & news directly to your inbox!

Level up your IaC knowledge. Stay up to date with educational devops guides & news in infrastructure as code, cloud & more!

Managing Infrastructure at Scale | Spacelift Blog

applepodcasts

spotify

missionctrl

Welcome to missionCTRL, a show dedicated to exploring the frontiers of modern DevOps. Listen (or watch) as we sit down with the practitioners who are pushing the boundaries of development

Kubernetes

Jack Roper

In this post, we will list and describe each commonly used category or component of Kubernetes (K8S) with appropriate <code class="gr kz la lb lc b">kubectl</code> commands for quick reference!
<code>kubectl</code> is a Kubernetes command-line tool that allows you to run commands against Kubernetes clusters. You can use <code>kubectl</code> to deploy applications, inspect and manage cluster resources, and view logs.
The descriptions are taken directly from the official documentation at <a class="au ld" href="https://kubernetes.io/" target="_blank" rel="nofollow noopener">kubernetes.io</a>, and each will link to the appropriate documentation page should you require more information or a deeper dive.

<ul class="">
<li id="ed33" class="mo mp io kf b kg mq kj mr km ms kq mt ku mu ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#common-options">Common Options</a></li>
<li id="56e2" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#configuration-files-also-referred-to-as-manifest-or-yaml-files">Configuration Files (Manifest files)</a></li>
<li id="c317" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#cluster-management-and-context">Cluster Management &amp; Context</a></li>
<li id="247e" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#daemonsets">Daemonsets</a></li>
<li id="1af2" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#deployments">Deployments</a></li>
<li id="9752" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#events">Events</a></li>
<li id="aff3" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#logs">Logs</a></li>
<li id="8eb3" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#namespaces">Namespaces</a></li>
<li id="9066" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#nodes">Nodes</a></li>
<li id="596c" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#pods">Pods</a></li>
<li id="31ae" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#replication-controllers">Replication Controllers</a></li>
<li id="29bd" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#replicasets">ReplicaSets</a></li>
<li id="1633" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#secrets">Secrets</a></li>
<li id="2066" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#services">Services</a></li>
<li id="52d4" class="mo mp io kf b kg mz kj na km nb kq nc ku nd ky mv mw mx my gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-cheat-sheet#service-accounts">Service Accounts</a></li>
</ul>
Anything that can be run via kubectl can be run within a Spacelift stack. Find out more about <a href="https://docs.spacelift.io/vendors/kubernetes/" target="_blank" rel="noopener">how Spacelift works with Kubernetes</a>.

In kubectl you can specify optional flags for use with various commands.
<code class="gr kz la lb lc b">alias</code> &#8211; Set an alias for kubectl.

<code class="gr kz la lb lc b">-o=json</code> &#8211; Output format in JSON.

<code class="gr kz la lb lc b">-o=yaml</code> &#8211; Output format in YAML.

<code class="gr kz la lb lc b">-o=wide</code> &#8211; Output in the plain-text format with any additional information, and for pods, the node name is included.

<code class="gr kz la lb lc b">-n</code> &#8211; Alias for <code class="gr kz la lb lc b">namespace</code>.

<code class="gr kz la lb lc b">-f</code> &#8211; Filename, directory, or URL to files to use to create a resource.

<code class="gr kz la lb lc b">-l</code> &#8211; Filter using a specified label.

<code class="gr kz la lb lc b">-h</code> &#8211; Help for kubectl.

<a href="https://kubernetes.io/docs/tasks/manage-kubernetes-objects/declarative-config/" target="_blank" rel="nofollow noopener">Configuration Files</a> (Also referred to as Manifest or YAML Files.) &#8211; Kubernetes objects can be created, updated, and deleted by storing multiple object configuration files in a directory and using <code class="gr kz la lb lc b">kubectl apply</code> to recursively create and update those objects as needed. This method retains writes made to live objects without merging the changes back into the object configuration files. <code class="gr kz la lb lc b">kubectl diff</code> also gives you a preview of what changes <code class="gr kz la lb lc b">apply</code> will make.

<code>kubectl create -f &lt;configuration file&gt;</code>  &#8211; Create objects.
<code>kubectl create -f &lt;configuration file directory&gt;</code> &#8211; Create objects in all manifest files in a directory.
<code>kubectl create -f &lt;‘url’&gt;</code> &#8211; Create objects from a URL.
<code>kubectl delete -f &lt;configuration file&gt;</code> &#8211; Delete an object.
Read more about <a href="https://spacelift.io/blog/kubectl-apply-vs-create" target="_blank" rel="noopener">Kubectl Apply vs. Kubectl Create – What’s the Difference?</a>.

Cluster management refers to querying information about the K8S cluster itself.
<code>kubectl cluster-info</code> &#8211; Display endpoint information about the master and services in the cluster.
<code>kubectl version</code> &#8211; Display the Kubernetes version running on the client and server.
<code>kubectl config view</code> &#8211; Get the configuration of the cluster.
<code>kubectl config view -o jsonpath='{.users[*].name}'</code> &#8211; Get a list of users.
<code>kubectl config current-context</code> &#8211; Display the current context.
<code>kubectl config get-contexts</code> &#8211; Display a list of contexts.
<code>kubectl config use-context &lt;cluster name&gt;</code> &#8211; Set the default context.
<code>kubectl api-resources</code> &#8211; List the API resources that are available.
<code>kubectl api-versions</code> &#8211; List the API versions that are available.
<code class="gr kz la lb lc b">-A</code> &#8211; List pods, services, daemonsets, deployments, replicasets, statefulsets, jobs, and <a href="https://spacelift.io/blog/kubernetes-cronjob" target="_blank" rel="noopener">CronJobs</a> in all namespaces, not custom resource types. Note the alias for <code class="gr kz la lb lc b">--all-namespaces</code> is <code class="gr kz la lb lc b">-A</code>

<ul>
<li><a href="https://spacelift.io/blog/provision-kubernetes-cluster-terraform-spacelift" target="_blank" rel="noopener">How to Provision Kubernetes Cluster with Spacelift and Terraform</a></li>
<li><a href="https://spacelift.io/blog/kubernetes-best-practices" target="_blank" rel="noopener">15 Kubernetes Best Practices to Follow</a></li>
<li><a href="https://spacelift.io/blog/terraform-vs-kubernetes" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Terraform vs. Kubernetes : Key Differences and Comparison&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Terraform vs. Kubernetes Comparison</a></li>
</ul>

<a href="https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/" target="_blank" rel="nofollow noopener">Daemonsets</a> &#8211; A DaemonSet ensures that all (or some) Nodes run a copy of a Pod. As nodes are added to the cluster, Pods are added to them. As nodes are removed from the cluster, those Pods are garbage collected. Deleting a DaemonSet will clean up the Pods it created.
<code>kubectl get daemonset</code> &#8211; List one or more daemonsets.
<code>kubectl edit daemonset &lt;daemonset_name&gt;</code> &#8211; Edit and update the definition of one or more daemonset.
<code>kubectl delete daemonset &lt;daemonset_name&gt;</code> &#8211; Delete a daemonset.
<code>kubectl create daemonset &lt;daemonset_name&gt;</code> &#8211; Create a new daemonset.
<code>kubectl rollout daemonset</code> &#8211; Manage the rollout of a daemonset.
<code>kubectl describe ds &lt;daemonset_name&gt; -n &lt;namespace_name&gt;</code> &#8211; Display the detailed state of daemonsets within a namespace.

<a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/" target="_blank" rel="nofollow noopener">Deployments</a> &#8211; A Deployment provides declarative updates for <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/pods/" target="_blank" rel="noopener ugc nofollow">Pods</a> and <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/" target="_blank" rel="noopener ugc nofollow">ReplicaSets</a>. You describe a desired state in a Deployment, and the Deployment <a class="au ld" href="https://kubernetes.io/docs/concepts/architecture/controller/" target="_blank" rel="noopener ugc nofollow">Controller</a> changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments. See <a href="https://spacelift.io/blog/statefulset-vs-deployment" target="_blank" rel="noopener">StatefulSet vs. Deployment</a>.
<code>kubectl get deployment</code> &#8211; List one or more deployments.
<code>kubectl describe deployment &lt;deployment_name&gt;</code> &#8211; Display the detailed state of one or more deployments.
<code>kubectl edit deployment &lt;deployment_name&gt;</code> &#8211; Edit and update the definition of one or more deployments on the server.
<code>kubectl create deployment &lt;deployment_name&gt;</code> &#8211; Create a new deployment.
<code>kubectl delete deployment &lt;deployment_name&gt;</code> &#8211; Delete deployments.
<code>kubectl rollout status deployment &lt;deployment_name&gt;</code> &#8211; See the rollout status of a deployment.
<code>kubectl set image deployment/&lt;deployment name&gt; &lt;container name&gt;=image:&lt;new image version&gt;</code> &#8211; Perform a rolling update (K8S default), set the image of the container to a new version for a particular deployment.
<code>kubectl rollout undo deployment/&lt;deployment name&gt;</code> &#8211; Rollback a previous deployment.
<code>kubectl replace --force -f &lt;configuration file&gt;</code> &#8211; Perform a replace deployment — Force replace, delete and then re-create the resource.
Read more about Kubernetes deployment strategies: <a href="https://spacelift.io/blog/kubernetes-deployment-strategies" target="_blank" rel="noopener">Different Types of Kubernetes Deployment Strategies (Examples)</a>

<code>kubectl get events</code> &#8211; List recent events for all resources in the system.
<code>kubectl get events --field-selector type=Warning</code> &#8211; List Warnings only.
<code>kubectl get events --sort-by=.metadata.creationTimestamp</code> &#8211; List events sorted by timestamp.
<code>kubectl get events --field-selector involvedObject.kind!=Pod</code> &#8211; List events but exclude Pod events.
<code>kubectl get events --field-selector involvedObject.kind=Node, involvedObject.name=&lt;node_name&gt;</code> &#8211; Pull events for a single node with a specific name.
<code>kubectl get events --field-selector type!=Normal</code> &#8211; Filter out normal events from a list of events.

<a href="https://kubernetes.io/docs/concepts/cluster-administration/system-logs/" target="_blank" rel="nofollow noopener">Logs</a> &#8211; System component logs record events happening in cluster, which can be very useful for debugging. You can configure log verbosity to see more or less detail. Logs can be as coarse-grained as showing errors within a component, or as fine-grained as showing step-by-step traces of events (like HTTP access logs, pod state changes, controller actions, or scheduler decisions).
<code>kubectl logs &lt;pod_name&gt;</code> &#8211; Print the logs for a pod.
<code>kubectl logs --since=6h &lt;pod_name&gt;</code> &#8211; Print the logs for the last 6 hours for a pod.
<code>kubectl logs --tail=50 &lt;pod_name&gt;</code> &#8211; Get the most recent 50 lines of logs.
<code>kubectl logs -f &lt;service_name&gt; [-c &lt;$container&gt;]</code> &#8211; Get logs from a service and optionally select which container.
<code>kubectl logs -f &lt;pod_name&gt;</code> &#8211; Print the logs for a pod and follow new logs.
<code>kubectl logs -c &lt;container_name&gt; &lt;pod_name&gt;</code> &#8211; Print the logs for a container in a pod.
<code>kubectl logs &lt;pod_name&gt; pod.log</code> &#8211; Output the logs for a pod into a file named ‘pod.log’.
<code>kubectl logs --previous &lt;pod_name&gt;</code> &#8211; View the logs for a previously failed pod.

<a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/" target="_blank" rel="nofollow noopener">Namespaces</a> &#8211; In Kubernetes, namespaces provides a mechanism for isolating groups of resources within a single cluster. Names of resources need to be unique within a namespace, but not across namespaces. Namespace-based scoping is applicable only for namespaced objects (e.g. Deployments, Services, etc) and not for cluster-wide objects (e.g. StorageClass, Nodes, <a href="https://spacelift.io/blog/kubernetes-persistent-volumes" target="_blank" rel="noopener">PersistentVolumes</a>, etc).
<code>kubectl create namespace &lt;namespace_name&gt;</code> &#8211; Create a namespace.
<code>kubectl get namespace &lt;namespace_name&gt;</code> &#8211; List one or more namespaces.
<code>kubectl describe namespace &lt;namespace_name&gt;</code> &#8211; Display the detailed state of one or more namespaces.
<code>kubectl delete namespace &lt;namespace_name&gt;</code> &#8211; Delete a namespace.
<code>kubectl edit namespace &lt;namespace_name&gt;</code> &#8211; Edit and update the definition of a namespace.
<code>kubectl top namespace &lt;namespace_name&gt;</code> &#8211; Display Resource (CPU/Memory/Storage) usage for a namespace.

<a href="https://kubernetes.io/docs/concepts/architecture/nodes/" target="_blank" rel="nofollow noopener">Nodes</a> &#8211; Kubernetes runs your workload by placing containers into Pods to run on Nodes. A node may be a virtual or physical machine, depending on the cluster. Each node is managed by the <a class="au ld" href="https://kubernetes.io/docs/reference/glossary/?all=true#term-control-plane" target="_blank" rel="noopener ugc nofollow">control plane</a> and contains the services necessary to run <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/pods/" target="_blank" rel="noopener ugc nofollow">Pods</a>. Typically you have several nodes in a cluster; in a learning or resource-limited environment, you might have only one node. The <a class="au ld" href="https://kubernetes.io/docs/concepts/overview/components/#node-components" target="_blank" rel="noopener ugc nofollow">components</a> on a node include the <a class="au ld" href="https://kubernetes.io/docs/reference/generated/kubelet" target="_blank" rel="noopener ugc nofollow">kubelet</a>, a <a class="au ld" href="https://kubernetes.io/docs/setup/production-environment/container-runtimes" target="_blank" rel="noopener ugc nofollow">container runtime</a>, and the <a class="au ld" href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/" target="_blank" rel="noopener ugc nofollow">kube-proxy</a>.
<code>kubectl taint node &lt;node_name&gt;</code> &#8211; Update the taints on one or more nodes.
<code>kubectl get node</code> &#8211; List one or more nodes.
<code>kubectl delete node &lt;node_name&gt;</code> &#8211; Delete a node or multiple nodes.
<code>kubectl top node &lt;node_name&gt;</code> &#8211; Display Resource usage (CPU/Memory/Storage) for nodes.
<code>kubectl get pods -o wide | grep &lt;node_name&gt;</code> &#8211; Pods running on a node.
<code>kubectl annotate node &lt;node_name&gt;</code> &#8211; Annotate a node.
<code>kubectl cordon node &lt;node_name&gt;</code> &#8211; Mark a node as unschedulable.
<code>kubectl uncordon node &lt;node_name&gt;</code> &#8211; Mark node as schedulable.
<code>kubectl drain node &lt;node_name&gt;</code> &#8211; Drain a node in preparation for maintenance.
<code>kubectl label node</code> &#8211; Add or update the labels of one or more nodes.

<a href="https://kubernetes.io/docs/concepts/workloads/pods/" target="_blank" rel="nofollow noopener">Pods</a> &#8211; Pods are the smallest deployable units of computing that you can create and manage in Kubernetes. A Pod (as in a pod of whales or pea pod) is a group of one or more <a class="au ld" href="https://kubernetes.io/docs/concepts/containers/" target="_blank" rel="noopener ugc nofollow">containers</a>, with shared storage and network resources, and a specification for how to run the containers. A Pod’s contents are always co-located and co-scheduled, and run in a shared context. A Pod models an application-specific “logical host”: it contains one or more application containers which are relatively tightly coupled. In non-cloud contexts, applications executed on the same physical or virtual machine are analogous to cloud applications executed on the same logical host. As well as application containers, a Pod can contain <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/pods/init-containers/" target="_blank" rel="noopener ugc nofollow">init containers</a> that run during Pod startup. You can also inject <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/" target="_blank" rel="noopener ugc nofollow">ephemeral containers</a> for debugging if your cluster offers this.
<code>kubectl get pod</code> &#8211; List one or more pods.
<code>kubectl get pods --sort-by='.status.containerStatuses[0].restartCount'</code> &#8211; List pods Sorted by Restart Count.
<code>kubectl get pods --field-selector=status.phase=Running</code> &#8211; Get all running pods in the namespace.
<code>kubectl delete pod &lt;pod_name&gt;</code> &#8211; Delete a pod.
<code>kubectl describe pod &lt;pod_name&gt;</code> &#8211; Display the detailed state of a pods.
<code>kubectl create pod &lt;pod_name&gt;</code> &#8211; Create a pod.
<code>kubectl exec &lt;pod_name&gt; -c &lt;container_name&gt; &lt;command&gt;</code> &#8211; Execute a command against a container in a pod. Read more: <a href="https://spacelift.io/blog/kubectl-exec" target="_blank" rel="noopener">Using Kubectl Exec: Connect to Your Kubernetes Containers</a>
<code>kubectl exec -it &lt;pod_name&gt; /bin/sh</code> &#8211; Get an interactive shell on a single-container pod.
<code>kubectl top pod</code> &#8211; Display Resource usage (CPU/Memory/Storage) for pods.
<code>kubectl annotate pod &lt;pod_name&gt; &lt;annotation&gt;</code> &#8211; Add or update the annotations of a pod.
<code>kubectl label pods &lt;pod_name&gt; new-label=&lt;label name&gt;</code> &#8211; Add or update the label of a pod.
<code>kubectl get pods --show-labels</code> &#8211; Get pods and show labels.
<code>kubectl port-forward &lt;pod name&gt; &lt;port number to listen on&gt;:&lt;port number to forward to&gt;</code> &#8211; Listen on a port on the local machine and forward to a port on a specified pod.

<a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicationcontroller/" target="_blank" rel="nofollow noopener">Replication Controllers</a> &#8211; Note: A <code class="gr kz la lb lc b"><a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/" target="_blank" rel="noopener ugc nofollow">Deployment</a></code> that configures a <code class="gr kz la lb lc b"><a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/" target="_blank" rel="noopener ugc nofollow">ReplicaSet</a></code> is now the recommended way to set up replication. A ReplicationController ensures that a specified number of pod replicas are running at any one time. In other words, a ReplicationController makes sure that a pod or a homogeneous set of pods is always up and available.
<code>kubectl get rc</code> &#8211; List the replication controllers.
<code>kubectl get rc --namespace=”&lt;namespace_name&gt;”</code> &#8211; List the replication controllers by namespace.

<a href="https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/" target="_blank" rel="nofollow noopener">ReplicaSets</a> &#8211; A ReplicaSet’s purpose is to maintain a stable set of replica Pods running at any given time. As such, it is often used to guarantee the availability of a specified number of identical Pods.
<code>kubectl get replicasets</code> &#8211; List ReplicaSets.
<code>kubectl describe replicasets &lt;replicaset_name&gt;</code> &#8211; Display the detailed state of one or more ReplicaSets.
<code>kubectl scale --replicas=[x]</code> &#8211; Scale a ReplicaSet.

<a href="https://kubernetes.io/docs/concepts/configuration/secret/" target="_blank" rel="nofollow noopener">Secrets</a> &#8211; A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a key. Such information might otherwise be put in a <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/pods/" target="_blank" rel="noopener ugc nofollow">Pod</a> specification or in a <a class="au ld" href="https://kubernetes.io/docs/reference/glossary/?all=true#term-image" target="_blank" rel="noopener ugc nofollow">container image</a>. Using a Secret means that you don’t need to include confidential data in your application code.
<code>kubectl create secret</code> &#8211; Create a secret.
<code>kubectl get secrets</code> &#8211; List secrets.
<code>kubectl describe secrets</code> &#8211; List details about secrets.
<code>kubectl delete secret &lt;secret_name&gt;</code> &#8211; Delete a secret.

<a href="https://kubernetes.io/docs/concepts/services-networking/service/" target="_blank" rel="nofollow noopener">Services</a> &#8211; An abstract way to expose an application running on a set of <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/pods/" target="_blank" rel="noopener ugc nofollow">Pods</a> as a network service. With Kubernetes you don’t need to modify your application to use an unfamiliar service discovery mechanism. Kubernetes gives Pods their own IP addresses and a single DNS name for a set of Pods, and can load-balance across them.
<code>kubectl get services</code> &#8211; List one or more services.
<code>kubectl describe services</code> &#8211; Display the detailed state of a service.
<code>kubectl expose deployment [deployment_name]</code> &#8211; Expose a replication controller, service, deployment, or pod as a new Kubernetes service.
<code>kubectl edit services</code> &#8211; Edit and update the definition of one or more services.

<a href="https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/" target="_blank" rel="nofollow noopener">Service Accounts</a> &#8211; A service account provides an identity for processes that run in a Pod.
Note: This document is a user introduction to Service Accounts and describes how service accounts behave in a cluster set up as recommended by the Kubernetes project. Your cluster administrator may have customized the behavior in your cluster, in which case this documentation may not apply.
When you (a human) access the cluster (for example, using <code class="gr kz la lb lc b">kubectl</code>), you are authenticated by the apiserver as a particular User Account (currently this is usually <code class="gr kz la lb lc b">admin</code>, unless your cluster administrator has customized your cluster). Processes in containers inside pods can also contact the apiserver. When they do, they are authenticated as a particular Service Account (for example, <code class="gr kz la lb lc b">default</code>).
<code>kubectl get serviceaccounts</code> &#8211; List service accounts.
<code>kubectl describe serviceaccounts</code> &#8211; Display the detailed state of one or more service accounts.
<code>kubectl replace serviceaccount</code> &#8211; Replace a service account.
<code>kubectl delete serviceaccount &lt;service_account_name&gt;</code> &#8211; Delete a service account.

<a href="https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/" target="_blank" rel="nofollow noopener">StatefulSet</a> &#8211; StatefulSet is the workload API object used to manage stateful applications. Manages the deployment and scaling of a set of <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/pods/" target="_blank" rel="noopener ugc nofollow">Pods</a>, and provides guarantees about the ordering and uniqueness of these Pods.
Like a <a class="au ld" href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/" target="_blank" rel="noopener ugc nofollow">Deployment</a>, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.
If you want to use storage volumes to provide persistence for your workload, you can use a StatefulSet as part of the solution. Although individual Pods in a StatefulSet are susceptible to failure, the persistent Pod identifiers make it easier to match existing volumes to the new Pods that replace any that have failed.
<code>kubectl get statefulset</code> &#8211; List StatefulSet
<code>kubectl delete statefulset/[stateful_set_name] --cascade=false</code> &#8211; Delete StatefulSet only (not pods).

<a href="https://docs.spacelift.io/" target="_blank" rel="noopener">Spacelift</a> helps you manage the complexities and compliance challenges of using Kubernetes. It brings with it a GitOps flow, so your Kubernetes Deployments are synced with your Kubernetes Stacks, and pull requests show you a preview of what they&#8217;re planning to change. It also has an extensive selection of <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">policies</a>, which lets you <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">automate compliance checks</a> and <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">build complex multi-stack workflows</a>. You can also use Spacelift to mix and match Terraform, Pulumi, CloudFormation, and Kubernetes Stacks and have them talk to one another. For example, you can set up Terraform Stacks to provision the required infrastructure (like an ECS/EKS cluster with all its dependencies) and then deploy the following via a Kubernetes Stack.

When using K8S, cut to the chase and use a cheat sheet!
Cheers!

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities s for infrastructure management.

15 Kubernetes Best Practices Every Developer Should Know

What Is a Kubernetes Cluster? Key Components Explained

How to Install Kubernetes - 4 Different Ways [Step-by-Step]

See the helpful list of each commonly used category or component of Kubernetes (K8S) with appropriate kubectl commands for quick reference!

91-kubernetes-cheat-sheet

Kubernetes cheat sheet

Kubectl Cheat Sheet - 15 Kubernetes Commands & Objects

In this post, we will delve into Kubernetes (K8s) deployment concepts and some common strategies, looking at the advantages and disadvantages of each. A suitable deployment strategy enables you to minimize downtime, enhance your customer experience, and increase reliability when releasing your application.

A K8s Deployment is a declarative statement usually configured in a YAML file that defines the application lifecycle and how updates to that application should be applied.
When deploying your applications to a <a href="https://spacelift.io/blog/kubernetes-cluster" target="_blank" rel="noopener">K8s cluster</a>, your chosen deployment strategy will determine how those applications are updated to a newer version from an older one. Some strategies will involve downtime, some will introduce testing concepts and enable user analysis. There are two basic commonly used K8s deployment strategies we will look at in this post:
<ul class="">
<li id="d400" class="mp mq io kf b kg kh kj kk km mr kq ms ku mt ky mu mv mw mx gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-deployment-strategies#1-recreating-deployments">Recreating</a></li>
<li id="8ab3" class="mp mq io kf b kg my kj mz km na kq nb ku nc ky mu mv mw mx gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-deployment-strategies#2-rolling-deployments">Rolling</a></li>
</ul>
The following three strategies are considered &#8220;Advanced deployment strategies&#8221; because the flow of traffic can be controlled in various ways:
<ul class="">
<li id="a009" class="mp mq io kf b kg kh kj kk km mr kq ms ku mt ky mu mv mw mx gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-deployment-strategies#3-bluegreen-deployments" rel="">Blue/Green</a></li>
<li id="ba8f" class="mp mq io kf b kg my kj mz km na kq nb ku nc ky mu mv mw mx gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-deployment-strategies#4-canary-deployments">Canary</a></li>
<li id="49bc" class="mp mq io kf b kg my kj mz km na kq nb ku nc ky mu mv mw mx gs" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-deployment-strategies#5-ab-deployments" rel="">A/B</a></li>
</ul>
K8s uses a Rolling deployment strategy as the default, but there are certain use cases when this may not be appropriate. Let&#8217;s discuss each in more detail!
See how Spacelift can help you manage the complexities and compliance challenges of using Kubernetes with the newest <a href="https://docs.spacelift.io/vendors/kubernetes/" target="_blank" rel="noopener">Kubernetes integration</a>. It brings with it a GitOps flow, so your Kubernetes Deployments are synced with your Kubernetes Stacks, and pull requests show you a preview of what they&#8217;re planning to change. It also has an extensive selection of <a href="https://docs.spacelift.io/concepts/policy/" target="_blank" rel="noopener">policies</a>, which lets you <a href="https://docs.spacelift.io/concepts/policy/terraform-plan-policy" target="_blank" rel="noopener">automate compliance checks</a> and <a href="https://docs.spacelift.io/concepts/policy/trigger-policy" target="_blank" rel="noopener">build complex multi-stack workflows</a>.

Recreating deployment terminates all the pods and replaces them with the new version. This can be useful in situations where an old and new version of the application cannot run at the same time. The amount of downtime incurred using this strategy will depend on how long the application takes to shut down and start back up. The application state is entirely renewed since they are completely replaced.
An example Spec: section in the manifest file could look like this:

Rolling deployments are the default K8S offering designed to reduce downtime to the cluster. A rolling deployment replaces pods running the old version of the application with the new version without downtime.
To achieve this, Readiness probes are used:
<ul class="">
<li id="ec87" class="mp mq io kf b kg kh kj kk km mr kq ms ku mt ky mu mv mw mx gs" data-selectable-paragraph="">Readiness probes monitor when the application becomes available. If the probes fail, no traffic will be sent to the pod. These are used when an app needs to perform certain initialization steps before it becomes ready. An application may also become overloaded with traffic and cause the probe to fail, preventing more traffic from being sent to it and allowing it to recover.</li>
</ul>
Once the readiness probe detects the new version of the application is available, the old version of the application is removed. If there is a problem, the rollout can be stopped and rolled back to the previous version, avoiding downtime across the entire cluster. Because each pod is replaced one by one, deployments can take time for larger clusters. If a new deployment is triggered before another has finished, the version is updated to the version specified in the new deployment, and the previous deployment version is disregarded where it has not yet been applied.
A rolling deployment is triggered when something in the pod spec is changed, such as when the image, environment, or label of a pod is updated. A pod image can be updated using the command <code class="gr nj nk nl nf b">kubectl set image</code>.
The spec: -&gt; strategy: section of the manifest file can be used to refine the deployment by making use of two optional parameters — <code class="gr nj nk nl nf b">maxSurge</code> and <code class="gr nj nk nl nf b">maxUnavailable</code>. Both can be specified using a percentage or absolute number. A percentage figure should be used when Horizontal Pod Autoscaling is used.
<ul class="">
<li id="bbc5" class="mp mq io kf b kg kh kj kk km mr kq ms ku mt ky mu mv mw mx gs" data-selectable-paragraph="">MaxSurge specifies the maximum number of pods the Deployment is allowed to create at one time.</li>
<li id="87a3" class="mp mq io kf b kg my kj mz km na kq nb ku nc ky mu mv mw mx gs" data-selectable-paragraph="">MaxUnavailable specifies the maximum number of pods that are allowed to be unavailable during the rollout.</li>
</ul>
For example, the configuration below would specify a requirement for 10 replicas, with a maximum of 3 being created at any one time, allowing for 1 to be unavailable during the rollout:

<ul>
<li><a href="https://spacelift.io/blog/provision-kubernetes-cluster-terraform-spacelift" target="_blank" rel="noopener">How to Provision Kubernetes Cluster with Spacelift and Terraform</a></li>
<li><a href="https://spacelift.io/blog/kubernetes-best-practices" target="_blank" rel="noopener">15 Kubernetes Best Practices to Follow</a></li>
<li><a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener">Top Most Useful CI/CD Tools for DevOps</a></li>
</ul>

A Blue/Green deployment involves deploying the new application version (green) alongside the old one (blue). A <a href="https://spacelift.io/blog/kubernetes-load-balancer" target="_blank" rel="noopener">load balancer</a> in the form of the service selector object is used to direct the traffic to the new application (green) instead of the old one when it has been tested and verified. Blue/Green deployments can prove costly as twice the amount of application resources need to be stood up during the deployment period.
To enable this, we set up a service sitting in front of the deployments. For example, the service selector section of the manifest file for the blue deployment for an app called web-app with v1.0.0 could look like the below:

And the deployment for the blue web app:

When we want to direct traffic to the new (green) version of the app, we update the manifest to point to the new version, v2.0.0.

The deployment for the green app:

A Canary deployment can be used to let a subset of the users test a new version of the application or when you are not fully confident about the new version&#8217;s functionality. This involves deploying a new version of the application alongside the old one, with the old version of the application serving most users and the newer version serving a small pool of test users. The new deployment is rolled out to more users if it is successful.
For example, in a <a href="https://spacelift.io/blog/provision-kubernetes-cluster-terraform-spacelift" target="_blank" rel="noopener">K8s cluster</a> with 100 running pods, 95 could be running v1.0.0 of the application, with 5 running the new v2.0.0 of the application. 95% of the users will be routed to the old version, and 5% will be routed to the new version. For this, we use 2 deployments side-by-side that can be scaled separately.
The spec section of the old application manifest would look like the following:

And the new application manifest:

In the example above, it might be impractical and costly to run 100 pods. A better way to achieve this is to use a load balancer such as <a class="au lo" href="https://www.nginx.com/" target="_blank" rel="noopener ugc nofollow">NGINX</a>, <a class="au lo" href="http://www.haproxy.org/" target="_blank" rel="noopener ugc nofollow">HAProxy</a>, or <a class="au lo" href="https://traefik.io/" target="_blank" rel="noopener ugc nofollow">Traefik</a>, or a <a href="https://spacelift.io/blog/what-is-a-service-mesh" target="_blank" rel="noopener">service mesh</a> like <a class="au lo" href="https://istio.io/" target="_blank" rel="noopener ugc nofollow">Istio</a>, <a class="au lo" href="https://www.consul.io/" target="_blank" rel="noopener ugc nofollow">Hashicorps Consul</a>, or <a class="au lo" href="https://linkerd.io/" target="_blank" rel="noopener ugc nofollow">Linkrd</a>.
For painless and efficient maintenance of your K8s clusters, follow these <a href="https://spacelift.io/blog/kubernetes-best-practices" target="_blank" rel="noopener">15 Kubernetes Best Practices</a>.

Similar to a Canary deployment, using an A/B deployment, you can target a given subsection of users based on some target parameters (usually the HTTP headers or a cookie), as well as distribute traffic amongst versions based on weight. This technique is widely used to test the conversion of a given feature, and then the version that converts the most is rolled out.
This approach is usually taken based on data collected on user behavior and can be used to make better business decisions. Users are usually uninformed of the new features during the A/B testing period, so true testing can be done, and experiences between the users using the old version and those using the new version can be compared. Rollouts can be slower using A/B deployments due to the additional testing period and analysis of the user experience.
A/B deployments can be automated using <a class="au lo" href="https://istio.io/" target="_blank" rel="noopener ugc nofollow">Istio</a> and <a class="au lo" href="https://flagger.app/" target="_blank" rel="noopener ugc nofollow">Flagger</a>, check out the <a class="au lo" href="https://docs.flagger.app/tutorials/istio-ab-testing" target="_blank" rel="noopener ugc nofollow">tutorial here</a> for more information on how to set it up.

In this article, we discussed five common K8s deployment strategies: Recreate, Rolling, Blue/Green, Canary, and A/B. Having an understanding of how these can be used, the tools that can be used to enable each of them, and the advantages and disadvantages of each is key when deciding how to deploy or upgrade your application to a newer version. Choosing the right strategy for your business needs can help reduce downtime, enable testing and improve the customer feedback loop enabling your team to develop a better product over time.
Cheers!

CI/CD Pipeline : Everything You Need To Know

Terraform vs. Kubernetes : Key Differences and Comparison

Kubernetes CI/CD Pipelines - 7 Best Practices and Tools

Delve into different Kubernetes (K8s) deployment strategies. Take a look at the advantages and disadvantages of each of them.

85-kubernetes-deployment-strategies

Kubernetes Deployment Strategies

5 Different Types of Kubernetes Deployment Strategies

In this article, we will briefly take a look at what Kubernetes is (abbreviated as K8S) and explain what a K8s cluster is. We will then look at the components of the master node and worker nodes that make up a K8s cluster.
Kubernetes is an open-source container orchestration system used for the management and deployment of containerized applications.

A <dfn>Kubernetes (K8s) cluster</dfn> is an open-source container orchestration platform that focuses on managing, deploying, and scaling containerized applications. It provides a way to manage containerized applications across multiple nodes, providing a flexible and scalable infrastructure for building, deploying, and managing modern applications. 
The K8s cluster can run on virtual or physical machines or can be used as a service inside multiple cloud providers. Organizations of all sizes widely adopt Kubernetes, and it has become the de facto standard for container orchestration.

A K8s cluster is made up of a group of nodes. Nodes are machines that are linked together to form a ‘cluster’. Nodes can be thought of as individual compute resources, so pretty much any group of compute resources can be used. Usually, nodes are made up of a group of virtual machines if the platform is running in the cloud behind the veil of a PaaS (Platform-as-a-service) offering. They can also be run on serverless offerings, such as Amazon Fargate. If the K8s cluster is running on-premises, then nodes can be physical machines. You can even run a K8s cluster on a group of Raspberry Pis!
K8s includes a control plane, which is responsible for the management and orchestration of the containers across the nodes and for maintaining the desired state. The goal of K8s is to provide maximum application uptime by scheduling applications across nodes in the most efficient manner to enable peak performance utilization and redundancy.
If you want easy and efficient maintenance of your K8s cluster, see <a href="https://spacelift.io/blog/kubernetes-best-practices" target="_blank" rel="noopener">15 Kubernetes Best Practices Every Developer Should Know</a>.

A K8s cluster consists of two types of nodes, master nodes and worker nodes.
Master nodes host the K8s control plane components. The master node will hold configuration and state data used to maintain the desired state. The control plane maintains communication with the worker nodes in order to schedule containers efficiently. In production, the control plane runs across multiple nodes to ensure redundancy should one fail.
Worker nodes are so-called because they run pods. Pods are usually single instances of an application. Containers run inside pods, Pods run on a node. Each cluster always contains at least one worker node. It is recommended that a cluster consists of at least three worker nodes for redundancy purposes.
Read more about <a href="https://spacelift.io/blog/kubernetes-architecture" target="_blank" rel="noopener">Kubernetes Architecture components</a>.

componets-of-a-kubernetes-cluster

componets of a kubernetes cluster

<ul>
<li><a href="https://spacelift.io/blog/provision-kubernetes-cluster-terraform-spacelift" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Provision Kubernetes Cluster with Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Provision Kubernetes Cluster with Terraform</a></li>
<li><a href="https://spacelift.io/blog/kubernetes-best-practices" target="_blank" rel="noopener">15 Kubernetes Best Practices to Follow</a></li>
<li><a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener">Top Most Useful CI/CD Tools for DevOps</a></li>
</ul>

<ul class="">
<li id="f6b3" class="mb mc ip jo b jp lv jt lw jx md kb me kf mf kj mg mh mi mj gt" data-selectable-paragraph="">API server (kube-apiserver)</li>
</ul>
The REST API acts as a front door to the control plane. Any requests for modifications to the cluster come through the API, as well as any requests from worker nodes. If the requests are valid the API server executes them. Communications with the cluster occurs either via the REST API, or command-line tools like <code class="gs mk ml mm mn b">kubeadm</code> or <code class="gs mk ml mm mn b">kubectl.</code>
<ul class="">
<li id="4ccc" class="mb mc ip jo b jp jq jt ju jx mo kb mp kf mq kj mg mh mi mj gt" data-selectable-paragraph="">Scheduler (kube-scheduler)</li>
</ul>
The scheduler determines where resources will be deployed on which nodes within the K8s cluster. It does this by scheduling pods of containers across nodes and makes these decisions based on monitoring resource utilization. It places the resources on healthy nodes that can fulfill the requirements of the application.
<ul class="">
<li id="8415" class="mb mc ip jo b jp jq jt ju jx mo kb mp kf mq kj mg mh mi mj gt" data-selectable-paragraph="">Controller (kube-controller-manager)</li>
</ul>
The controller runs a number of background processes and also initiates any requested changes to the state. It ensures that the cluster state remains consistent. The controller can identify failed nodes and take action, and also runs jobs as required (jobs are one-time tasks). Also when a new namespace is created, this controller creates default accounts and API access tokens.
<ul class="">
<li id="b91a" class="mb mc ip jo b jp jq jt ju jx mo kb mp kf mq kj mg mh mi mj gt" data-selectable-paragraph="">Etcd (key-value store)</li>
</ul>
Configuration data about the state is stored in a key-value store that all nodes can access in the cluster. It is a critical component that is distributed and fault-tolerant. Etcd can be a target for malicious actors as it provides effective control over the cluster and should be secured.
<ul class="">
<li id="b465" class="mb mc ip jo b jp jq jt ju jx mo kb mp kf mq kj mg mh mi mj gt" data-selectable-paragraph="">Cloud Controller Manager (cloud-controller-manager)</li>
</ul>
An optional component, the cloud controller manager can be used to link the K8s API to the cloud providers API. Once linked, it can enable the cluster to scale horizontally. Components that interact with the cloud providers are separated from those that only interact with the K8s API to optimize performance.

<ul class="">
<li id="1a6a" class="mb mc ip jo b jp lv jt lw jx md kb me kf mf kj mg mh mi mj gt" data-selectable-paragraph="">Pods &amp; Containers</li>
</ul>
Worker nodes are so-called because they run pods that do the work. Pods are usually single instances of the application itself. Containers run inside pods, Pods run on a node. If more resource is needed in the K8s cluster to run the workloads, more nodes can be added to the cluster.
<ul class="">
<li id="be33" class="mb mc ip jo b jp jq jt ju jx mo kb mp kf mq kj mg mh mi mj gt" data-selectable-paragraph="">Kubelet</li>
</ul>
Each worker node runs a small application called the <code class="gs mk ml mm mn b">kubelet</code>, which enables communication with the control plane. If the control plane needs to make any changes on the worker nodes, the request will go via the <code class="gs mk ml mm mn b">kubelet</code>.
<ul class="">
<li id="67c2" class="mb mc ip jo b jp jq jt ju jx mo kb mp kf mq kj mg mh mi mj gt" data-selectable-paragraph="">Kube-proxy</li>
</ul>
Each worker node also runs a component to control the networking, within and outside the cluster, called <code class="gs mk ml mm mn b">kube-proxy</code>. Traffic can be forwarded independently or using the Operating systems packet filtering layer. It acts as a <a href="https://spacelift.io/blog/kubernetes-load-balancer" target="_blank" rel="noopener">load balancer</a> and forwards requests from the control plane to the correct pods.

Creating a Kubernetes cluster can be a challenging task if you want to install all the components manually. Nevertheless, if you’ve never done it before, it is the recommended approach to follow through with one of <a href="https://kubernetes.io/docs/setup/production-environment/tools/" target="_blank" rel="nofollow noopener">the installation processes</a> to get familiar with all of the components. 
On the other hand, if you want to get up and running quickly, <a href="https://kind.sigs.k8s.io/docs/user/quick-start/" target="_blank" rel="nofollow noopener">Kind</a> might be a very easy solution to use on your local machine or a VM. It uses Docker containers to simulate Kubernetes nodes, making it possible to create and run a Kubernetes cluster with a single command. Additionally, Kind is compatible with many popular Kubernetes tools, including kubectl and Helm, making it easy to use in existing workflows. There are also other lightweight tools available for this, like <a href="https://minikube.sigs.k8s.io/docs/start/" target="_blank" rel="nofollow noopener">Minikube</a> or <a href="https://k3s.io/" target="_blank" rel="nofollow noopener">K3s</a>. 
If you are using a Cloud Provider like AWS, Azure, GCP, or OCI, you can easily use their managed Kubernetes Services like <a href="https://aws.amazon.com/eks/" target="_blank" rel="nofollow noopener">EKS</a>, <a href="https://azure.microsoft.com/en-us/products/kubernetes-service#:~:text=Azure%20Kubernetes%20Service%20(AKS)%20offers,edge%2C%20and%20multicloud%20Kubernetes%20clusters." target="_blank" rel="nofollow noopener">AKS</a>, <a href="https://cloud.google.com/kubernetes-engine" target="_blank" rel="nofollow noopener">GKE</a>, and <a href="https://www.oracle.com/cloud/cloud-native/container-engine-kubernetes/" target="_blank" rel="nofollow noopener">OKE</a> in order to spin up a Kubernetes cluster in a couple of minutes.

A K8S cluster has the desired state, usually defined using multiple YAML (or JSON) files.
The desired state means that rather than issuing specific individual instructions to the system, the desired end state is described and K8s decides which commands to issue to achieve the end goal.
The YAML files define things such as which pods to run, how many pods to run, and the amount of resources the pods require. Requests are made to a K8s cluster through the K8s API, using <code class="gs mk ml mm mn b">kubectl</code> via the command line, or via the REST API.
You can also take a look at <a href="https://docs.spacelift.io/vendors/kubernetes/" target="_blank" rel="noopener">how Kubernetes is integrated into Spacelift</a>. Spacelift helps you manage the complexities and compliance challenges of using Kubernetes. It brings with it a GitOps flow, so your Kubernetes Deployments are synced with your Kubernetes Stacks, and pull requests show you a preview of what they&#8217;re planning to change. If you aren’t already using Spacelift, <a href="https://spacelift.io/free-trial" target="_blank" rel="noopener">sign up for a free trial</a>.

Understanding the components that make up a K8s cluster is important to give you the foundations in K8s operations. Knowing what each component of the cluster does aids in the operation, management, and troubleshooting of any issues that might arise with the cluster.

Spacelift is a flexible orchestration solution for IaC development. It delivers enhanced collaboration, automation and controls to simplify and accelerate the provisioning of cloud based infrastructures.

Initial Availability of Kubernetes Integration in Spacelift

Enable Auto-completion for Kubectl in a Linux Bash Shell

Learn the foundation of K8s operations. Take a look at the components of the master node and worker nodes that make up a Kubernetes Cluster.

90-kubernetes-cluster

What is a Kubernetes Cluster?

In this post, I will run through some best practices to use when using Kubernetes (K8s).
As the most popular container orchestration system, K8s is the de-facto standard for the modern cloud engineer to get to grips with. K8s is a notoriously complex system to use and maintain, so getting a good grasp of what you should and should not be doing, and knowing what is possible will get your deployment off to a solid start.
These recommendations cover common issues within 3 broad categories, application development, governance, and cluster configuration.
<h3 id="b48f" class="lq lr ip bn ls lt lu lv lw lx ly lz ma kn mb mc md kr me mf mg kv mh mi mj mk gt">Kubernetes Best Practices:</h3>
<ol class="">
<li id="d51a" class="ml mm ip kg b kh mn kk mo kn mp kr mq kv mr kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-namespaces" rel="">Use namespaces</a></li>
<li id="10ef" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-readiness-and-liveness-probes">Use readiness and liveness probes</a></li>
<li id="39ca" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-autoscaling">Use resource requests and limits</a></li>
<li id="b033" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#deploy-your-pods-as-part-of-a-deployment-daemonset-replicaset-or-statefulset-across-nodes">Deploy your Pods as part of a Deployment, DaemonSet, ReplicaSet or StatefulSet across nodes.</a></li>
<li id="257b" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-multiple-nodes">Use multiple nodes</a></li>
<li id="3469" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-rolebased-access-control-rbac">Use Role-based access control (RBAC)</a></li>
<li id="3bde" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#host-your-kubernetes-cluster-externally-use-a-cloud-service">Host your Kubernetes cluster externally (use a cloud service)</a></li>
<li id="8020" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#upgrade-your-kubernetes-version">Upgrade your Kubernetes version</a></li>
<li id="aceb" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#monitor-your-cluster-resources-and-audit-policy-logs">Monitor your cluster resources and audit policy logs</a></li>
<li id="1c77" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-a-version-control-system">Use a version control system</a></li>
<li id="5eae" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-a-gitbased-workflow-gitops">Use a Git-based workflow (GitOps)</a></li>
<li id="00e8" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#reduce-the-size-of-your-containers">Reduce the size of your containers</a></li>
<li id="5459" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#organize-your-objects-with-labels">Organize your objects with labels</a></li>
<li id="8a71" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-network-policies">Use network policies</a></li>
<li id="eecf" class="ml mm ip kg b kh mw kk mx kn my kr mz kv na kz ms mt mu mv gt" data-selectable-paragraph=""><a href="https://spacelift.io/blog/kubernetes-best-practices#use-a-firewall">Use a firewall</a></li>
</ol>

Namespaces in K8s are important to utilize in order to organize your objects, create logical partitions within your cluster, and for security purposes. By default, there are three namespaces in a <a href="https://spacelift.io/blog/kubernetes-cluster" target="_blank" rel="noopener">K8s cluster</a>, <code class="gs nl nm nn no b">default</code>, <code class="gs nl nm nn no b">kube-public</code> and <code class="gs nl nm nn no b">kube-system</code>.
RBAC can be used to control access to particular namespaces in order to limit the access of a group to control the blast-radius of any mistakes that might occur, for example, a group of developers may only have access to a namespace called <code class="gs nl nm nn no b">dev</code> , and have no access to the <code class="gs nl nm nn no b">production</code> namespace. The ability to limit different teams to different namespaces can be valuable to avoid duplicated work or resource conflict.
LimitRange objects can also be configured against namespaces to define the standard size for a container deployed in the namespace. ResourceQuotas can also be used to limit the total resource consumption of all containers inside a Namespace. Network policies can be used against namespaces to limit traffic between pods.
Check out our <a href="https://spacelift.io/blog/kubernetes-namespaces" target="_blank" rel="noopener">Working with Kubernetes Namespaces</a> tutorial.

Readiness and Liveness probes are essentially types of health checks. These are another very important concept to utilize in K8s.
Readiness probes ensure that requests to a pod are only directed to it when the pod is ready to serve requests. If it is not ready, then requests are directed elsewhere. It is important to define the readiness probe for each container, as there are no default values set for these in K8s.
For example, if a pod takes 20 seconds to start and the readiness probe was missing, then any traffic directed to that pod during the startup time would cause a failure. Readiness probes should be independent and not take into account any dependencies on other services, such as a backend database or caching service.
<a href="https://spacelift.io/blog/kubernetes-liveness-probe" target="_blank" rel="noopener">Liveness probes</a> test if the application is running in order to mark it as healthy. For example, a particular path of a web app could be tested to ensure it is responding. If not, the pod will not be marked as healthy and the probe failure will cause the <code class="gs nl nm nn no b">kubelet</code>to launch a new pod, which will then be tested again. This type of probe is used as a recovery mechanism in case the process becomes unresponsive.

Where it is appropriate, autoscaling can be employed to dynamically adjust the number of pods (horizontal pod autoscaler), the amount of resources consumed by the pods (vertical autoscaler), or the number of nodes in the cluster (cluster autoscaler), depending on the demand for the resources.
The horizontal pod autoscaler can also scale a replication controller, replica set, or <a href="https://spacelift.io/blog/kubernetes-statefulset" target="_blank" rel="noopener">statefulset</a> based on CPU demand.
Using scaling also brings some challenges, such as not storing persistent data in the container&#8217;s local filesystem, as this would prevent horizontal autoscaling. Instead, a PersistentVolume could be used. (Read more about <a href="https://spacelift.io/blog/kubernetes-persistent-volumes" target="_blank" rel="noopener">Kubernetes Persistent Volumes</a>).
The cluster autoscaler is useful when highly variable workloads exist on the cluster that may require different amounts of resources at different times based on demand. Removing unused nodes automatically is also a great way to save money!

Resource requests and limits (minimum and maximum amount of resources that can be used in a container) should be set to avoid a container starting without the required resources assigned, or the cluster running out of available resources.
Without limits, pods can utilize more resources than required, causing the total available resources to be reduced which may cause a problem with other applications on the cluster. Nodes may crash, and new pods may not be able to be placed corrected by the scheduler.
Without requests, if the application cannot be assigned enough resources, it may fail when attempting to start or perform erratically.
Resource requests and limits define the amount of CPU and Memory available in millicores and mebibytes. Note that if your process goes over the memory limit, the process is terminated, so it may not always be appropriate to set this in all cases. If your container goes over the CPU limit, the process is throttled.

<ul>
<li><a href="https://spacelift.io/blog/terraform-vs-kubernetes" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Terraform vs. Kubernetes : Key Differences and Comparison&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Terraform vs. Kubernetes Comparison</a></li>
<li><a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;5 Most Useful CI/CD Tools for DevOps in 2022&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">Top Most Useful CI/CD Tools for DevOps</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices" target="_blank" rel="noopener">16 DevOps Best Practices to Follow</a></li>
</ul>

A single pod should never be run individually. To improve fault tolerance, instead, they should always be part of a Deployment, DaemonSet, ReplicaSet or StatefulSet. The pods can then be deployed across nodes using anti-affinity rules in your deployments to avoid all pods being run on a single node, which may cause downtime if it was to go down.
See <a href="https://spacelift.io/blog/statefulset-vs-deployment" target="_blank" rel="noopener">StatefulSet vs. Deployment</a>.

Running K8s on a single node is not a good idea if you want to build in fault tolerance. Multiple nodes should be employed in your cluster so workloads can be spread between them.

<a href="https://spacelift.io/blog/kubernetes-security#1-use-rbac" target="_blank" rel="noopener">Using RBAC in your K8s cluster</a> is essential to properly secure your system. Users, Groups, and Service accounts can be assigned permissions to perform permitted actions on a particular namespace (a Role), or to the entire cluster (ClusterRole). Each role can have multiple permissions. To tie the defined roles to the users, groups, or service accounts, RoleBinding or ClusterRoleBinding objects are used.
RBAC roles should be set up to grant using the principle of least privilege, i.e. only permissions that are required are granted. For example, the admin&#8217;s group may have access to all resources, and your operator&#8217;s group may be able to deploy but not be able to read <a href="https://spacelift.io/blog/kubernetes-secrets" target="_blank" rel="noopener">Secrets</a>.

Hosting a K8s cluster on your own hardware can be a complex undertaking. Cloud services offer K8s clusters as platform as a service (PaaS), such as AKS (Azure Kubernetes Service) on Azure, or EKS (Amazon Elastic Kubernetes Service) on Amazon Web Services. Taking advantage of this means the underlying infrastructure will be managed by your cloud provider, and tasks around scaling your cluster, such as adding and removing nodes can be much more easily achieved, leaving your engineers to the management of what is running on the K8s cluster itself.

As well as introducing new features, new K8s versions also include vulnerability and security fixes, which make it important to run an up-to-date version of K8s on your cluster. Support for older versions will likely not be as good as newer ones.
Migrating to a new version should be treated with caution however as certain features can be depreciated, as well as new ones added. Also, the apps running on your cluster should be checked that they are compatible with the newer targeted version before upgrading.

Monitoring the components in the K8s control plane is important to keep resource consumption under control. The control plane is the core of K8s, these components keep the system running and so are vital to correct K8s operations. Kubernetes API, kubelet, etcd, controller-manager, kube-proxy and kube-dns make up the control plane.
Control plane components can output metrics in a format that can be used by <a href="https://spacelift.io/blog/prometheus-kubernetes" target="_blank" rel="noopener">Prometheus</a>, the most common K8s monitoring tool.
Automated monitoring tools should be used rather than manually managing alerts.
Audit logging in K8s can be turned on whilst starting the kube-apiserver to enable deeper investigation using the tools of your choice. The audit.log will detail all requests made to the K8s API and should be inspected regularly for any issues that might be a problem on the cluster. The Kubernetes cluster default policies are defined in the audit-policy.yaml file and can be amended as required.
A log aggregation tool such as Azure Monitor can be used to send logs to a log analytics workspace from AKS for future interrogation using Kusto queries. On AWS Cloudwatch can be used. Third-party tools also provide deeper monitoring functionality such as Dynatrace and Datadog.
Finally, a defined retention period should be in place for the logs, around 30–45 days is common.

K8s configuration files should be controlled in a version control system (VCS). This enables a raft of benefits, including increased security, enabling an audit trail of changes, and will increase the stability of the cluster. Approval gates should be put in place for any changes made so the team can peer-review the changes before they are committed to the main branch.

Successful deployments of K8s require thought on the workflow processes used by your team. Using a git-based workflow enables automation through the use of CI/CD (Continuous Integration / Continuous Delivery) pipelines, which will increase application deployment efficiency and speed. CI/CD will also provide an audit trail of deployments. Git should be the single source of truth for all automation and will enable unified management of the K8s cluster. You can also consider using a dedicated infrastructure delivery platform such as <a class="au lp" href="https://spacelift.io" target="_blank" rel="noopener">Spacelift</a>, which recently introduced Kubernetes support.

Smaller image sizes will help speed up your builds and deployments and reduce the amount of resources the containers consumed on your K8s cluster. Uneccesery packages should be removed where possible, and small OS distribution images such as Alpine should be favored. Smaller images can be pulled faster than larger images, and consume less storage space.
Following this approach will also provide security benefits as there will be fewer potential vectors of attack for malicious actors.

K8s labels are key-value pairs that are attached to objects in order to organize your cluster resources. Labels should be meaningful metadata that provide a mechanism to track how different components in the K8s system interact.
Recommended labels for pods in the <a class="au lp" href="https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/" target="_blank" rel="noopener ugc nofollow">official K8s documentation</a> include <code class="gs nl nm nn no b">name, instance, version, component, part-of,</code> and <code class="gs nl nm nn no b">managed-by</code>.
Labels can also be used in a similar way to using tags in a cloud environment on resources in order to track things related to the business, such as object <code class="gs nl nm nn no b">ownership</code>, and the <code class="gs nl nm nn no b">environment</code>an object should belong to.
Also recommended is to use labels to detail security requirements, including <code class="gs nl nm nn no b">confidentiality</code> and <code class="gs nl nm nn no b">compliance</code>.

Network policies should be employed to restrict traffic between objects in the K8s cluster. By default, all containers can talk to each other in the network, something that presents a security risk if malicious actors gain access to a container, allowing them to traverse objects in the cluster. Network policies can control traffic at the IP and port level, similar to the concept of security groups in cloud platforms to restrict access to resources. Typically, all traffic should be denied by default, then allow rules should be put in place to allow required traffic.

As well as using network policies to restrict internal traffic on your K8s cluster, you should also put a firewall in front of your K8s cluster in order to restrict requests to the API server from the outside world. IP addresses should be whitelisted and open ports restricted.

Following the best practices listed in this article when designing, running, and maintaining your Kubernetes cluster will put you on the path to success on your modern application journey! You can also check out <a href="https://spacelift.io/blog/kubernetes-challenges" target="_blank" rel="noopener">15 challenges and pitfalls to look for each time you use Kubernetes</a>.

Provisioning AWS EKS Cluster with Terraform - Tutorial

Infrastructure as Code : Best Practices, Benefits & Examples

Follow the best practices for Kubernetes listed in this article for painless and efficient maintenance of your K8s cluster.

73-kubernetes-best-practices

Kubernetes Best Practices

Provision a Kubernetes Cluster with Spacelift and Terraform

Learn more about new features, updates, and Spacelift product announcements.

Product Features and Updates | Spacelift Blog

Product

Sean O'Dell

By now, you have probably heard about the initial availability of Kubernetes integration in Spacelift. You can <a href="https://spacelift.io/blog/spacelift-kubernetes-integration-beta" target="_blank" rel="noopener">read more about the announcement</a> and review the documentation.
Spacelift, like many of our customers, utilizes Kubernetes in a variety of ways. Before the integration, we operated Kubernetes or Public worker pools, testing and hosting the Spacelift platform. Now that the Kubernetes integration is initially available, we thought it would be good to provide a simple way for you to provision a Kubernetes cluster on AWS utilizing Terraform with Spacelift.
This blog will guide you through the process of deploying a Terraform based deployment of an EKS cluster with Spacelift. Below is an example of the cloned Terraform repository.

image2-18

devrel-k8s-worker-cluster

Start by cloning the repository found on <a href="https://github.com/hashicorp/learn-terraform-provision-eks-cluster" target="_blank" rel="nofollow noopener">GitHub</a> and commit to your respective VCS provider. We made a few changes, but that was only to help with the naming and possible confusion in our dev environment. Otherwise, utilizing the repository as-is will work.

In Spacelift, click the Add Stack button to create a Stack.
<h3>Integrate VCS</h3>
<ul>
<li style="font-weight: 400;" aria-level="1">Select the repository you created in the initial step, as seen in the picture.</li>
</ul>

image11-5

Integrate VCS

<h3>Configure Backend</h3>
For this example, I will keep the default selections.

image7-7

Configure Backend

<h3>Define Behavior</h3>
If you need to modify the configuration, select the arrow next to Show Advanced Options. You can find out more about these options by reading the <a href="https://docs.spacelift.io/concepts/stack/creating-a-stack#define-behavior" target="_blank" rel="noopener">documentation</a>.

image4-15

Define Behavior

<h3>Name Stack</h3>
Name the stack and provide any labels or descriptions.

image8-7

Name Stack

<ul>
<li><a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener">Top Most Useful CI/CD Tools for DevOps</a></li>
<li><a href="https://spacelift.io/blog/kubernetes-best-practices" target="_blank" rel="noopener">15 Kubernetes Best Practices to Follow</a></li>
<li><a href="https://spacelift.io/blog/how-specialized-solution-can-improve-your-iac" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;10 Ways to Improve Your Infrastructure as Code&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">10 Ways to Improve Your Infrastructure as Code</a></li>
</ul>

To provision the EKS cluster, this Stack requires integration with AWS. 
Navigate to the Settings, Integrations page and select the dropdown arrow to access the following selection screen:

image1-18

image1

The following links will help you set up the necessary integration with your Cloud Provider of choice.
<ul>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/aws" target="_blank" rel="noopener">AWS</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/azure" target="_blank" rel="noopener">Azure</a></li>
<li style="font-weight: 400;" aria-level="1"><a href="https://docs.spacelift.io/integrations/cloud-providers/gcp" target="_blank" rel="noopener">GCP</a></li>
</ul>
To simplify things, I decided to utilize an AWS role that already exists with the necessary permissions to provide the essential resources associated with this Terraform example.

image9-4

To Trigger a Run, select Trigger on the right side of the Stacks view.

image3-16

Trigger a Run

<h3>Trigger Run Status</h3>
<h4>Planning</h4>
After you manually trigger the Run in the Stack view, Spacelift will deploy a runner image, initialize the Cloud Provider, and run a Terraform Plan based upon the contents of the Terraform code.
After a successful planning phase, you can check the log to see the planned changes.

image5-12

Trigger Run Status - Planning

To confirm the Triggered run, click the CONFIRM button at the top of the screen.
<h4>Finished Deployment</h4>
The following screen highlights the Finished Run and output from a successful deployment to your Kubernetes cluster.

image6-7

Trigger Run Status - Finished Deployment

To review the outputs associated with this EKS cluster deployment, navigate to the outputs page. You can find all pertinent information about this cluster.

image12-4

Outputs

You have completed the deployment of an EKS cluster via Terraform with Spacelift!

Get started on your journey by signing up for a <a href="https://spacelift.io/free-trial" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin around the world! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift effectively manages Terraform state, more complex workflows, supports policy as code, programmatic configuration, context sharing, drift detection, resource visualization and includes many more features.

How to Deploy your Infrastructure in CI/CD using Terraform

Managing Azure Resources using Spacelift

Running Spacelift CI/CD workers in Kubernetes using DinD

Now that the Kubernetes integration is initially available learn how to complete the deployment of an EKS cluster via Terraform with Spacelift!

70-kubernetes-cluster-with-terraform-and-spacelift

At its most basic form, <a href="https://spacelift.io/blog/infrastructure-as-code" target="_blank" rel="noopener">Infrastructure as Code (IaC)</a> is the provisioning and lifecycle management of infrastructure through code instead of manual processes. Over time, the code part has evolved, and each language is either imperative or declarative. The IaC tooling landscape has matured over the years. We have plenty of options to choose from; open-source and commercial. Over the past several years, a majority of the infrastructure related developments is summed up in a single word or ecosystem… Kubernetes!
Kubernetes is nothing new to Spacelift. The platform utilizes Kubernetes for infrastructure, our Public workers run on Kubernetes, and Spacelift support’s our customer&#8217;s deployment of <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">Private workers</a> on Kubernetes via <a href="https://helm.sh/" target="_blank" rel="nofollow noopener">Helm</a>.

image1-17

Kubernetes (Beta)

Kustomize drives Kubernetes support in Spacelift with native support in kubectl. The screenshot below highlights the ability to run kubectl from a Task within a given Stack.

image4-14

Tasks

The team is very excited to see what our customers and community decide to do with the newest integration. The following screenshot is a sample of the output from Spacelift during a Kubernetes Planning phase.

image2-17

Kubernetes Planning phase

<ul>
<li><a href="https://spacelift.io/blog/infrastructure-problems-that-spacelift-solves" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Infrastructure Problems that Spacelift Solves&quot;}" data-sheets-userformat="{&quot;2&quot;:4224,&quot;10&quot;:2,&quot;15&quot;:&quot;Arial&quot;}">Infrastructure Problems that Spacelift Solves</a></li>
<li><a href="https://spacelift.io/blog/why-devops-engineers-recommend-spacelift" target="_blank" rel="noopener">Why DevOps Engineers Recommend Spacelift</a></li>
<li><a href="https://spacelift.io/blog/spacelift-worker-pools-on-kubernetes" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;Spacelift Worker Pools on Kubernetes with Spacelift&quot;}" data-sheets-userformat="{&quot;2&quot;:4224,&quot;10&quot;:2,&quot;15&quot;:&quot;Arial&quot;}">Spacelift Worker Pools on Kubernetes with Spacelift</a></li>
</ul>

Get started on your journey by signing up for a <a href="https://spacelift.io" target="_blank" rel="noopener">Free Trial</a> and taking it for a spin worldwide! Sign-up is as easy as authenticating with the provider of your choice.

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

Achieving the Atlantis Pull Request Automation with Spacelift

How Spacelift Can Improve Your Infrastructure as Code

Kubernetes is nothing new to Spacelift, but the team is very excited to see what our customers and community decide to do with the newest integration.

63-kubernetes-integration

Running Spacelift CICD Workers in Kubernetes Using DinD

Check out the Spacelift Engineering blog to discover internal technical architecture posts written by Spacelift's very own engineers. 

Engineering | Spacelift Blog

Engineering

Adam Connelly

For a while now at Spacelift we’ve been hearing from users that they’d like to be able to run <a href="https://docs.spacelift.io/concepts/worker-pools" target="_blank" rel="noopener">Spacelift worker pools</a> in their Kubernetes clusters. As a first step towards that we decided to investigate whether we could run workers in Kubernetes using a Docker-in-Docker sidecar container. The rest of this post gives a rough overview of the architecture of Spacelift workers, and also explains how the sidecar container strategy works.
I’d just like to say thanks to <a href="https://medium.com/@v.m_hs" target="_blank" rel="nofollow noopener">Vadym Martsynovskyy</a> for his great article about running <a href="https://medium.com/hootsuite-engineering/building-docker-images-inside-kubernetes-42c6af855f25" target="_blank" rel="nofollow noopener">Jenkins agents in Kubernetes</a>. In that article he outlines the general approach taken here, and also describes some of the pros and cons of using Docker-in-Docker. It’s definitely well worth a read.

Before going into more detail, it’s worth explaining what a Spacelift worker is, and giving a quick overview of our architecture. At Spacelift, we provide a hybrid-SaaS model like many other CI/CD systems. What this means in practice is that the control plane that handles scheduling runs is managed by us, but the execution of runs happens in a separate process called a worker. We provide a public worker pool, but also allow you to self-host your own workers, providing complete control over your infrastructure.
Perhaps confusingly, if you follow the instructions to create a private worker pool, you’ll notice that it involves downloading something called the “launcher”. The launcher is responsible for communicating with the Spacelift mothership (control plane), and creating and destroying worker containers using Docker for each Spacelift run:

image2-5

Spacelift mothership (control plane)

As you can see, the launcher process needs access to a Docker daemon in order to create a container for each run.

<ul>
<li><a href="https://spacelift.io/blog/ci-cd-tools" target="_blank" rel="noopener">Top Most Useful CI/CD Tools for DevOps</a></li>
<li><a href="https://spacelift.io/blog/terraform-in-ci-cd" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;How to Deploy your Infrastructure in CI/CD using Terraform&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">How to Deploy your Infrastructure in CI/CD using Terraform</a></li>
<li><a href="https://spacelift.io/blog/devops-best-practices" target="_blank" rel="noopener">16 DevOps Best Practices to Follow</a></li>
</ul>

We found that the launcher architecture maps very nicely to Kubernetes using a sidecar container to run the Docker daemon. For each worker in the pool, we need to create a Kubernetes Pod with two containers: one for the launcher; and another for Docker. The following shows a stripped down Deployment definition for creating a worker pool with 4 workers:

The launcher communicates with the Docker-in-Docker sidecar container via TCP, which is configured via the <code>DOCKER_HOST</code> environment variable for the launcher container. This works because the containers in a Kubernetes Pod share the same <a href="https://kubernetes.io/docs/concepts/workloads/pods/#pod-networking" target="_blank" rel="nofollow noopener">IP address and port space, and can communicate with each other via localhost</a>.
In addition, a shared volume called <code>launcher-storage</code> is mounted into each container. This is used by the launcher to store the workspaces for runs, along with other things like cached tool binaries (for example Terraform). The Docker sidecar needs access to the run workspaces in order to mount them into the worker containers, and it also uses that volume to store its image cache.
The last thing worth pointing out is that the <code>dind</code> container sets <code>securityContext.privileged</code> to <code>true</code>. This is required for Docker-in-Docker to function correctly.

The Spacelift launcher is distributed as a statically linked binary, making it simple to build an image. As you can see, the <a href="https://spacelift.io/blog/dockerfile" target="_blank" rel="noopener">Dockerfile</a> simply copies the launcher into an Alpine container, and then sets the startup command:

This image is rebuilt and published to our public ECR repository any time the launcher binary is updated.

We provide <a href="https://spacelift.io/blog/what-are-terraform-modules-and-how-do-they-work" target="_blank" rel="noopener">Terraform modules</a> to make it really easy for users to deploy worker pools to AWS, Azure and GCP, and we wanted to provide a similar experience for Kubernetes. To achieve this, we created a Helm chart that makes it simple to deploy workers to Kubernetes clusters: <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">https://github.com/spacelift-io/spacelift-workerpool-k8s</a>.
Assuming you’ve already configured your worker pool in Spacelift and have access to the credentials for the pool, deploying this chart to your cluster simply requires two steps.
First, add the Spacelift Helm chart repository and update your local chart cache:

Next, install the chart:

Replace <code>&lt;worker-pool-token&gt;</code> and <code>&lt;worker-pool-private-key&gt;</code> with your own credentials, and make sure to base64-encode the private key.
If all goes well, you should be able to view the pods for your worker pool using <code>kubectl get pods</code>:

You should also be able to view the workers in your pool in Spacelift:

image1-5

Bitbucket DC Local Pool

When implementing in a production environment, you may want to use an alternative approach to providing credentials, and may want to configure a custom storage volume for the worker Pods. For more information about configuring the chart, check out the README in the <a href="https://github.com/spacelift-io/spacelift-workerpool-k8s" target="_blank" rel="nofollow noopener">chart GitHub repo</a>.

If you’re interested in running <a href="https://spacelift.io/blog/spacelift-worker-pools-on-kubernetes" target="_blank" rel="noopener">Spacelift workers in Kubernetes</a>, we’d welcome any feedback about the approach, contributions to the Helm chart, and also any issues you encounter so that we can make improvements. Also, if you aren’t already using Spacelift but are interested in trying it, you can signup and <a href="https://spacelift.io/free-trial" target="_blank" rel="noopener">start your free evaluation of Spacelift</a>.

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities for infrastructure management.

This article is an overview of the architecture of Spacelift workers with an explanation of how the sidecar container strategy works.

Kubectl Cheat Sheet – 15 Kubernetes Commands & Objects

Get the newsletter