A pod is the smallest unit in Kubernetes (K8S). They should run until they are replaced by a new deployment. Because of this, there is no way to restart a pod, instead, it should be replaced.
There is no
kubectl restart [podname] command for use with K8S (with Docker you can use
docker restart [container_id] ), so there are a few different ways to achieve a pod ‘restart’ with
You may need to ‘restart’ a pod in several situations, commonly when the pod status is stuck in a ‘terminating’ state, or when a pod is erroring and that error cannot be fixed without a restart. In certain situations, you may need to restart a pod after modifying the configuration, such as amending the resource limits to restrict a pod’s memory limits, or changing the persistent volume storage a pod is using.
A pod has five possible statuses:
- Pending: This state shows at least one container within the pod has not yet been created.
- Running: All containers have been created, and the pod has been bound to a Node. At this point, the containers are running, or are being started or restarted.
- Succeeded: All containers in the pod have been successfully terminated and will not be restarted.
- Failed: All containers have been terminated, and at least one container has failed. The failed container exists in a non-zero state.
- Unknown: The status of the pod cannot be obtained.
If you notice a pod in an undesirable state where the status is showing as ‘error’, you might try a ‘restart’ as part of your troubleshooting to get things back to normal operations. You may also see the status ‘CrashLoopBackOff’, which the default when an error is encountered, and K8S trys to restart the pod automatically.
You can use the following methods to ‘restart’ a pod with
kubectl. Once new pods are re-created they will have a different name than the old ones. A list of pods can be obtained using the
kubectl get pods command.
kubectl rollout restart
This method is the recommended first port of call as it will not introduce downtime as pods will be functioning. A rollout restart will kill one pod at a time, then new pods will be scaled up. This method can be used as of K8S v1.15.
kubectl rollout restart deployment <deployment_name> -n <namespace>
This method will introduce an outage and is not recommended. If downtime is not an issue, this method can be used as it can be a quicker alternative to the
kubectl rollout restart method (your pod may have to run through a lengthy Continuous Integration / Continuous Deployment Process before it is redeployed).
If there is no YAML file associated with the deployment, you can set the number of replicas to 0.
kubectl scale deployment <deployment name> -n <namespace> --replicas=0
This terminates the pods. Once scaling is complete the replicas can be scaled back up as needed (to at least 1):
kubectl scale deployment <deployment name> -n <namespace> --replicas=3
Pod status can be checked during the scaling using:
kubectl get pods -n <namespace>
kubectl delete pod and kubectl delete replicaset
Each pod can be deleted individually if required:
kubectl delete pod <pod_name> -n <namespace>
Doing this will cause the pod to be recreated because K8S is declarative, it will create a new pod based on the specified configuration.
However, where lots of pods are running this is not really a practical approach. Where lots of pods have the same label however, you could use that to select multiple pods at once:
kubectl delete pod -l “app:myapp” -n <namespace>
Another approach if there are lots of pods, then ReplicaSet can be deleted instead:
kubectl delete replicaset <name> -n <namespace>
kubectl get pod | kubectl replace
The pod to be replaced can be retrieved using the
kubectl get pod to get the YAML statement of the currently running pod and pass it to the
kubectl replace command with the
--force flag specified in order to achieve a restart. This is useful if there is no YAML file available and the pod is started.
kubectl get pod <pod_name> -n <namespace> -o yaml | kubectl replace --force -f -
kubectl set env
Setting or changing an environment variable associated with the pod will cause it to restart to take the change. The example below sets the environment variable
DEPLOY_DATE to the date specified, causing the pod to restart.
kubectl set env deployment <deployment name> -n <namespace> DEPLOY_DATE="$(date)"
Having a range of commands to use in your toolbox when running into issues with pods in K8S will enable you to use the appropriate method to restart them, depending on how you have deployed the pods, the necessity for application uptime, and the speed you need the restart to happen. In general, the best approach is to use the
kubectl rollout restart method described above as it will avoid application downtime.
A restart will not resolve the problem that caused the pods to have issues in the first place, so further investigation will be required into the root cause.
If you need more help with Kubernetes, Spacelift helps you manage the complexities and compliance challenges of using Kubernetes. Anything that can be run via kubectl can be run within a Spacelift stack. It brings with it a GitOps flow, so your Kubernetes Deployments are synced with your Kubernetes Stacks, and pull requests show you a preview of what they’re planning to change. Find out more about how Spacelift works with Kubernetes.
Continuous Integration and Deployment for your IaC
Spacelift allows you to automate, audit, secure, and continuously deliver your infrastructure. It helps overcome common state management issues and adds several must-have features for infrastructure management.