OpenTofu is now part of the Linux Foundation 🎉

Read more here →

Kubernetes

CronJob in Kubernetes – Automating Tasks on a Schedule

Running Cron Jobs in Kubernetes

CronJobs are a way to run a task on a time-based schedule and have been around for a long time in Linux and UNIX systems. They can be used in Kubernetes (K8S) to run recurring tasks such as backup jobs, triggering emails, report generation, or automating restarts of containers.

In this article, we will dive into how to define a CronJob, look at how to implement them in K8S manifest files with an example, and the available options.

How Do CronJobs Work?

Traditionally on Unix-based systems, CronJobs work as follows:

  1. The user creates a cron job by using the crontab command to edit their “crontab” file. This file contains a list of commands or scripts to be executed and the times when they should be executed.
  2. The cron daemon, a background process that runs continuously, reads the crontab files of all users and checks if any jobs are scheduled to run at the current time.
  3. If a job is scheduled to run, the cron daemon executes the command or script associated with the job.

In K8S, CronJobs are automatically managed by the cluster control plane. The cluster creates regular jobs with the pod spec from your CronJob object. CronJobs are a higher-level abstraction than standard K8S jobs that repeat the cycle periodically.

CronJob Syntax

To define a Cronjob, the schedule is defined using the CronJob syntax below:

# ┌───────────── minute (0 - 59)
# │ ┌───────────── hour (0 - 23)
# │ │ ┌───────────── day of the month (1 - 31)
# │ │ │ ┌───────────── month (1 - 12)
# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sun to Sat;
# │ │ │ │ │                      7 is also Sunday on some systems)
# │ │ │ │ │                   OR sun, mon, tue, wed, thu, fri, sat
# │ │ │ │ │
# * * * * *
0 23 * * *

A job running every minute would look like this:

* * * * *

Check out Crontab.guru to experiment with defining CronJobs.

For CronJobs with no time zone specified, the kube-controller-manager interprets schedules relative to its local time zone. As of Kubernetes v1.25 [beta] the CronJobTimeZone feature gate can be enabled, which enables a specific time zone to be specified should it be required. For example:

spec.timeZone: "Etc/UTC"

CronJob Example

apiVersion: batch/v1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "* * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox:1.28
            imagePullPolicy: IfNotPresent
            command:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure

Create the deployment:

kubectl create -f .\cronjob.yaml

Verify the CronJob has been created:

kubectl get cronjob hello
kubectl get jobs --watch

To view the pods that have been created to run the jobs:

kubectl get pods
kubectl get pods

If you have lots of running pods you’ll want to filter the selection to the job name in your system using the --selector argument. Note that only the last three pods will be shown by default unless a different value has been specified by the optional field spec.successfulJobsHistoryLimit.

kubectl get pods --selector=job-name=hello-27827258

To view the logs from the pod to verify the command ran successfully:

kubectl logs hello-27827258--1-rdf4s
kubectl logs hello-27827258--1-rdf4s

To clean up, delete the CronJob. Deleting the CronJob removes all the jobs and pods it created and stops it from creating additional jobs:

kubectl delete cronjob hello

CronJob Spec Options

1. startingDeadlineSeconds

  • Allow (default): The CronJob allows concurrently running jobs.
  • Forbid: The CronJob does not allow concurrent runs; if it is time for a new job run and the previous job run hasn’t finished yet, the CronJob skips the new job run.
  • Replace: If it is time for a new job run and the previous job run hasn’t finished yet, the CronJob replaces the currently running job run with a new job run.

3. suspend

kubectl get cronjob hello
kubectl get cronjob hello

Common Kubernetes CronJobs Errors & Troubleshooting

If you encounter errors when setting up a CronJob, check the following:

  1. Syntax: CronJobs use the same syntax as traditional UNIX cron jobs, which can be complex and difficult to get right. Common errors include specifying the wrong number of fields, using incorrect wildcards, or mistyping the cron schedule. Check the syntax section of the article and copy your expression into crontab.guru to make sure the syntax is correct.
  2. Timezone: CronJobs run in the timezone of the Kubernetes cluster by default, which may not match the timezone of the user or application. This can lead to scheduling conflicts or unexpected behavior.
  3. Image: If a CronJob specifies an image that is not available, the job will fail.
  4. Resources: Resource limits on images that are set too high will cause jobs to fail.
  5. Job concurrency: If a CronJob is set to run too frequently or with too many replicas, it might lead to excessive load on the cluster and cause other jobs to fail.
  6. Permissions: Check the CronJob has sufficient permissions to access resources or perform actions that are defined.

To troubleshoot CronJob errors, the K8S logs will be the first port of call. You can use the kubectl logs command to interrogate the K8S server API logs, CronJob controller logs, Pod logs, and Container logs.

kubectl logs -n <namespace> <cronjob-controller-pod-name> -c cronjob-controller

If you are using a centralized logging solution for your cluster (as is recommended), such as Elasticsearch, Fluentd, or Kibana (EFK), to collect and analyze logs from multiple nodes and containers in the cluster, you should check the logs using those tools for deeper insight. 

Monitoring solutions such as Prometheus, Datadog, Grafana, or New Relic can be used to track job execution, such as the number of successful and failed jobs, job duration, and resource usage.

Key Points

And take a look at how Spacelift helps you manage the complexities and compliance challenges of using Kubernetes. Anything that can be run via kubectl can be run within a Spacelift stack. Find out more about how Spacelift works with Kubernetes.

The Most Flexible CI/CD Automation Tool

Spacelift is an alternative to using homegrown solutions on top of a generic CI. It helps overcome common state management issues and adds several must-have capabilities s for infrastructure management.

Start free trial