Jobs and CronJobs¶
Status: AI draft, 2026-11-16. Not yet verified by me. Commands and output were run on a local kind cluster (Kubernetes 1.37); generated IDs are replaced with placeholders.
In one line¶
A Job runs pods until a set number complete successfully, retrying failures up to backoffLimit and stopping at activeDeadlineSeconds; a CronJob creates a Job on a cron schedule, in a time zone you choose, with a concurrencyPolicy for overlapping runs. Make the work idempotent, because retries and duplicate runs will happen.
The plain idea¶
Most workloads run forever: a web server should never "finish". Some work should run, finish and stop: a database migration, a nightly report, a batch of images to resize. If a Deployment ran it, Kubernetes would restart it after every successful finish. A Job is the controller for finite work: it runs pods until enough of them succeed, retries the ones that fail (up to a limit), and then stops.
A CronJob is the cluster's version of cron (Cron and timers): at each scheduled time it creates a new Job. The same three questions apply as on a server: which time zone, what if a run is missed, and what if a run is still going when the next one is due.
How it works¶
Job¶
flowchart TD
J["Job: completions 4, parallelism 2, backoffLimit 6"] --> P1["Pod 1"]
J --> P2["Pod 2"]
P1 -->|"exit 0"| C["Count a success"]
P2 -->|"exit non-zero"| F["Count a failure,<br/>start a replacement after a back-off"]
C --> D{"4 successes?"}
D -->|"yes"| X["Job Complete"]
F --> L{"Failures above backoffLimit?"}
L -->|"yes"| Y["Job Failed: BackoffLimitExceeded"]
| Field | Meaning | Default |
|---|---|---|
completions |
How many pods must succeed | 1 |
parallelism |
How many pods may run at once | 1 |
backoffLimit |
How many retries before the Job is marked failed | 6 |
activeDeadlineSeconds |
Fail the Job after this long, whatever the retries | None |
ttlSecondsAfterFinished |
Delete the finished Job and its pods after this long | Keep forever |
template.spec.restartPolicy |
Never (a failure creates a new pod) or OnFailure (the container restarts in the same pod); Always is not allowed |
Must be set |
Retries back off exponentially. With restartPolicy: OnFailure, the failed attempts' logs are lost when the container restarts in place; Never keeps each failed pod for inspection. An indexed Job (completionMode: Indexed) gives each pod an index, so pod 3 of 10 knows to process its own slice of the work.
CronJob¶
| Field | Meaning |
|---|---|
schedule |
Five-field cron expression |
timeZone |
An IANA zone name such as Asia/Kolkata; without it, the schedule uses the controller manager's time zone, usually UTC |
concurrencyPolicy |
Allow (overlapping runs), Forbid (skip the new run if the last one is still active), or Replace (stop the old run, start the new) |
startingDeadlineSeconds |
If a run could not start on time (controller down), start it late only within this window |
successfulJobsHistoryLimit, failedJobsHistoryLimit |
How many finished Jobs to keep (3 and 1 by default) |
suspend |
true pauses future runs without deleting the CronJob |
A CronJob can occasionally create two Jobs for one scheduled time, or none, around controller restarts and clock issues. The documentation is explicit that jobs should be idempotent: running twice must do no harm.
A concrete example¶
A nightly report must run at 02:30 India time, never overlap, and keep a little history.
apiVersion: batch/v1
kind: CronJob
metadata:
name: nightly-report
namespace: demo
spec:
schedule: "30 2 * * *"
timeZone: "Asia/Kolkata"
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 1
jobTemplate:
spec:
template:
spec:
restartPolicy: OnFailure
containers:
- name: report
image: busybox:1.36
command: ["sh", "-c", "echo report generated"]
timeZoneremoves the UTC arithmetic: the schedule means 02:30 in India.Forbidskips a run if yesterday's is somehow still going, instead of two copies writing the same report.- To test it now, rather than waiting until 02:30, create a Job from its template:
kubectl create job manual-report --from=cronjob/nightly-report. - The report writes to a path that includes the date and overwrites it if run twice, so a duplicate run is harmless.
- An alert fires if no successful Job has completed in 26 hours, so a silently skipped run is noticed (Alerting and Alertmanager).
Why not just...?¶
- ...run batch work as a Deployment? When the process finishes, the Deployment restarts it, forever. A Job knows "done" from "crashed".
- ...use
restartPolicy: OnFailureeverywhere? It restarts in the same pod, so you lose the logs of earlier failed attempts, and a pod stuck on a bad node keeps failing there.Neverwith a sensiblebackoffLimitis easier to debug. - ...set no
backoffLimitor deadline? A Job that can never succeed (wrong credentials, missing table) keeps retrying for a long time and hides the failure. Set limits so it fails visibly, and alert on failed Jobs.
Commands to know¶
A parallel Job and a failing one:
$ kubectl get pods -n demo -l job-name=batch-ok
batch-ok-<id> Running
batch-ok-<id> Running
$ kubectl wait --for=condition=Complete job/batch-ok -n demo --timeout=60s
job.batch/batch-ok condition met
$ kubectl get job batch-ok -n demo
NAME STATUS COMPLETIONS DURATION AGE
batch-ok Complete 4/4 13s 13s
Two pods ran at a time (parallelism: 2) until four had succeeded.
$ kubectl get job batch-fail -n demo
NAME STATUS COMPLETIONS DURATION AGE
batch-fail Failed 0/1 53s 53s
$ kubectl get pods -n demo -l job-name=batch-fail --no-headers | awk '{print $3}' | sort | uniq -c
3 Error
$ kubectl get job batch-fail -n demo -o jsonpath='{range .status.conditions[*]}{.type}={.status} reason={.reason}{"\n"}{end}'
FailureTarget=True reason=BackoffLimitExceeded
Failed=True reason=BackoffLimitExceeded
With backoffLimit: 2, the first attempt plus two retries made three failed pods, each kept for its logs because the policy was Never.
A CronJob, and running it on demand:
$ kubectl get cronjob nightly-report -n demo
NAME SCHEDULE TIMEZONE SUSPEND ACTIVE LAST SCHEDULE AGE
nightly-report 30 2 * * * Asia/Kolkata False 0 <none> 53s
$ kubectl create job manual-report --from=cronjob/nightly-report -n demo
job.batch/manual-report created
$ kubectl logs -n demo job/manual-report
report generated
| Command | What it does |
|---|---|
kubectl get jobs -n <ns> |
Completions, status and duration. |
kubectl logs job/<name> |
Logs of one of the Job's pods. |
kubectl describe job <name> |
Events: pods created, failures, deadline. |
kubectl patch cronjob <name> -p '{"spec":{"suspend":true}}' |
Pause a CronJob. |
kubectl get jobs -n <ns> --sort-by=.status.startTime |
When each run started, to check a schedule. |
kubectl delete job <name> |
Delete a Job and, by default, its pods (Finalizers and deletion). |
How it breaks¶
| Symptom | Layer | Check | Likely cause |
|---|---|---|---|
Job Failed, BackoffLimitExceeded |
Application | Logs of the failed pods | The work fails every time: credentials, a missing dependency, a bug |
Job Failed, DeadlineExceeded |
Deadline | activeDeadlineSeconds against real duration |
Work takes longer than allowed, or hangs |
| Job never finishes | Pod or sidecar | Pods' states | A process that never exits, or a regular sidecar container that keeps running (Pods) |
| CronJob ran at the wrong hour | Time zone | timeZone field |
Schedule interpreted in UTC |
| CronJob run skipped | Concurrency or deadline | concurrencyPolicy, startingDeadlineSeconds, events |
Previous run still active with Forbid, or the controller missed the window |
| Thousands of old completed pods | Clean-up | History limits, TTL | No ttlSecondsAfterFinished, or Jobs created outside CronJobs with no clean-up |
| The same record processed twice | Idempotency | Job design | A retried or duplicate run repeated work that was not idempotent |
Interview questions¶
1. When do you use a Job instead of a Deployment?
Answer
- For finite work that should run to completion and stop: migrations, batch processing, reports.
- A Deployment restarts processes that exit, even successfully.
- A Job tracks successful completions and retries failures up to a limit.
2. What do completions, parallelism and backoffLimit do?
Answer
completions: how many pods must succeed.parallelism: how many run at the same time.backoffLimit: how many retries before the Job is marked failed.
3. What are the CronJob concurrency policies?
Answer
Allow: runs may overlap.Forbid: skip a new run while the previous one is active.Replace: stop the active run and start the new one.
4. Why must CronJob work be idempotent?
Answer
- Retries rerun failed work, and the controller can occasionally create two Jobs for one schedule (or none).
- Running twice must not double-charge, double-send or corrupt data.
- Use idempotency keys, upserts, or outputs that are overwritten rather than appended.
5. A CronJob scheduled at 02:30 ran at 08:00. Why?
Answer
- The schedule was evaluated in UTC (or the controller's zone), and 02:30 UTC is 08:00 in India.
- Set
timeZone: Asia/Kolkata, or convert the schedule to UTC.
6. How do you test a CronJob without waiting for its schedule?
Answer
kubectl create job <name> --from=cronjob/<cronjob>creates a Job from the CronJob's template immediately.- Watch it with
kubectl get jobandkubectl logs job/<name>.
7. A Job with a mesh sidecar never completes. Why, and how do you fix it?
Answer
- The main container finished, but the sidecar, as a regular container, keeps running, so the pod never completes.
- Make it a native sidecar (an init container with
restartPolicy: Always), which is stopped after the main containers finish.
Try it¶
Step 1. On a kind cluster, run a Job that always fails, with backoffLimit: 2. Predict how many pods it creates and the Job's final status, then watch.
kubectl apply -f - <<'EOF'
apiVersion: batch/v1
kind: Job
metadata: { name: always-fails }
spec:
backoffLimit: 2
template:
spec:
restartPolicy: Never
containers:
- name: w
image: busybox:1.36
command: ["sh", "-c", "echo trying; exit 1"]
EOF
kubectl get pods -l job-name=always-fails -w
Press Ctrl+C after a minute, then run kubectl get job always-fails.
Step 2. Create a CronJob that runs every minute and trigger it manually as well.
kubectl create cronjob tick --image=busybox:1.36 --schedule="* * * * *" -- sh -c 'date'
kubectl create job tick-now --from=cronjob/tick
kubectl get jobs -w
Step 3. Clean up.
kubectl delete job always-fails tick-now && kubectl delete cronjob tick
Zoom out¶
Jobs and CronJobs complete the workload controllers (Deployments and rollouts, StatefulSets and DaemonSets), and carry the same scheduling questions as cron on a server. In CI/CD they run migrations and test suites; Helm and Argo CD hooks are often Jobs (Releases, hooks and debugging). The principle to carry: finite work needs limits, a time zone, a concurrency rule, and idempotency.