← All guides

Guide · Kubernetes

Monitor a Kubernetes CronJob

CronJobs fail quietly in two ways: a Job runs but errors, or the controller stops scheduling them at all (missed deadlines, clock skew, a bad patch). A dead man's switch catches both, because it fires on silence, not on an error event.

1. Create a check

Create a check in Mortemain with the same period as your schedule, and copy its ping URL.

2. Ping when the container succeeds

Chain a curl after your command so it only pings if the job exited cleanly. No image change beyond the command:

 cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-etl
spec:
  schedule: "0 3 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: etl
              image: your/etl:latest
              command: ["/bin/sh", "-c"]
              args:
                - >-
                  run-etl.sh &&
                  curl -fsS -m 10 https://ping.mortemain.com/your-check-uuid

3. No curl in the image?

Use wget if it's there, or run the ping as a second step with a minimal image so your app image stays untouched:

 options
# if wget is available instead of curl
run-etl.sh && wget -qO- https://ping.mortemain.com/your-check-uuid

# or add an init/next container using a tiny image, e.g. curlimages/curl

Test it

kubectl create job --from=cronjob/nightly-etl test-run and watch the check flip to up. Then pause the CronJob and confirm the down alert arrives after the window, that's the failure mode Kubernetes won't tell you about.