Monitor rsync with a dead man's switch

1. Create a check

Create a check in Mortemain with your job's schedule and grace window, and copy its unique ping URL. It looks like this (the UUID is the only secret, keep it unguessable):

https://ping.mortemain.com/your-check-uuid

2. Wrap rsync and check its exit code

Run rsync from a small script, capture its exit code, and ping the right endpoint depending on the result. Skip set -e here, you need the exit code in hand before deciding what to do with it:

#!/usr/bin/env bash
set -uo pipefail   # no -e: we need rsync's own exit code below
URL=https://ping.mortemain.com/your-check-uuid

rsync -az --delete /data/ backup-host:/backups/data/
rc=$?

# 24 = some source files vanished mid-transfer, usually fine on a live filesystem
if [ "$rc" -eq 0 ] || [ "$rc" -eq 24 ]; then
  curl -fsS -m 10 "$URL"
else
  curl -fsS -m 10 "$URL/fail"
  exit "$rc"
fi

rsync's exit code 24 just means some source files disappeared while it was reading them, common when backing up a filesystem that's still being written to. Treat it as a pass; any other non-zero code is a real failure worth paging on.

3. Add the cron line

Point cron at the script. Since the script reports its own result, the cron line stays simple:

# nightly at 02:00; rsync-backup.sh pings Mortemain itself
0 2 * * *  /usr/local/bin/rsync-backup.sh

Prefer a one-liner with no wrapper script? Chain it, just note this treats exit code 24 as a failure too:

# any non-zero rsync exit, including 24, counts as failure here
0 2 * * *  rsync -az --delete /data/ backup-host:/backups/data/ && curl -fsS -m 10 https://ping.mortemain.com/your-check-uuid || curl -fsS -m 10 https://ping.mortemain.com/your-check-uuid/fail

4. Track how long it takes (optional)

Ping /start right before rsync begins and Mortemain will record each run's duration, so a sync that suddenly takes 5× longer shows up as a warning instead of a surprise.

curl -fsS -m 10 "$URL/start"
rsync -az --delete /data/ backup-host:/backups/data/
rc=$?

Test it

Run the script by hand once. The check should flip to up within a second or two. Then point it at a destination that doesn't exist and run it again: you should get the failure alert immediately, instead of waiting for the window to lapse.