How to Get Alerts When a Cron Job Fails: Stop Silent Failures
You wake up. Coffee. Check your phone. Nothing seems broken. But underneath, one of your nightly cron jobs — the one that syncs customer data, cleans up expired sessions, or sends out invoices — failed silently three days ago. Nobody noticed. No alerts fired. No panic. Just a slow, quiet accumulation of technical debt and angry users waiting to happen.
Getting cron job alerts when something goes wrong isn't just a nice-to-have. It's the difference between catching a bug at 2 AM with a quick fix and finding out at 2 PM on Monday when half your database is corrupted.
This guide walks you through why cron jobs fail silently, how to detect those failures in real time, and the simplest way to set up alerts that actually work. No fluff. No enterprise monitoring suites. Just practical steps you can implement today.
The Problem
Cron jobs are everywhere. Every developer has them. Backup scripts. Data processing pipelines. Email digests. Cache warmers. They run on a schedule, do their thing, and (hopefully) finish cleanly.
But here's the thing: cron itself doesn't care if your script fails. It fires off the command, waits for the process to exit, and moves on. If your script crashes with a non-zero exit code, cron doesn't retry. It doesn't send you an email. It doesn't page you. It just... stops.
The job might fail because:
- A dependency updated and broke your script
- The database was unreachable for 30 seconds
- Disk space ran out
- An API rate limit kicked in
- The server restarted mid-execution
And because there's no built-in alerting, the failure goes unnoticed until someone manually checks logs or a downstream system breaks. By then, it's often too late.
Why It Happens
Cron is a scheduler, not a monitor. Its only job is to execute commands at specified intervals. That's it.
When you write 0 2 * * * /usr/local/bin/backup.sh, cron will:
- Wake up at 2:00 AM
- Execute
backup.sh - Wait for it to finish
- Log the exit code (if you've configured logging)
- Go back to sleep
If backup.sh exits with code 1 (error), cron doesn't interpret that as "something went wrong, alert the human." It just records the exit and waits for the next scheduled run.
Most developers assume their cron jobs work because they usually work. They test once, deploy, and forget. Until one day, it doesn't work. And nobody knows.
Why It's Dangerous
Silent cron job failures create a false sense of security. Here's what actually happens when a critical job fails unnoticed:
Data loss. Your backup script failed last night. You don't find out until the server crashes three weeks later and there's nothing to restore from.
Stale data. Your data sync job hasn't run in five days. Your dashboard shows incorrect metrics. Your customers see wrong numbers. Your CEO asks questions you can't answer.
Cascading failures. One failed job blocks another. The cleanup script didn't run, so disk space fills up. Then the logging service crashes. Then the whole system goes down.
Revenue impact. Your invoicing job failed. Customers weren't billed. Churn goes up. Cash flow goes down. You find out during your monthly review.
The common thread? You didn't know until it was too late.
How to Detect It
The key insight is simple: instead of checking whether a cron job failed, check whether it succeeded.
This is the heartbeat pattern. Your cron job sends a signal (a "heartbeat") to a monitoring service when it completes successfully. If the monitoring service doesn't receive a heartbeat within the expected window, it knows something went wrong and alerts you.
Think of it like a dead man's switch. As long as the signal keeps coming, everything is fine. When the signal stops, someone gets notified.
This approach has several advantages:
- It detects missing runs, not just failed ones. If cron itself crashes or the server goes down, you still get alerted.
- It's simple. Your script only needs to make one HTTP request at the end.
- It's language-agnostic. Bash, Python, Node.js, Ruby — doesn't matter. Just curl a URL.
Simple Solution (with Example)
Here's how you set up heartbeat monitoring for a cron job in under two minutes.
Let's say you have a backup script:
#!/bin/bash
# /usr/local/bin/backup.sh
pg_dump mydb > /backups/db-$(date +%F).sql
if [ $? -ne 0 ]; then
echo "Backup failed" >&2
exit 1
fi
echo "Backup complete"
Right now, if this fails, nothing happens. Let's add a heartbeat:
#!/bin/bash
# /usr/local/bin/backup.sh
pg_dump mydb > /backups/db-$(date +%F).sql
if [ $? -ne 0 ]; then
echo "Backup failed" >&2
exit 1
fi
# Send heartbeat
curl -fsS --retry 3 https://quietpulse.xyz/ping/YOUR-CRON-ID > /dev/null
echo "Backup complete"
That's it. The curl command sends a GET request to a monitoring endpoint. The flags mean:
-f: Fail silently on HTTP errors (non-2xx responses)-s: Silent mode (no progress meter)-S: Show errors even in silent mode--retry 3: Retry up to 3 times if the request fails
Now, when the backup script completes successfully, it pings the monitoring service. If the service doesn't receive a ping within the expected time window (say, every 24 hours), it sends you an alert via email, Slack, Telegram, or webhook.
Setting up the monitor itself is straightforward. With a tool like QuietPulse, you create a monitor, give it a name ("Database Backup"), set the expected interval (daily), and configure your alert channels. The service gives you a unique ping URL. You drop that URL into your script. Done.
Instead of building this logic yourself, you can use a simple heartbeat monitoring tool like QuietPulse. It handles the ping tracking, alert routing, and escalation so you don't have to maintain another service.
Common Mistakes
Here are the most frequent mistakes developers make when setting up cron job monitoring:
1. Pinging at the start instead of the end. If you send the heartbeat before your job runs, a successful ping tells you nothing. The job could crash immediately after. Always ping after the critical work is done.
2. Not checking the exit code before pinging. Your script should only send the heartbeat if it actually succeeded. If you ping unconditionally, you're lying to your monitoring service.
3. Setting the timeout window too short. If your job usually takes 5 minutes, don't set the alert threshold to 6 minutes. Network hiccups, slow APIs, and database locks happen. Give yourself a buffer — 2x or 3x the normal runtime is a good starting point.
4. Ignoring flapping. If your job succeeds 90% of the time and fails 10%, you'll get constant alerts. Either fix the root cause or adjust your monitoring to alert on consecutive failures, not single misses.
5. Monitoring too many things with one endpoint. Each cron job should have its own unique ping URL. If you reuse the same endpoint for multiple jobs, you won't know which job failed.
Alternative Approaches
Heartbeat monitoring is the simplest and most reliable approach, but it's not the only one. Here are other ways people track cron job health:
Log parsing. Parse system logs (/var/log/syslog or /var/log/cron) for non-zero exit codes. Tools like logwatch or custom scripts can scan logs and send alerts. The downside? You have to manage log rotation, parsing logic, and alerting infrastructure yourself.
Email output. Cron can email you the output of every job by setting MAILTO=you@example.com in your crontab. This works for small setups, but it doesn't scale. You'll drown in emails, miss important ones, and have no way to track trends.
Uptime monitoring. Some teams wrap cron jobs in HTTP endpoints and monitor them with uptime checkers like UptimeRobot or Pingdom. This adds complexity (you need a web server) and doesn't distinguish between "job didn't run" and "job ran but failed."
Centralized logging. Send job output to a service like Datadog, ELK, or Papertrail. Set up alerts on error patterns. This is powerful but requires significant infrastructure and expertise.
For most developers and small teams, heartbeat monitoring strikes the best balance between simplicity and reliability.
FAQ
What's the difference between exit code monitoring and heartbeat monitoring?
Exit code monitoring checks whether a process returned 0 (success) or non-zero (failure). Heartbeat monitoring checks whether a signal was received within an expected time window. The key difference: heartbeat monitoring also catches cases where the job never ran at all (server down, cron crashed, job deleted). Exit code monitoring only works if the job actually started.
How often should I expect heartbeats?
This depends on your cron schedule. If a job runs daily, expect one heartbeat per day. If it runs every hour, expect 24 heartbeats. Set your monitoring service's grace period to account for normal variance — if a job usually takes 10 minutes, a 30-minute grace period gives room for occasional delays without false alarms.
Can I monitor cron jobs on servers without internet access?
If your server is completely offline, HTTP-based heartbeats won't work. In that case, you can use internal monitoring: write completion markers to a shared database, use a local message queue, or set up an internal webhook endpoint. The principle is the same — signal successful completion — but the transport mechanism changes.
Conclusion
Cron jobs will fail. It's not a matter of if, but when. The question is whether you'll find out before your users do.
Adding heartbeat monitoring to your critical cron jobs takes minutes and saves hours of debugging, data recovery, and apology emails. Ping when the job succeeds. Get alerted when it doesn't. That's the whole game.
Start with your most important jobs — backups, invoicing, data syncs. Add heartbeats. Configure alerts. Sleep better.