All 22 of my uptime monitors were green. Three services had been broken for months.

My lab: a single Raspberry Pi 5 (16 GB) on a 1 TB NVMe over the PCIe HAT, booting from the NVMe, running about 36 containers. Pi-hole in front of a fully recursive Unbound (no upstream resolver at all), Immich for photos, Nextcloud, Vaultwarden, Syncthing, Samba, ioBroker for home automation, Ollama + Open WebUI for local models, and a monitoring stack of Prometheus, Grafana, node_exporter, cAdvisor and Uptime Kuma. Tailscale for remote access, CrowdSec and fail2ban on the host. Backups run in three tiers: local for 7 days, USB for 90, and restic to off-site storage. Not a rack — one small box that the household actually depends on, which is exactly why the following bothered me so much.

I audit it every few weeks. The last audit turned up three services that had been dead for a long time without a single alert:

• Samba — port 445 open, TCP handshake fine, every single login rejected. Broken for about five months. I'd been mounting shares from a cached credential on one machine and never noticed.
• Vaultwarden backups — nine days of .tar.gz files that were valid, fully extractable, and did not contain the database. The backup script hit a permission error partway through, so tar aborted — but it had already written the files it could read. tar -tzf returns 0 for that.
• Portainer — HTTP 200 on the login page, but it had lost its connection to the Docker socket four months earlier. It just showed an empty environment.

All 22 of my uptime monitors were green the entire time.

None of these was an outage. Every one of them would have been a disaster on the day it actually mattered.

The monitors weren't wrong. They were answering a different question than the one I cared about. "Is port 445 open" and "can I log in" are not the same question. "Is the archive readable" and "does the archive contain my data" are very much not the same question.

So I wrote a second layer of checks that ask the other question, and put the templates on GitHub in case they're useful to anyone else:

github.com/DanielEnki420/correctness-checks

The idea behind them is simple enough: do the real work the service exists for, then look at what came back. Log in, don't check whether the port accepts a connection. Resolve a name and read the answer. List the archive and look for a specific file inside it. Health endpoints are the worst offenders here — plenty of them return a hardcoded {"ok":true} that stays true long after the database behind them is gone.

The part I see skipped most often is testing the negative case. A DNSSEC validator that cheerfully accepts a knowingly broken signature will pass any positive-only test you write, while protecting nothing at all. So my DNSSEC check queries dnssec-failed.org and fails if it doesn't get SERVFAIL back. Same thing in the blocking check: it confirms the blocklist blocks, then confirms a control domain still resolves — otherwise a resolver that blocked everything would pass the first half.

The one I'd argue hardest for: if you can't verify something, that's a failure, not a skip. If the SMART output changes format and a field disappears, the check fails and says so. Tempting to wave those through — it's not really an error, the disk is probably fine — but that's precisely the habit that gave me nine days of empty backups. A check that quietly gives up while reporting green is worse than no check, because now you trust it.

And one thing that's free: push on success too, not just on failure. A push monitor goes red when heartbeats stop arriving, so a check that reports its successes doubles as a dead man's switch. If it gets dropped from cron, or the box running it dies, you hear about it.

Six templates in there: backup archive contents, NVMe SMART, DNSSEC both directions, DNS blocking (with a counter-test for over-blocking), a generic JSON field assertion, and a skeleton to write your own. Plus a test suite whose pushes go to a stub, never to a real monitor — I learned that one the hard way by copying a check for testing and leaving the real push token in it. The simulated failure went straight to production and turned the monitor red.

How you'd actually run it: plain bash calling docker exec, curl, tar, nvme, smbclient. No Docker image and no compose file, deliberately — there's no daemon here, just scripts you run from cron on a host that can already reach your services. Clone it, copy config.example.env, edit paths and container names, add a cron line. Written against Uptime Kuma push monitors; any push-based monitor should work, but Kuma is the only one I've actually tested.

Honest limitations:

• It's bash. That's deliberate — no runtime to install, easy to read, easy to adapt. If you want a framework, this isn't it.
• The templates are examples, not a product. You will have to adapt paths, container names, and thresholds. That's the point.
• No installer, no packaging.
• Only tested on Debian/Raspberry Pi OS against my own services.

AI disclosure (this post is flaired Mostly AI Generated): AI involvement here was substantial and I'd rather be specific than vague about it. The three failures above are real and from my own machine — they turned up during a routine audit of my own lab, not from a tutorial or a hypothetical. The scripts were then written together with Claude: I described what each check had to prove, it drafted, and we went back and forth. The README and code comments are largely AI-drafted and edited by me. What I did not do is ship untested output — every check runs against my actual services, and the negative cases were verified by deliberately breaking things (querying dnssec-failed.org, feeding a truncated archive to the backup check) rather than by assuming they'd work.

MIT licensed, no monetization, no affiliate links, nothing to sell.

The thing I'd genuinely like feedback on: what else in a homelab fails silently like this? I found three. I'm fairly sure that's not all of them.

Project posting prompt — answers

• Project Name: correctness-checks
• Repo/Website Link: https://github.com/DanielEnki420/correctness-checks
• Description: Monitoring templates that check whether a service still does its job, not whether it responds. Six bash checks plus a template to write your own, reporting into Uptime Kuma push monitors.
• Deployment: Plain bash from cron on the host. No Docker image, no compose file. Clone, copy config.example.env, edit, add a cron line. README covers setup, each check, and three traps worth knowing.
• AI Involvement: Substantial — see the AI disclosure paragraph above. Failures are real and from my own lab, scripts drafted with Claude and verified against real services including deliberately broken negative cases, README and comments largely AI-drafted and edited by me.