Homelab
Two Dell OptiPlex micro PCs running Proxmox. They host the Dexter bots, my DNS, monitoring and backups, and a pile of apps.
node1
OptiPlex 7050 Micro · i7-7700T · 24 GB
node2
OptiPlex 7050 Micro · i7-6700T · 16 GB
gold VM, the rest are containers
What's running
- Network
- AdGuard Home and Unbound for DNS (two of them, kept in sync), Caddy in front of everything, a Cloudflare tunnel, and Authelia so there's one login.
- Watching
- Prometheus, Grafana, Loki and Gatus. Gatus runs 46 checks a minute and anything that breaks pings my phone through ntfy.
- Backups
- Proxmox Backup Server on the NAS every night, with a two-day copy on node2. Once a month a script restores a couple of containers to make sure the backups actually work.
- Apps
- The Dexter bots, Lavalink for the music bots, Jellyfin, Immich, Home Assistant, Forgejo, Paperless, Mealie and a few others.
Things that broke
node2 stuck at 800 MHz
Turns out the BIOS throttles the CPU when it doesn't recognize the power brick. I swapped bricks between the two nodes and the problem moved with it. node2 needs a real Dell 65 W adapter.
Bots freezing at 08:30 UTC
Two of the bots stalled for a few seconds every night. That's when the nightly backup reached their VM. Turning on fleecing fixed it, and enabling discard on the VM got back 21 GB.
Seven-minute power cut
No UPS yet. Everything came back, but Jellyfin and Immich started before the NAS was up and saw empty folders. node2 now waits for the NAS before starting them.
Cron jobs that never worked
Three of them. cron runs with a bare PATH and the Proxmox tools live in /usr/sbin, so they'd been failing quietly since day one. Every cron file sets PATH now.
node2's disk filled up
At 2:47 am, and everything on it went read-only. The main backups lived on that same disk. They're on the NAS now, and every disk has an alert.