46 containers, one box, zero open ports
Every system on this site — the CRM, the booth platform, the automations, even the git server that hosts them — runs on hardware I own and operate. No managed cloud between a bad decision and its consequences.
From the internet to a container
Each request crosses three gates before it reaches an app, and each app sits on its own private network.
- 1 · EdgeCloudflare TunnelOutbound-only connector. Nothing listens on the router.
- 2 · IdentityCloudflare AccessEmail policies for people, one service token per kiosk.
- 3 · ProxyCaddy + TinyAuthPer-host rules; forged identity headers stripped.
- 4 · AppIts own auth, againDefault-deny allowlists and device tokens in the app itself.
- BusinessCRM & FinOps · e-signing · client portal · booth platform · guest galleries
- Automationn8n · Windmill · Telegram bots
- EngineeringForgejo · Actions runner · Playwright E2E · draw.io
- PersonalFinance app · health tracker · Vaultwarden · ownCloud · Paperless-ngx
- OperationsCheckmk · Dockhand · Snipe-IT asset register
Two locks on every door
The API allowlist is enforced twice on purpose — in the proxy and again in each app — so widening one layer never silently opens the other.
The weak spot was the proxy: apps trust an identity header, so a single proxy block that forgot to strip it would be a full bypass. A CI check now reads every Caddyfile and fails the build if any block that reaches an app doesn’t strip it.
My own pipeline, on my own box
- Playwright E2E per app — the website’s build is served from a throwaway container and tested as a hard gate on every commit.
- Checks image — typecheck, lint, formatting and unit tests before anything merges.
- Daily coverage report — how much of each published API surface any test actually claims.
- Docs that can’t go stale — every infra commit updates the README and agent brief; a git hook prints the checklist.
Moving house in under ten minutes
In June 2026 the whole stack moved from an old HP EliteDesk to a Ryzen 9 mini PC — with 9 minutes 42 seconds of customer downtime at 2 a.m., and every content count matching afterwards.
- Pre-seed live, cut over briefly~14 GB synced over a private link while the old box kept serving; only database stacks stopped for the final delta.
- Move the running images, don’t rebuild themByte-identical to what customers were using — and a rebuild would have broken, because an upstream base image had drifted.
- An adversarial review firstIt caught four traps — split compose folders, named volumes that don’t travel with bind mounts, a misplaced tunnel token, an orphan volume — each a silent data loss.
Three copies, and a ritual
Every two days an n8n workflow asks me on Telegram to plug in a drive, runs the backup and tells me when to unplug it. Two USB disks rotate, so one bad snapshot never takes out both.
- Consistent dumpspg_dump and VACUUM INTO before each snapshot — never live database files.
- Encrypted resticKeep 7 daily, 4 weekly, 6 monthly, 2 yearly.
- A monthly restore drillChecks the repo for bit-rot and test-restores a real database, instead of trusting “it probably worked”.
The backup that quietly stopped
The third copy — on a standby machine — silently failed for 48 days. The assumed cause was an expired VPN key. Kernel logs said otherwise: identical crashes on two kernels, in random unrelated processes, with mangled pointer bits. Failing RAM.
The rotating disks held the line. The standby is being replaced by a desktop woken on LAN for each backup — and the lesson stuck: a backup needs an alarm for silence, not just for errors.
The night the passwords disappeared
A password-manager extension auto-updated ahead of its server. Server healthy, API returning 200, vault empty — only the client console told the truth.
Quote anything with a $
An unquoted .env value once let a parser mangle a hashed secret. Now every special-character value is quoted, full stop.
De-Googled the living room
Debloated the TVs over network ADB and replaced the launcher with a tiny app I wrote that boots straight to HDMI.