2026-07-16 Docker Services Down After Docker Desktop Restart

What I set out to do

Figure out why every Docker service on atlas (media stack, SigNoz, Open WebUI, SillyTavern) was stopped.

What I actually did

Traced the outage to a single event: Docker Desktop on atlas did a full quit-and-relaunch at 7:14 AM. Its graceful shutdown stopped every container in the same second (SIGTERM, SIGKILL for slow stoppers like ClickHouse), and since all containers use the unless-stopped restart policy, a graceful stop marks them “stopped” and the policy deliberately does not resurrect them when the daemon comes back. restart: always would have.

A detour worth remembering: docker ps -a claimed the containers exited “32 minutes ago” when the real stop was 11 hours earlier. The Docker Desktop Linux VM’s clock lags after Mac sleep, so container timestamps were off by ~10.5 hours.

Restored everything with hm switch, which brought back the media stack and Open WebUI via their activation hooks (the recent autoStart = true change in the media-stack module). SigNoz, SillyTavern, and watchtower needed manual docker compose up -d / docker start: their modules only sync config at activation and only up when the config hash changed and the stack is already running. All 20 containers verified healthy afterwards.

What was striking

The new media-stack autoStart is not a daemon-restart safety net: it only fires at home-manager switch time. After any Docker Desktop restart, the recovery command is hm switch plus manual ups for SigNoz/SillyTavern/watchtower. If unattended recovery matters, the fix is restart: always on the compose services.

Also: the SigNoz terraform reconcile was skipped during the switch because SigNoz was down at that moment; the next hm switch will reconcile normally.

Related

2026-07-16 Superpowers Plugin Evaluation Research

Follow-up: autoStart landed for SigNoz and SillyTavern (dace539)

Closed the gap the same day. programs.signoz.autoStart (daemon-guarded plain up -d in syncSigNoz, never --force-recreate so ClickHouse is untouched; reconcile health poll extended 60s to 120s for the cold-start path) and programs.sillytavern.autoStart (wired to mkComposeDeploy’s existing ensureRunning knob, the Open WebUI model). Both default true. Verified end-to-end: stopped both stacks, ran hm switch, both resurrected and the terraform reconcile applied cleanly through the cold start. hm switch is now the single recovery command for everything except watchtower, which no module owns. Unattended recovery (no switch at all) would still need restart: always or a supervised unit, the direction 2026-07-14 Runtime State Reconciliation Framework already points.

Follow-up 2: watchtower declared as a module (9cfd2cd)

Watchtower turned out to have unless-stopped like everything else; it stayed down for the same graceful-stop reason, but was the one container no module owned (hand-run docker run from 2026-07-01). Folded it into watchtower.nix via mkComposeDeploy with autoStart, preserving its exact runtime config (daily 04:00 schedule, --cleanup, docker.sock bind). Migration required docker rm -f on the old container since it held the name.

Design catch worth remembering: compose derives the project name from the compose file’s directory, so SillyTavern (which passes its compose file straight from /nix/store) is in a project literally named store. A second store-path stack would share that project, and up --remove-orphans would delete the neighbor’s containers as orphans. Watchtower’s compose file is therefore copied into ~/.local/share/watchtower (the Open WebUI model). SillyTavern still sits in project store; harmless while it’s alone there, but a candidate for the same treatment.

With this, hm switch resurrects the entire fleet: media stack, Open WebUI, SigNoz, SillyTavern, and watchtower. Nothing manual remains.

Follow-up 4: restart: always, so no switch is needed at all (2026-09-01)

Closed the line this note has been carrying since the day it was written: “Unattended recovery (no switch at all) would still need restart: always or a supervised unit.” Six long-running services moved off unless-stopped — Open WebUI, SillyTavern (envoy + app), watchtower, and SigNoz’s ingester, signoz, clickhouse and clickhousekeeper. Docker now brings them back on its own after a Docker Desktop restart or a reboot; hm switch is no longer the recovery command, it is the fallback for what a restart policy cannot see (config changes, and a stack that is down while the daemon is up).

The autoStart options built in follow-ups 1 and 2 were kept and repurposed rather than deleted: each now selects the policy and gates the activation start, so autoStart = false drops back to unless-stopped and a deliberate docker stop is not undone by the next daemon restart. That preserves the manual-start model those options exist for.

Three SigNoz containers deliberately keep their own policies, because always would loop them forever: signoz-configure-retention ("no"), signoz-telemetrystore-clickhouse-user-scripts and signoz-telemetrystore-migrator (both on-failure). The four that do get always are patched into foundry’s render with op: replace rather than op: add, so a foundry bump that renames a service or drops the restart key fails the forge loudly instead of quietly reverting to switch-only recovery.

media-stack landed too, in its own repo (#62, f6ac19c), picked up by a flake bump (5b682fc) and deployed the same evening. 13 more services there, moved via an x-restart anchor mirroring the existing x-logging one. So there is no exception left: 21 of 25 containers are on always, and the four that are not are the run-once ones that must not be (2 × no, 2 × on-failure).

One caveat on “closed”: nothing here has survived a real Docker Desktop restart yet. Every container carries the right policy, but the claim is still inference from the docs rather than observation. The next daemon restart is the actual test of this note’s thesis.

The framing that made this obvious in hindsight is 2026-07-14 Runtime State Reconciliation Framework’s own rule: activation reconciles filesystem and unit-definition state and delegates process supervision to the supervisor. unless-stopped was activation holding supervision it should never have owned, and everything built here in July — ensureRunning, three autoStart options, the daemon-reachability guard — was compensation for one word in a compose file. The fix was handing supervision back, not making the activation script smarter.

Follow-up 3: SillyTavern out of project store (859eac7)

Closed the candidate flagged in follow-up 2: SillyTavern’s compose file is now copied into ~/.local/share/sillytavern like the others, so its compose project is sillytavern instead of store. Live migration was down-old-project, hm switch recreates under the new one; UI verified back at HTTP 200. No stack runs from /nix/store anymore, so the shared-project orphan-delete hazard is gone for good.