2026-07-13 Darwin Nixpkgs Regressions Blocking hm switch

What I set out to do

Run the routine update -c (flake update → nix flake check → hm switch). It failed partway through, so the day became a debugging session instead of a five-minute chore.

What I actually did

Two independent Darwin-only regressions from the same nixpkgs bump, found one behind the other. Both are now fixed in overlays in nix/flake.nix and the switch is applied.

1. opentelemetry-exporter-otlp-proto-grpc — fixed-port test collision. Its tests/test_otlp_exporter_mixin.py starts a real gRPC server on the hard-coded 127.0.0.1:4317 in setUp. My SigNoz Docker collector already publishes 4317, so the build died with Address already in use (21 failed tests), taking litellm and the whole hm switch down with it.

The key insight: macOS Nix builds share the host’s loopback. There are no network namespaces on Darwin, this box runs sandbox = false, and the derivation even opts in explicitly with __darwinAllowLocalNetworking = true. Linux builds get a private netns from the sandbox and never collide, which is why upstream CI stays green and it looks like a Darwin-only phantom.

Nothing in the flake bump actually broke. The collector simply happened to be running. Repointing the port was a dead end (I checked the source): setUp builds the exporter on the default endpoint and tests assert the literal localhost:4317 / server.port == 4317, so moving only the bind would have aimed the client at the real collector. Fixed with a Darwin-gated disabledTestPaths entry in the pythonPackagesExtensions overlay, keeping Linux coverage intact.

2. qbittorrent 5.2.2 — ld64 hardening crash. Revealed only once the first fix cleared. The cctools linker aborts with Trace/BPT trap: 5 / exit 133. Not an OOM and not a corrupt store path: the linker itself is crashing, from a nixpkgs hardening change. Upstream has been patching victims one at a time (kitty, ffmpeg, opencv, sdl3, yabai, telegram-desktop) while the real fix (NixOS/nixpkgs#536365, “ld64: disable hardening again”) moves through staging. Ported the still-open PR #540769 verbatim (-fuse-ld=lld + llvmPackages.lld), tagged @upstream-issue so the scanner retires it once merged.

Verified end to end: activation package built, hm switch applied, just check green, qBittorrent.app deployed.

What was striking

The first failure read exactly like a flake-update regression, and it wasn’t one at all — it was my own running infrastructure colliding with the build sandbox that isn’t really a sandbox. lsof on the port would have answered in five seconds what the build log made look like an upstream bug. That’s the reusable lesson: on a Darwin build failure mentioning a bind or a port, check what’s listening before suspecting the bump.

The second one rewarded searching nixpkgs for the error signature rather than the package name. qbittorrent had an open PR fixing precisely this, so the correct move was to port it, not invent a fix — and to tag it so it deletes itself later. The wave of near-identical “work around ld64 hardening issue” PRs made the shape of the problem obvious once I recognized it.

Also worth noting: fixing one blocker only unmasked the next. Worth remembering not to declare victory on the first green build step.

Postscript: the qBittorrent fix outlived its reason

Later the same day I deleted the whole second fix, because I deleted the package it was for.

Checking why qbittorrent was even in the closure surfaced the obvious: my actual torrenting runs in the ~/docker/media-stack compose stack (lscr.io/linuxserver/qbittorrent), alongside Sonarr and Radarr that depends_on it. The native macOS app in guiApps was a second, unrelated qBittorrent with its own config and its own state, sharing nothing with the container but the name and a claim on port 6881. Whichever started first would win the port and the other would quietly fail to listen.

So: dropped qbittorrent from guiApps in packages.nix, and with it the overrideAttrs overlay in flake.nix (dead code once the package is gone), which retires the @upstream-issue tag on PR #540769. hm switch applied, just check green, binary and app bundle both gone. Committed as d295d3d.

The lesson isn’t about ld64. It’s that I spent real debugging effort porting an upstream patch to build a package I did not need, and never asked whether I needed it. The build failure was the first time in months anything made me look at that line of packages.nix. A hard-won workaround is a sunk cost, not an asset. When a package fights you, “why do I have this?” deserves to come before “how do I fix this?” The general ld64 lesson from the original session still stands and still applies to the next Darwin link crash.

The thread kept pulling: from one dead package to every script’s dependencies

Deleting qBittorrent raised the obvious follow-up: what else is in here that I don’t use? Measured it properly, by exclusive closure size (paths reachable only from that package) rather than closure size, which double-counts the shared Python base. Total HM closure is 12.9 GB and litellm alone is 3.55 GB exclusive, 27% of everything. Confidently dead, about 1.1 GB: zotero (never launched, and it carries that elaborate codesign override), go+gopls (zero invocations in 15,447 commands, no .go files on disk), pyright (config present but not in vim.lsp.enable; ty is the one actually wired), antigravity-cli, iterm2 (never launched, I run Ghostty), vhs, pyrefly, gh-dash. Not yet removed, pending a decision on zotero and go.

Two network tools stood out for a better reason. ifstat-legacy and bandwhich entered in one commit (1bdf9da, 2025-07-10, “WIP: tmux status line transfer”) as raw material for the tmux network widget being built in that same commit. The widget shipped. It uses macOS /usr/sbin/netstat instead. Neither package was ever wired to anything, and they sat in the closure for a year and three days. Removed in b352d52.

That’s when the real question landed: the dependency wasn’t declared where it was needed. packages.nix is a flat global list, scripts pick tools out of ambient PATH, and nothing in Nix connects network.sh to netstat+awk+bc, or records that nothing needs ifstat. So I audited both directions and then closed the gap the Nix way: every deployed script is now a writeShellApplication (or a zsh equivalent, since writeShellApplication is bash-only and lints with shellcheck, which has no zsh dialect) with runtimeInputs declaring its tools, and every call site interpolates the store path. ~/.config/tmux/scripts/ no longer exists. Six commits, aaad8c9 through a42a711.

What the rollout actually taught me, in order of how much it stung:

• The empty-environment test is the only honest one. My first pilot declared gawk and bc and looked fine, because writeShellApplication prefixes PATH rather than replacing it, so undeclared tools still leak in from ambient. Running the built binary under env -i immediately failed on tail. Ambient PATH had been hiding the gap from me exactly the way it hides it in production.
• I asserted a bash semantic I hadn’t checked, and it was wrong. I claimed set -e would abort on [ -n "$X" ] && ! tmux has-session && X="", restructured six scripts around it, and wrote it into the commit message. Then my own test printed REACHED THE SWITCH LOGIC. Bash exempts a failing non-final command of an && list (and !-inverted statuses) from errexit. The genuine hazards were narrower: nounset on $1, and a failing command substitution in an assignment. The rewrites were still worth keeping; the reasoning I’d published for them was false, and I had to go back and correct the comments and the record.
• Declaring a dependency is not the same as declaring the right one. Re-running the audit afterward caught claude-detect.sh still resolving nvim ambiently. The fix is not pkgs.neovim but programs.neovim.finalPackage: it probes claudecode.nvim’s API over RPC, and a vanilla nvim would answer without the plugin and silently detect nothing, which is worse than failing. Same shape as the three tools I deliberately left ambient in hm/update: system nix (Determinate), the profile’s own home-manager, and the cctools-wrapped uv. The question is never “is the tool present” but “is it the exact instance this config owns”.
• The shellcheck gate earns its keep immediately. It rejected notify on first build over the \e\\ DCS terminator in its tmux passthrough printf (SC1003), and it linted update, 300 lines, for the first time ever. Clean.

And I broke something. To “smoke test” the wrapped ~/bin scripts I ran tmux-clean, whose entire purpose is to kill every session but the current one, against my live tmux server. It killed the docs session. Only a head -2 on the pipeline saved the rest: SIGPIPE aborted its kill loop after one. Nothing unsaved was lost, but the layout, working dirs and pane processes are gone. The claim under test was “the wrapper has its deps baked into PATH”, which rg 'export PATH' <wrapper> answers by reading the artifact. Execution added nothing and cost a session. Before running any script against live state, ask what it does when it succeeds, not whether it exits 0. The name is the warning: *-clean, *-kill, *-prune, *-reset.

The general principle: declare it where it’s used

The script rollout was one instance of something broader, and once named it turned up everywhere. When a declaration has to reach across a boundary for its subject, it’s in the wrong file. ifstat-legacy in a global package list instead of next to the widget that wanted it. A script resolving awk from ambient PATH instead of runtimeInputs. And then, once I went looking:

• Shell aliases. l/ll/lt sat in shell.nix while eza was enabled in eza.nix. Drop eza in a cleanup and you’d get three broken aliases and a switch that still applies cleanly. home.shellAliases merges across modules, so each alias now lives with its provider and cannot outlive it. This also fixed a bug in waiting: the headless home-server profile curates its own import list, so aliases parked in shell.nix would have survived a module being dropped from that profile and dangled there.
• Nine packages with two owners. git, git-lfs, bat, eza, ripgrep, fzf, delta, tmux, starship were declared in packages.nix and installed by their own enabled programs.<x> module. Nothing was broken (same store path either way), but prune it from one place and it stays silently installed by the other.
• git.nix configuring tools it didn’t own. It hand-wrote core.pager, interactive.diffFilter and the whole [delta] block while delta’s package lived in packages.nix; same for the filter.lfs block and git-lfs. programs.delta and programs.git.lfs.enable own package + settings + git wiring as one unit, and point git at absolute store paths instead of bare command names, so the pager no longer depends on PATH at all.
• tmux-mem-cpu-load in packages.nix, called by the tmux status bar through whatever PATH tmux hands its shell. It’s a tmux tool. It moved, and the widget now calls it by store path.
• tclean, which is where Adam caught me: the alias was in shell.nix pointing at a script built in shell.nix, but tmux-clean is a tmux tool. All four tmux ~/bin scripts moved to tmux.nix, and the alias now sits directly beside the script it calls.

Upstream keeps validating the pattern. HM’s own programs.fd sets home.packages and the fd alias together in one module; enabling it meant deleting my hand-written alias, not moving it, because it would have collided. The same idea, already shipped.

Two things I got wrong

I asserted a bash semantic I hadn’t checked. I claimed set -e aborts on [ -n "$X" ] && ! tmux has-session && X="", restructured six scripts around it, and put it in a commit message. My own test then printed REACHED THE SWITCH LOGIC. Bash exempts a failing non-final command of an && list (and !-inverted statuses) from errexit. The real hazards were narrower — nounset on $1, and a failing command substitution in an assignment. The rewrites were still worth keeping; the reasoning I’d published for them was false, and I had to go back and correct the comments and the record.

I over-commented, and Adam had to call it. 200 comment lines against 272 lines of code, on a refactor that mostly moved things. The bad ones all had the same tell: they explained the change to a reviewer rather than the code to a reader. History (“previously the package was declared in…”), restatement (I wrote “co-located with the package that provides it” three separate times — the placement is the statement), a 13-line essay on errexit semantics that belongs in a note like this one rather than in a module, and pointers whose only future is to be wrong (“the tmux tools are NOT here”). Cut to 75. The keepers are all short and all say “don’t do the obvious thing, here’s why” — programs.neovim.finalPackage not pkgs.neovim, because a vanilla nvim answers claude-detect’s RPC without the plugin and silently detects nothing.

The last holdouts: Claude’s hooks, and a silent telemetry drop

Adam pointed at the surface I’d left alone: the Claude hooks. All eight were built with pkgs.writeShellScript, which declares nothing, and they pull direnv, nvim, tmux, jq, curl, git and rg out of ambient PATH. Worse than the tmux scripts had been, because hooks run under Claude Code’s environment, not a login shell — the “whatever’s on PATH” assumption is least safe exactly where I’d left it standing. mkHook now builds a writeShellApplication with per-hook runtimeInputs (858e180).

One subtlety worth keeping: hook exit codes are semantic. prevent-permission-prompts exits 2 to block a tool call. Imposing the house errexit default on a hook written without it could turn a benign command failure into a block, so bashOptions mirrors each hook’s own set line rather than a blanket policy. The same audit found notify calling nvim completely undeclared, and the 5 Claude LSP plugins enabled unconditionally while the servers they spawn from PATH are gated behind local.devTools — flip that flag off and you’d get five enabled LSPs with no binaries, failing quietly.

Then a final sweep for anything still undeclared, which turned up the best bug of the day (f0afc6f). hook-otel-emit declared curl, jq and coreutils — and called python3. For a nanosecond timestamp. It resolved to /usr/bin/python3, the Xcode CLT copy, not the profile’s Python: my hook telemetry was quietly leaning on an unmanaged Apple binary. And the script guards every tool with command -v x || exit 0, so without CLT it wouldn’t error — it would exit 0 and drop every hook telemetry record, silently, forever. The comment directly above the derivation already explained that this guard is precisely why curl and jq must be declared. python3 walked straight through the reasoning written to stop it.

The fix removed the dependency instead of declaring it: coreutils was already an input and GNU date does +%s%N, so the timestamp needs no interpreter at all. A Python closure deleted, not added.

And the verification mattered more than usual, because exit 0 is also what the silent drop returns. Running it under env -i and seeing 0 would have proved nothing — it’s the exact failure signature. I had to go look in SigNoz and confirm the record actually landed, with an intact nanosecond timestamp. Same lesson as the ntfy alerts: the test endpoint that “succeeds” is not delivery.

Rounded it out by pinning the statusline’s bare npx and the tmux CPU widgets’ bare tr/sed to store paths (6b19203). The sed swap crosses BSD→GNU, and the widget’s live output changes every invocation, so I diffed the substitution on a fixed fixture rather than eyeballing the bar: byte-identical.

The through-line of the whole day: a guard that turns a missing dependency into silence is not a safety feature, it’s a way to never find out. Every one of these — ifstat unused for a year, aliases outliving their package, python3 borrowed from Xcode — was invisible precisely because nothing failed.

Related

• Dotfiles Host Profiles via Private Flake