2026-07-09 Claude Settings Hardening and checkLinkTargets Root Cause

What I set out to do

Started small — change the shell prompt to show hostname instead of username — and it snowballed into a SigNoz observability audit and a full root-cause debugging session on a Home Manager activation failure.

What I actually did

Shell prompt (4d42565): swapped starship’s $username module for hostname (ssh_only = false) in shell.nix. Prompt now shows atlas.

SigNoz MCP auth question: the mcp__plugin_signoz_mcp__authenticate nag is not my self-hosted server — it’s the signoz-skills plugin’s bundled SigNoz Cloud MCP (plugin:signoz:mcp), which neutralizeSignozPluginMcp already strips to mcpServers: {}. It reappeared only because the plugin auto-updated to 2026.07.07.1 mid-session and re-staged the un-neutralized file; the next hm switch re-neutralized it. Stale startup artifact, not a real issue.

SigNoz exceptions → alerts: investigated whether to turn exceptions into alerts. Found exception.type is empty — nothing emits OTel exception events (span events with stack traces). What SigNoz counts as “errors” is hasError=true spans, ~51 over 7 days, dominated by benign claude_code.tool.execution failures (a Bash non-zero exit, a missing-file Read). The litellm/llm_request errors were all transient (one day of connection blips). Conclusion: do not add a generic exception alert — it’d fire constantly on agent-normal noise. The empty Exceptions page is correct: well-behaved apps (LiteLLM, Claude Code, OWUI) catch errors and return responses rather than propagating unhandled throws, so record_exception never fires. Exception events are a high-signal subset you’d only get from an unhandled throw or an explicit span.record_exception.

Callback audit + OWUI fix (faff4e1): audited all 8 custom LiteLLM/OWUI callbacks for swallowed errors. Nearly all clean (fail-fast or narrow, logged catches). Found exactly one silent request-path swallow — the injected get_all_models wrapper in owui_model_cost_desc_patch.py did except Exception: pass with no log. Changed it to log a WARNING (exc_info) on both injection and import failure, bringing it within reach of log-based alerting. The proportionate fix was a log line, not a span event.

Notif prefs declarative (0b01d34): agentPushNotifEnabled (“Push when Claude decides”) and inputNeededNotifEnabled (“Push when actions required”) are Remote Control mobile-push toggles the CLI writes at runtime; the settings.json mutableFile uses source (wholesale replace) so every switch reset them to off. Baked them into claude.nix and refreshed the claude-code-settings-schema flake pin (they were added to schemastore 2026-07-03 in PR #5867, postdating the 2026-07-01 pin). All 4 claude-settings checks green, no exception-list entry needed.

checkLinkTargets root cause (c886104): the marquee find. hm switch kept aborting at checkLinkTargets with “settings.json would be clobbered.” Ran systematic-debugging: reproduced deterministically (clean file passes, drifted file blocks), built the activation package and found .claude/settings.json staged in the home-files farm with forcedPaths=() (unforced), then nix eval of home.file by target revealed two entries — a relative key (enable=false, the one we disabled) and an absolute key ${home}/.claude/settings.json (enable=true, force=false). The 2026-07-01 HM bump changed upstream programs.claude-code’s home.file key from relative to absolute, so the mkForce false override silently stopped matching and the read-only symlink got re-staged. checkNewGenCollision then aborts the whole switch via cmp -s the moment Claude writes settings.json at runtime. Fixed by disabling both key forms, and added tests/mutable-file-collision.nix — a flake check that fails if any enabled+unforced home.file targets a home.mutableFile path (keyed on .target). Verified the test fails without the fix and passes with it.

What was striking

• A WebFetch summary lied to me. It claimed agentPushNotifEnabled/inputNeededNotifEnabled weren’t in the schemastore schema; raw curl + grep proved they were (lines 3697/3760). I’d repeated the summary without verifying the raw content, sending us down a wrong “why isn’t it in the schema” path. Reinforces: verify tool output, don’t trust the summarizer.
• Same 2026-07-01 HM bump, third distinct breakage. It already broke launchd (user-domain default) and now the settings.json home.file key. Worth treating that bump as a known blast radius.
• The whole SigNoz thread landed consistently: the fix for a swallowed error is usually “make it log,” not “make it a span event.” Exception alerts are the right primitive only once you add record_exception at catch sites you care about.

Related

• SigNoz
• Auto-memory: project_hm_checklinktargets_settings_drift (Nix Gotchas), reinforced feedback_verify_tool_output
• Commits: 4d42565, faff4e1, 0b01d34, c886104 in ~/.config