Achhina's Digital Garden

2026-07-14 OpenRouter Interceptor Storage Accuracy and Dedupe

5 min read

2026-07-14 OpenRouter Interceptor Storage Accuracy and Dedupe

What I set out to do

The settings Data Management tab showed 6404 KB for 100 network requests, and I wanted to (1) know how accurate that number is and confirm it against runtime storage, and (2) dedupe network requests at the storage level — one entry per character like the sidebar already displays — since prior individual requests are never reviewed. Follow-up to 2026-07-11 OpenRouter Interceptor Capture Cap and Settings Rewiring and 2026-07-09 OpenRouter Interceptor Capture Consolidation.

What I actually did

Confirmed the reported number’s provenance by reading the real runtime storage. The extension lives in the Firefox PWA “JAI” profile (01K3EX8H…), extension UUID 11d27b3f…, using Firefox’s IndexedDB backend for storage.local. openrouter_captures is externalized to a standalone blob: 5,353,252 bytes (5,228 KB) on disk, vs the 6,404 KB the options page reported. The old formula (JSON.stringify(captures).length / 1024) is wrong two ways: .length counts UTF-16 code units not bytes, and it measured only captures (ignoring openrouter_downloaded ~80 KB, openrouter_queries ~64 KB, openrouter_identity_map ~20 KB). Firefox runtime is 154.0a1 Nightly, well past 144 where storage.local.getBytesInUse landed.

Then ran the full brainstorm → spec → plan → subagent-driven execution loop. Shipped 10 tasks, 14 commits, merged --no-ff to main (c4aba9e); 381/381 unit tests, typecheck + build clean.

  • Part 1 (accurate storage): updateStats() now calls storage.local.getBytesInUse(null) (native, all-keys, byte-accurate) with a tested estimateStorageBytes/formatBytes Blob-sum fallback for older Firefox.
  • Part 2 (storage-time dedupe): extracted a shared pickRepresentative (reused by groupCaptures, the upsert, and migration — DRY); added requestCount; upsertCaptureByCharacter dedupes on insert (merge, bump count, move to front) so maxCaptures now bounds distinct characters; a one-time migrateCapturesToDeduped consolidates the legacy per-request store on load; dropdown relabeled “characters” (1000 default).

What was striking

  • The 07-11 trap I walked straight into. That session explicitly rejected storage-layer dedup because analysis queries persist a captureId and the sidebar resolves “view capture” by c.id === query.captureId (sidebar.ts:388, button at :439, expand at :458). Storage-time dedupe changes capture identity: migration discards every non-representative id, and the upsert adopts the incoming capture’s id when its prompt is longer. Both orphan persisted analysis-query references. Neither the spec, the plan, nor the multi-agent review (including a final opus whole-branch pass) caught it — the reference lives in the openrouter_queries store, outside the reviewed diff. Consulting the vault before wrapping up is what surfaced it — and it was fixed in a follow-up branch same session (see Open follow-ups).
  • Recurring toolchain trap: lint-staged runs prettier --write then eslint --fix; eslint’s curly rule collapses brace-less ifs into single-line {return x;} after prettier, so brace-less source commits in a non-prettier state. Bit three haiku implementer commits; fixed each and switched remaining dispatches to multi-line braces + a post-commit prettier check.
  • Reported logical bytes (6404 KB) exceed on-disk (5228 KB): Firefox’s structured-clone store is more compact than escaped JSON. Logical bytes is still the honest, stable number to surface (it’s what quota counts).
  • The destructive one-time migration was never exercised on a live populated store — no Firefox runtime in-session, so it’s static-review-only.

Verification

test:unit 381 pass (27 files) · typecheck clean (3 tsconfigs) · build clean · prettier clean. Merged to main c4aba9e. Not verified: live migration smoke test on the PWA; analysis-query orphaning (newly identified).

Open follow-ups

  • Analysis-query orphaning — FIXED (fix/dedupe-preserves-analysis-refs, merged bd9146c). Did both defences: upsertCaptureByCharacter now keeps existing.id on merge (adopts only the fuller payload), and migrateCapturesToDeduped pins the survivor to the first-seen id and returns an idMap (collapsed id → surviving id). New remapQueryCaptureIds rewrites openrouter_queries through that map, wired into loadStoredCaptures so the one-time migration also repairs references. 387/387 unit tests (+6), typecheck/build clean. This is the concern the 07-11 session raised, now handled rather than avoided.
  • Live migration smoke test on a backed-up profile (still not run — no Firefox runtime in-session). This is the remaining unverified item.
  • Dedupe keyed by chat URL, not character — FIXED (fix/dedupe-by-character-id, merged post-session). After the user migrated, runtime storage dropped 5.4 MB → 376 KB (openrouter_captures 5.35 MB external blob → 160 KB inline). But it produced 3 cards that were all the same character (“Dragon”, character_id 0f5b58ed-1073-4c64-8dd9-0e01253aaaf2) across 3 chats — decoded the Snappy structured-clone blob + identity map to confirm all 3 chatIds → one character_id. Root cause: characterDedupKey falls back to the per-chat URL when the capture has no characterId, and the capture store never attached it (though auto-download already resolved it via resolveCharacterId). Pre-existing, not caused by the migration. Fix: stamp characterId at insert; new reconsolidateWithCharacterIds backfills existing captures via the identity map and re-folds; extracted a shared consolidate() loop so migrate + reconsolidate don’t duplicate dedupe logic. 391/391 tests. On next rebuilt load the 3 Dragon cards fold to 1 (requestCount ~114), analysis refs remapped across both passes.

Graph View

Backlinks