2026-06-22 OWUI Per-Message Cost Tracking

What I set out to do

Get an info toggle in Open WebUI showing per-message token usage and cost in chat. Started as a simple “where’s the ⓘ icon” question; turned into a deep debugging arc through OWUI internals, LiteLLM, and SigNoz.

What I actually did

• Found why no ⓘ icon: OWUI only renders it when the message has a usage object, and the frontend only requests usage (stream_options.include_usage) when the model advertises capabilities.usage (default off). OWUI also discards LiteLLM’s x-litellm-response-cost header — so cost has to be captured ourselves.
• Shipped a global Filter (seeded declaratively into OWUI’s function table via sqliteSeed, hex-embedded to dodge SQL/Nix quoting): inlet forces include_usage for every model; outlet emits a status line with tokens + LiteLLM cost + round-trip count.
• Fixed tool-call undercounting: OWUI persists only the final LLM call’s usage per message. The filter’s stream method (runs once per round-trip inside stream_body_handler) now accumulates the per-turn total and rewrites each usage chunk, so OWUI stores the full total — fixing the chat line, native popup, and analytics.
• Folded image-generation cost, then image-edit cost — via a container entrypoint shim (patch-and-start.sh + owui_image_cost_patch.py) that applies marker-guarded, drift-safe string patches on every boot (image gen has no chat-filter hook).
• Enabled image editing (ENABLE_IMAGE_EDIT + IMAGE_EDIT_* config) and verified end-to-end.

What was striking

• “Telemetry miss” was the key insight that wasn’t. The cost was in SigNoz all along — as the gen_ai.client.token.cost metric, just not a span attribute (metrics flow on a separate path by design). That breakdown revealed the image is ~95% of turn cost ($0.069vs$0.003 chat), and that my “image fold works” claim was wrong — the displayed cost was chat-only.
• The real bugs only surfaced by driving the actual UI (Playwright — the API needs a token even with WEBUI_AUTH=False, and forging a JWT from the secret was correctly blocked by the classifier): (1) images are generated via the generate_image builtin tool (native function-calling), not the features.image_generation handler I’d patched; (2) the per-turn __metadata__ dict differs between payload and response phases, so cost folded there never reached the outlet → moved the accumulator to request.state (one object shared across tool, stream, outlet); (3) image_generations does metadata = metadata or {}, silently discarding a falsy empty dict — had to pass a truthy one.
• Verified live: 14,937 in / 152 out | $0.069253 | 3 calls on an edit turn.

Lessons worth keeping

• Verify against the live system before claiming success — I asserted the image fold worked twice before it actually did. The SigNoz metric cross-check and UI-driven e2e were what finally proved it.
• Cost is not in the OTel GenAI semconv (only gen_ai.usage.input/output_tokens); LiteLLM cost is a proprietary extension, surfaced only on the metric / response header.

Top 3 next

1. Watch upstream open-webui#25617 / PR #25659 — when a pre-summed per-turn usage lands, remove our stream-accumulation and image patches (the outlet already handles a usage list for forward-compat).
2. Non-streaming turns still won’t capture image cost (the stream filter never runs) — acceptable since chats stream by default, but note it if non-stream ever matters.
3. After OWUI image updates, check the entrypoint patcher log for any SKIP (anchor not found) — drift means a fold silently no-ops.

Related

Open WebUI · LiteLLM · SigNoz