Achhina's Digital Garden

2026-06-21 Open WebUI Image Models On Demand

5 min read

2026-06-21 Open WebUI Image Models On Demand

What I set out to do

Figure out how to access different image models on demand in my Open WebUI setup, instead of the single hardcoded image-button model. It started as a usage question and turned into two distinct bug fixes through the LiteLLM proxy. (Continuation of the day’s Open WebUI thread: 2026-06-21 Open WebUI Deployment and Auth Cascade, 2026-06-21 Tailscale Serve for Open WebUI.)

What I actually did

Worked it end to end with Claude over one session, research → two fixes.

Framed the real answer first. Checked Open WebUI’s docs and issues: per-chat image-model selection isn’t supported (the image button is single-global by design; the “multiple image models” request #8491 was closed not planned, PR dropped). The endorsed pattern is “expose each model as a selectable model” via a middle layer — which I already have: LiteLLM is that layer. So the on-demand path is selecting image-output models from the normal model dropdown.

Bug 1 — image model 404s on tool use. Selecting openrouter/openai/gpt-5-image-mini gave OpenrouterException ... No endpoints found that support tool use. Try disabling "get_current_timestamp". Traced it: my global DEFAULT_MODEL_PARAMS={"function_calling":"native"} makes OWUI auto-inject its builtin system tools into every request, and OpenRouter image models reject any tools array. Verified get_current_timestamp wasn’t from my config or the Obsidian MCP (note-tools only) and that OWUI’s tool/function tables are empty — it’s a native builtin. The deeper point: OWUI can’t negotiate capability over an OpenAI-compatible connection. The “no tools” fact exists upstream (OpenRouter supported_parameters omits tools) and inside LiteLLM (get_model_info → supports_function_calling: False), but it’s flattened away at the /v1/models boundary OWUI reads. So the proxy is the only place that can fix it.

Fix: a LiteLLM CustomLogger.async_pre_call_hook (litellm_strip_unsupported_tools.py) that strips tools/tool_choice when get_model_info reports supports_function_calling is False. Capability-driven, not a model list; benefits every client. Built TDD (12 tests). Verified live: the same request with a tools array returned 200 + an image. Committed 9c04166.

Bug 2 — streamed image overflows OWUI’s SSE buffer. Next attempt died with 400 ... Got more than 131072 bytes when reading: b'data: {...}'. Confirmed it was OWUI’s own aiohttp (not LiteLLM, which streamed fine): image-output models emit the whole base64 image as one >1 MB SSE data: line, and aiohttp’s readline() caps a line at 128 KiB.

Detour worth recording: I went to annotate the fix with the check-upstream-issues mechanism, found PR #24928 (AIOHTTP_READ_BUFSIZE) and assumed it merged — wrong, it was closed unmerged. Maintainer tjbck rejected it because OWUI already handles oversized SSE via stream_chunks_handler/iter_chunks(), gated by CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE. So there’s no open upstream issue to track, and the annotation was moot. I’d built a second proxy hook (force-non-stream for image-output models, TDD, verified returning JSON), but given the native knob exists I put the choice up: proxy hook (no size ceiling, fails loud) vs the OWUI env var (native, but silently drops lines over the bound as data: {}). Chose the env var, reverted all the force-non-stream scaffolding + the enrichment flag it needed, and set CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=16777216 in open-webui.nix. Verified by feeding a 1.2 MB SSE line through the container’s real stream_chunks_handler with that value — survives intact, no drop. Committed eb8f43f.

What was striking

  • Capability negotiation is the whole story. Every layer knows the model can’t take tools / outputs images — OpenRouter’s catalog, LiteLLM’s get_model_info — except the one OWUI reads. The OpenAI /v1/models shape is the lossy boundary.
  • get_model_info only works because of the enrichment plugin. Standalone it raised This model isn't mapped yet; the proxy resolves it only because litellm_openrouter_enrichment registers the whole OpenRouter catalog at startup. Reproduced the in-proxy state (register → look up) before trusting it.
  • Verify, don’t infer — twice. I briefly called the streaming fix “still 404ing” off a log line that was actually timestamped 18:32 (pre-restart); checking timestamps corrected it. And an “empty” curl was a slow 36 s image gen, not a failure.
  • The native knob has a silent-drop footgun. Lines over CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE become data: {} with no error — the proxy non-stream approach was arguably more robust, but the env var is maintainer-endorsed and simpler. Sized to 16 MiB over ~1 MB images.
  • Two fixes, one approach reverted. Good reminder that “built and verified” doesn’t mean “shipped” — the better-fitting native mechanism won.

Top 3 next

  1. Seed a few image-output models active in the OWUI workspace so they actually show in the picker (the on-demand UX the whole thing was for).
  2. Confirm an authenticated image gen renders in the UI (couldn’t drive OWUI’s /openai route from CLI — auth rejected a minted token).
  3. If a higher-res image model could exceed a 16 MiB SSE line, bump the buffer — over-limit lines vanish silently.

Graph View

Backlinks