Achhina's Digital Garden

2026-09-02 Open-Weight Model Shortlist for the M4 Max

4 min read

2026-09-02 Open-Weight Model Shortlist for the M4 Max

What I set out to do

After the qwen3.5-4b baseline (2026-09-02 Local Qwen3.5-4B Inference Baseline), work out which open-weight models are worth pulling onto the M4 Max for coding, classification, and general use, with benchmark categories and results dug out properly rather than from a first-pass shortlist.

What I actually did

Five parallel research passes: Qwen3.8-27B and Flash-Next card tables (HTML tables parsed to markdown), Laguna S/XS 2.1 cards, blog, tech report, GGUF headers, and llama.cpp PR history, a survey of every other sub-100 GiB model released since March, independent boards (Artificial Analysis component evals, Scale SEAL, LiveBench, Arena, Epoch, BFCL, Aider, LiveCodeBench, Terminal-Bench, LLM Stats), and OpenRouter usage via its frontend rankings JSON. Consolidated into Local Open-Weight Model Shortlist 2026-09 and a published report page.

Ranking: Qwen3.8-27B first, Qwen3.8-Flash-Next second (blocked on memory and runner build), Ornith 1.5 35B-A3B third (unverified but MIT and fast), Laguna S 2.1 fourth (parked), Muse Glimmer 30B fifth.

What was striking

The first-pass shortlist was wrong in three places. Qwen3.6-35B-A3B, gpt-oss-120b, and Laguna S 2.1 were all “pull first” picks yesterday. Independent numbers put the first two at AA 32 and 24 against 52 for Qwen3.8-27B, and Laguna cannot load on the runner at all. The initial list was built from model-card headlines and a stale sense of which models were current.

Laguna’s model card is stale in a way that matters. The official Q4_K_M was silently replaced on 2026-07-27 with an 89 GiB mixed-BF16 build; the card still says 68 GB. The card also still says llama.cpp support is “not yet upstream” when it merged at b10087 on 2026-07-22. And there is not one general-capability benchmark for either 2.1 model anywhere, including Artificial Analysis, which has no page for Poolside at all. A model with real OpenRouter adoption (16th by tokens) and zero independent evaluation is an unusual combination.

OpenRouter’s task-share endpoint reads the market better than any leaderboard. Gemma 4’s top apps are character platforms and it ranks 5th in roleplay; gpt-oss-120b’s are tagging and summarization bots. Neither appears in any code task. The rankings page’s programming category is a 404 shell now; the data lives at api/frontend/v1/rankings/task-spend and rankings/models?view=week.

Independent boards have not caught up with August. Scale SEAL, BFCL (April), Aider (Oct 2025), LiveCodeBench (April 2025), Epoch, FrontierSWE: none has a single August model. AA and LiveBench are the only third parties with Qwen3.8 numbers. For Ornith there is nothing, which makes its 79 SWE-bench Verified claim the biggest open question on the list.

The runner’s llama.cpp build is a hidden gate. Docker Model Runner reports b9879 (2026-07-06). Every architecture merged after that date is a load test, not a pull. This was invisible until the Laguna PR date was checked against the tag.

Open

  • Pull Qwen3.8-27B and Ornith 1.5 35B-A3B, run both through the permission-suggestion eval set with thinking off and at low effort, and record tok/s from docker model logs.
  • Check whether a Docker Desktop update moves the runner past b10087 before trying Laguna or Flash-Next.
  • Ling 3.0 Flash (124B-A5B, July) surfaced only from the OpenRouter models list and was not evaluated.

Follow-up, same evening

Pulled Qwen3.8-27B (unsloth UD-Q4_K_M, 15.3 GiB, ~12 min at ~38 MB/s). It loads on b9879 and answers. Measured 21.9 tok/s single-stream with thinking off and 18.9 tok/s at low effort, where the model still spent ~1,400 tokens reasoning on a 100-word prompt (82 s wall). The “25 to 35 tok/s” estimate in the report was optimistic; dense 27B at Q4 on this machine is ~22.

Also checked GLM-5.3-Flash at 1-bit on request: architecture glm5next is in neither b9879 nor llama.cpp master (PR #27754 open, fork-only), IQ1_S is 86.7 GiB, and unsloth’s own KL divergence for it is 0.67 (top-1 agreement 71%). Not viable; the OpenRouter route already in litellm is the right way to use that model.

docker model pull prints no progress when stdout is not a TTY; watch ~/.docker/models/blobs/sha256/*.incomplete or the backend’s bytes-in instead.

Graph View

Backlinks