~70 tok/sGemma 4 12B · 5080 · 4/4 coding · ~10.4GB
~28 tok/sQwen 27B IQ4_XS · same 4 tasks · ~15.6GB
~43 / ~18 / ~7Same 27B · IQ4_XS vs library Q4 vs 46-layer spill
~60–90 wallFreeToken 35B-A3B MoE · engine ~130–150 decode
~12–15 → <15080 27B IQ4 when fit vs after VRAM spill
~55–70B70-class ballpark · tuned ~35B MoE Q4 ≤32GB
Write-ups (on LLM Lanes Builds)
Quality tied — speed and VRAM decide
~70 vs ~28 tok/s · same quality score
Read on llmlanes →Same 27B, three quants
~43 tok/s full-GPU IQ4_XS · spills kill speed
Read on llmlanes →FreeToken MoE on 16GB
35B-A3B yes · Flash-Next no
Read on llmlanes →Local LLM lanes map
5080 · B70 · Spark · Halo · Mac tok/s chart
Read on llmlanes →Full Builds index: llmlanes.com/builds.html