Home / tok/s reviews

tok/s reviews

Tables live here — tok/s rows restated from published LLM Lanes Builds. Method notes lives on llmlanes.com/measure.

B70 vs 5080 dual-GPU

Source: llmlanes Build (full run cards) · CSV · 2026-09-21 · Neo numbers. Different weight sets (GGUF SYCL vs Ollama tags — not same-GGUF A/B).

Lane by GPU outcome matrix
LaneB70 SYCL5080 CUDA
textpass llama-climixed Ollama
describefail llama-mtmd-clipass Ollama
image-genblocked n/a — no SYCL Comfy pathblocked n/a — ComfyUI/weights not installed
video-genblocked n/a — no SYCL Comfy pathblocked n/a — ComfyUI/weights not installed

Image-gen / video-gen stay blocked (not installed). Logo-card keepers are side/background chrome only — not inside matrix cells. No invented tok/s. Locked hexes. Art: media-create@673d219.

llama.cpp CLI (SYCL)

Same text prompts as Ollama (CUDA); hardware is a column.

ModelHardwarebugfixtooljsonrefactor
qwen3-coder-30b-a3bB70passpasspass
gemma4-26b-a4bB70passpasspass
gpt-oss-20bB70passpasspass
qwen38-27b-iq4xsB70passpasspass

Serving: llama-cli · Engine: llama.cpp SYCL

Ollama (CUDA)

Same text prompts as llama.cpp CLI (SYCL); hardware is a column.

ModelHardwarebugfixtooljsonrefactor
qwen38-27b-iq4xs5080passpasspass
gemma4-12b-agent-hi5080passfailfail
gemma4-12b-vision-agent5080passpassfail
llama32-3b5080passpasspass

Serving: Ollama · Engine: llama.cpp CUDA

Full Experience→Silicon columns (all 30 cards): open the Build run-card tables. No local machine paths on this page.

Gemma12-5060 GrokBuild 5060 Ti

Source: llmlanes Build (full run cards) · speed stamp 20260930-150835-1 · quality still 20260930-121551 · Neo run-cards only. Eng tok/s leads; wall tok/s secondary when present; no invented tok/s.

Serving: Ollama · Engine: Ollama CUDA servergen1 · Harness: GrokBuild

Speed cell (suite=speed · stamp 150835-1)

ModelHardwareEngine tok/sWall tok/sttft_msBuild
gemma4:12b (Q4_K_M) RTX 5060 Ti 16GB 88.18 39.59 24000 Build

Quality + per-suite tok/s_gen

SuiteStatus / accuracyWall tok/sEngine tok/sBuild card
speedok (150835)39.5988.1820260930-150835-1
bugfixpass 1/11.7718.8820260930-121551-2
refactorok1.9917.520260930-121551-3
tooljsonpass 4/43.7917.5520260930-121551-4
scriptreviewpartial 3/618.4420.6820260930-121551-5

Describe / image-gen / video-gen: not run this stamp (no cards). Full Experience→Silicon columns on the Build run-card table.

Method notes (same desk)

  • Named box: house RTX 5080 16GB (Rumble) unless a row says otherwise.
  • Label the clock: engine/decode tok/s and wall are different; both appear when they differ.
  • Fit vs spill: full-GPU residency vs system-RAM spill changes the number more than marketing quants.
  • Thinking off for coding comparisons and the quant bar (Ollama) unless noted.
  • Negative results stay: Flash-Next no; spilled recipes stay spilled.
  • Source of truth: the linked LLM Lanes Build — this page is the table view.

Tables live here. Method notes lives on llmlanes.com/measure.

Coding comparison — Gemma 12B vs Qwen 27B IQ4

Source: Quality tied — speed and VRAM decide (2026-09-08). Four identical coding tasks. Ollama, thinking off.

Model Quant GPU / VRAM tok/s or wall Notes Build
Gemma 4 12B (agent-hi) (house default) RTX 5080 · ~10.4GB ~70 tok/s 4/4 quality score Build
Qwen 27B Unsloth IQ4_XS RTX 5080 · ~15.6GB (tight) ~28 tok/s 4/4 same tasks — quality tied Build

Same 27B — three quants (fit vs spill)

Source: Same 27B, three quants (2026-09-08). Ollama ctx 4096, thinking off.

Model Quant GPU / VRAM tok/s or wall Notes Build
Qwen 27B class Unsloth UD-IQ4_XS RTX 5080 · ~15.3GB · 100% GPU ~43 coding tok/s Clears the bar — full-GPU Build
Qwen 27B class Library Q4 RTX 5080 · spilled ~18 tok/s Spill case Build
Qwen 27B class Video-like UD-Q4_K_M (~46 GPU layers) RTX 5080 · spilled worse ~7 tok/s Spill case — layer recipe not a plan on 16GB Build

FreeToken MoE on 16GB

Source: FreeToken on WSL2 (2026-09-07 → 2026-09-08). Engine vs wall labeled separately.

Model Quant GPU / VRAM tok/s or wall Notes Build
Qwen3.6-35B-A3B NVFP4 MoE RTX 5080 · ~15.7GB VRAM + host RAM for experts engine ~130–150 decode · wall ~60–90 (short prompts) Yes — MoE offload worked Build
Qwen3.8 Flash-Next NVFP4 (~126GB on disk) RTX 5080 16GB — No — never got usable KV even with CPU MoE Build

Lanes map — speed vs capacity (published chart)

Source: Local LLM lanes (2026-09-09). 5080 rows are house readings; B70 / Spark / Halo / Mac tok/s are community ballparks from that research pass — labeled as such on the Build.

Model / job Quant GPU / VRAM tok/s or wall Notes Build
27B IQ4 short ctx IQ4 RTX 5080 16GB ~12–15 tok/s when fit; <1 after spill House Ollama / FreeToken readings Build
~35B MoE Q4 Q4 tuned B70-class ~32GB ~55–70 tok/s Speed-lane ballpark (research pass) Build
Dense ~70B (varies) DGX Spark ~128GB ~3–8 tok/s (some FP8 ~2.7) Capacity lane — community ballpark Build
Dense ~70B (varies) Strix Halo ~128GB ~6–12 tok/s Community ballpark Build
Dense ~70B (varies) Mac Max ~128GB ~10–15 tok/s Community ballpark — chat decode champ in UMA class Build

Measured runs (one card each)

Status

B70 vs 5080

2026-09-21 · text · vision · gen blocked

Result: B70 text 4×3 pass; 5080 mixed; img describe B70 FAIL / 5080 PASS; gen not installed.
~70 t/s

Gemma 12B vs Qwen 27B IQ4

2026-09-08 · ~70 vs ~28 tok/s

Result: Quality tied 4/4; speed and VRAM decide daily-driver feel.
~43 t/s

Same 27B, three quants

2026-09-08 · ~43 / ~18 / ~7

Result: Only IQ4_XS stayed full-GPU; spills killed speed.
MoE

FreeToken MoE on 16GB

2026-09-07 → 2026-09-08 · wall ~60–90

Result: 35B-A3B yes; Flash-Next no.
Lanes

Local LLM lanes map

2026-09-09 · 5080 · B70 · Spark · Halo · Mac

Result: Speed ≤32GB vs capacity 128GB — chart, not one hybrid.

Thumbnails are placeholders — not run yet / no fake video IDs. Live channel: @toklanes; sister channel (live): @llmlanes. Open the Build.

Builds write-ups: llmlanes.com/builds · Method notes: llmlanes.com/measure · About: About