Run a 30B model on a 2016 desktop (quantprobe setup)

Two lines. It downloads nothing and answers in about a second.

pip install quantprobe
quantprobe plan --model qwen3-30b

It reads your machine on its own. No hardware flags needed.

What comes back

quantprobe plan - Qwen3-30B-A3B @ 2.5-bit on THIS machine [auto-detected]
  model 10.6 GB | active 1.53 GB/token | est. quality cost x1.07

  *   22.2 tok/s  split experts: 34%->VRAM, rest->RAM
      19.0 tok/s  hybrid: attention->VRAM, experts->RAM
      13.2 tok/s  pure CPU (GPU idle)

  binding constraint: BANDWIDTH-BOUND (system RAM bandwidth) - 51% of every
  decode token is spent there.

  run it:  llama-server -m model.gguf -ngl 99 -ot "..." --no-mmap ...

Three placements, ranked, and the exact llama-server command for the winner. You can paste it straight in.

The line that decides whether spending money helps

That binding constraint line is the one almost nothing else prints, and it is the whole reason to run this before you buy anything.

• DISK-BOUND at 3 tok/s means more RAM genuinely helps. Buy it.
• BANDWIDTH-BOUND at 3 tok/s means it will not. Save your money.

Same 3 tok/s on screen, opposite decisions.

The honest limits, up front

• Validated 7B to 744B, 8.4% median absolute error across a 14 model ladder.
• That 8.4% was measured on a deliberately quiesced machine: no browser, no coding agent. Your number on a working machine will be worse.
• The project's own note says the median moved 9.0% to 8.4%, which is inside its own plus or minus 1 point noise floor. So read it as "roughly 9%", not as a precision claim.
• The 2016 desktop result is 20.8 tok/s median across 1,231 requests, 17.4 to 21.4 typical. A real working session, not a clean bench. The box is a GTX 1060 6GB, 16GB DDR4, SATA SSD.
• A 110B on that same box is 0.19 tok/s. One token every five seconds. The project calls it a capacity demo rather than usable inference, and that is the honest label.

The part that surprised me most, and it is not about this tool

They ran the same command five times, unchanged, and got 11.3 to 70.7 tok/s.

Nothing differed between runs except which config had been benchmarked just before. If the model is bigger than your free RAM, the page cache carries the previous run's working set into the next process.

• Runs whose predecessor matched: 0.8 to 1.8% spread.
• The same runs whose predecessor differed: 21 to 72% spread.
• A 6.3x span from run order alone.

Which means a lot of published local-LLM benchmarking is comparing cache states rather than configurations. The fix costs nothing: interleave your arms, repeat them, and only compare readings whose predecessor matched. Three passes puts everything inside 2%.

It cost them their own headline, too. A published 14.86 tok/s came back as 11.0 days later on the same box, because nothing had recorded free RAM next to the number. They logged it as correction C-32 and left it in.

That is the bit I would have wanted to know before trusting any speed number, including my own.

---

github.com/FedericoTs/quantprobe, MIT.

What did your binding constraint line say?

Paste your quantprobe output in The Rabbit Hole and I will tell you whether more RAM would actually help.

It is the room where I keep the parts that don't fit on a page like this: the full run of numbers, what broke the first time, and the ones that looked good and then didn't work at all.

Two catches, up front, because that's the rule here. It's private, so you have to be inside to read anything. And it's new, so you'd be early rather than joining something busy.

Free: skool.com/the-rabbit-hole-3206