Run a 125B model on a 48GB laptop (slotstream setup)

What it is. slotstream runs Qwen3.8-Flash-Next, a 125B mixture of experts model, on an Apple Silicon Mac with far less RAM than the model by keeping the weights on the SSD and holding a working set of experts in RAM (a 33 GB target on a 48 GB machine, about 152 experts per layer, per docs/HARDWARE.md), loading the rest from disk as needed. One Swift binary, no Python. It speaks the Ollama and OpenAI chat API subsets, so Open WebUI and your own code point at it as if it were Ollama.

Every number on this page is from the README and docs/HARDWARE.md, read 17 Sep 2026. Source: https://github.com/carloslfu/slotstream (370 stars, MIT, created 28 Aug 2026).

The numbers, and the four caveats

Machine	RAM	Measured speed
MacBook Pro, M5 Pro, 0.2.19 at a 22 GB target	48 GB	15.86 tok/s
Same M5 Pro, historical 0.2.3 result	48 GB	about 12 tok/s (12.8 with --mtp)
Mac mini, M2	16 GB	1.41 tok/s
MacBook Air, M5	32 GB	6.22 tok/s
MacBook Pro 16", M5 Max, auto (34.6 GB target)	128 GB	about 21 to 22 tok/s
Same M5 Max, 48 GB target	128 GB	about 26.9 tok/s
Same M5 Max, 73 GB target	128 GB	about 31.5 tok/s

1. The 18, 24 and 36 GB sizes have no reports, and 8 GB Macs do not run the model. The speeds the README shows for unmeasured sizes are the planner's estimates: the 16 GB and 32 GB results came in below theirs, the 128 GB one above.
2. The model on disk is 105.3 GB of 4 bit MLX weights in 25 files (88.3 GB compressed for the download). This is a different format from the GGUF builds, whose smallest quant is 67.6 GB, so both numbers are true.
3. slotstream is not the only project streaming experts off SSD on Apple Silicon. mlx-moe-offload and streamlx are named in the same HN thread.
4. The three M5 Max rows are one machine with a larger expert cache each time, not a scaling curve; docs/HARDWARE.md says so in as many words. That Mac is also outside slotstream's target range, since the whole model fits in its memory, and engines that keep it resident report faster replies there.

Step 0: what you need

An Apple Silicon Mac, macOS 14 or later, and about 110 GB of free SSD before you start. Check free space first, the model is much larger than the program. An 8 GB Mac needs swap even at the minimum and can slow the whole machine.

Step 1: install

curl -fsSL https://raw.githubusercontent.com/carloslfu/slotstream/main/install.sh | sh

Installs the latest release into ~/.slotstream/bin. If the terminal cannot find slotstream, open a new terminal window. Run the same line again to upgrade.

Step 2: check your machine before downloading anything

slotstream doctor

Prints the memory plan, the estimated speed for your RAM, and free disk space, without loading or downloading the model. If the estimate is under about 4 tok/s, decide now whether an evening of downloading is worth it.

Step 3: first reply

slotstream run --prompt "Why is the sky blue?"

On first use it offers to download the model, shows the size, destination and free space, and asks for confirmation. The transfer alone is about 2 hours at 100 Mbps or 8 hours at 25 Mbps. Interrupted downloads resume, every file is checked, no Hugging Face account needed. You download once.

Step 4: use it from your tools

It serves the Ollama and OpenAI chat API subsets, so Open WebUI, a chat app, or your own code can talk to it the way they talk to Ollama. The API doc is docs/API.md in the repo. A client in a container needs its own networking setup.

What to expect, honestly

The first reply is slower than the steady state because the cache has to warm up. 15.86 tok/s is the 48 GB M5 Pro figure on the current build (about 12 on the build this page first quoted); a 32 GB M5 Air got 6.22 tok/s, and a 16 GB M2 got 1.41 tok/s, which is usable for a question and painful for a conversation. If your Mac is an 18, 24 or 36 GB size, you are the measurement, and the author is collecting them in docs/HARDWARE.md.

Where this lives

The Rabbit Hole is free to join. The course library is free; optional Premium adds weekly AI Builder Office Hours. It is where the full numbers and the follow ups go: https://www.skool.com/the-rabbit-hole-3206/about