Run an 80B model in 4.3GB of RAM (Swiftlet setup)

Swiftlet is a Swift and Metal runtime that runs Qwen Mixture-of-Experts models on Apple devices. It keeps the small dense core of the model resident in memory and streams the routed expert weights off storage as each token needs them. That is the whole trick, and it is why an 80B fits in 4.3GB of RAM.

Model	Disk	Peak RAM	Decode speed (M5 Mac)
Qwen3.6-35B-A3B, 4-bit	18 GB	2.6 GB	7 to 11 tok/s
Qwen3.6-35B-A3B, 8-bit	34 GB	7.6 GB	3.5 to 4 tok/s
Qwen3-Next-80B-A3B, 4-bit	42 GB	4.3 GB	4.5 to 5 tok/s

The 8-bit 35B is the quality tier for Macs. Tested head to head on identical prompts, it removes the repetition artifacts the 4-bit build can show on longer writing tasks, and pays for that in disk, RAM and speed.

Install

Requirements: Apple Silicon, macOS 14 or later, and free SSD space for whichever container you pick.

git clone https://github.com/leonickson1/Swiftlet.git && cd Swiftlet
swift build -c release

Get a model

The 80B. 42 GB on disk, about 4.3 GB of RAM:

.build/release/swiftlet-repack \
  --from-url https://pub-c0cfece2dbc340dbb2cd9d94310a7d68.r2.dev/qwen3-next-80b-qpack \
  --output ~/models/qwen3-next-80b.qpack

Start with the 35B instead if 42 GB is a lot to give up. It is 18 GB on disk, 2.6 GB of RAM, and it is faster:

.build/release/swiftlet-repack \
  --from-url https://pub-c0cfece2dbc340dbb2cd9d94310a7d68.r2.dev/qwen3.6-35b-qpack \
  --output ~/models/qwen3.6-35b.qpack

The same containers are on Hugging Face with --from-hf Leonickson/Qwen3-Next-80B-A3B-qpack, but anonymous downloads there are rate limited, so the mirror is usually much faster.

Run it

.build/release/swiftlet chat ~/models/qwen3-next-80b.qpack \
  "What is the capital of Spain?" "And what about France?"

It applies the model chat template and keeps conversation state, so a follow up only prefills the new turn.

Try it on your phone

The 35B runs on iPhone inside Priv AI on the App Store. Open Settings, then Experimental Models, then download the model. It streams from storage and runs on device with no server involved.

Checked 18 August 2026: Experimental Models ships in the newest app version, which was still in App Store review, so it may not have appeared for you yet. If it has not, the app is open source at leonickson1/localLLM and you can build it onto your own phone.

Where it will disappoint you

You need 42 GB of free disk for the 80B. This is the part the headline number hides. The model does not fit in 4.3 GB, it runs in 4.3 GB. It lives on your SSD and gets read from there on every single token. The 35B wants 18 GB, the 8-bit 35B wants 34 GB.

Only about 3B parameters fire per token. It chats and writes like a large model and recalls facts like a small one. That is the design rather than a bug, and it makes this a better writing partner than an encyclopedia.

Apple Silicon only. macOS 14 or later, or iOS 17 or later. There is no Intel Mac, Windows or Linux path, so if you are on a laptop with no graphics card this is not your one. Run AI on a laptop with no graphics card is.

On an iPhone it is about 1 token a second. That is real and it is slow. Want it for the privacy, not for the speed.

Source

github.com/leonickson1/Swiftlet, Apache 2.0, Swift and Metal. Every layer of the forward pass is validated against mlx-lm reference implementations, and an expert answers identically whether it comes from cache or from disk.

How much free disk have you got, and which Mac?

Ask me in The Rabbit Hole and I will tell you which one to pull.

It is the room where I keep the parts that don't fit on a page like this: the full run of numbers, what broke the first time, and the ones that looked good and then didn't work at all.

Two catches, up front, because that's the rule here. It's private, so you have to be inside to read anything. And it's new, so you'd be early rather than joining something busy.

Free: skool.com/the-rabbit-hole-3206