Run a 14MB AI on a Raspberry Pi (Needle2 setup)

Needle2 is a 45M parameter model that ships as about 14MB, and a full session runs in roughly 28MB of RAM. It is not a chatbot. It does tool calling, device control, and pulling structured data out of messy text. That narrowness is the whole reason it fits on hardware this small.

On a Raspberry Pi 5 it decodes around 500 tokens a second. On sub-$200 phones, 300 to 700. On VR headsets, 400 to 1,500.

Install

On a Mac:

brew install cactus-compute/cactus/cactus

Everywhere else:

git clone https://github.com/cactus-compute/cactus && cd cactus
source ./setup

Run it

cactus run Cactus-Compute/needle

With your own tools:

cactus run Cactus-Compute/needle --tools my_tools.json

Tool definitions use the OpenAI function-calling format, so if you have written tools for GPT before, the same JSON works here. It ships with a demo toolset if you just want to watch it work first.

Putting it in something

There are bindings for Swift, Kotlin, Flutter, React Native, Python and Rust, so the model can sit inside an app rather than a terminal. The weights live under the Cactus-Compute organisation on Hugging Face, Apache 2.0.

What to actually build with it

The useful framing is not "a small ChatGPT". It's the layer that turns a sentence into a function call, on a device with no internet.

• A light switch that understands "it's too bright in here"
• A wearable that turns a mumbled note into {title, due_date, tags}
• A robot that maps spoken commands onto the functions it already has
• Anything on a Pi, an ESP32-S3, or a Quest where sending audio to a server is not an option

Where it will disappoint you

It will not answer questions. 45M parameters holds almost no world knowledge. Ask it the capital of France and you are using the wrong tool.

It needs your tools to be good. The model is only as useful as the function definitions you give it. Vague tool descriptions produce vague calls, and at this size it cannot reason its way out of a bad spec.

Source

cactuscompute.com/needle and github.com/cactus-compute/cactus, Apache 2.0.

---

What device do you want to put it on?

Ask me in The Rabbit Hole and I'll tell you if it fits.

It is the room where I keep the parts that don't fit on a page like this: the full run of numbers, what broke the first time, and the ones that looked good and then didn't work at all.

Two catches, up front, because that's the rule here. It's private, so you have to be inside to read anything. And it's new, so you'd be early rather than joining something busy.

Free: skool.com/the-rabbit-hole-3206