Needle: An 8-29 MB On-Device "Automation Foundation Model" for Tool Calls and Structured Extraction on Phones, Wearables and Microcontrollers

Needle is a very small open-source model from the Cactus Compute team: the whole model is one binary of 8 to 29 MB, aimed at phones, watches and wearables, robots, smart homes, cars and even microcontrollers. The project calls it an "automation foundation model" and says it traded general chat capacity for results: it claims to beat models 10x its size on mobile tool calls and match models 2 to 3x bigger on extraction. Those comparisons come from the README, and we saw no third-party replication, so read them as the project's own claims. As of October 13, 2026 the repository has about 13,887 stars and 940 forks, an Apache-2.0 license, mostly Python code, and a last push on October 11. What it does is specific. First, tool calls: given the functions your app exposes, Needle picks the right ones and fills every argument from what the user said. Ask for two things and you get two calls in order; ask for something no tool covers and you get an empty list, not a guess. Second, structured extraction: declare a shape, such as an invoice, booking, notification or form, hand over messy text, and get typed fields back, with a decode grammar that guarantees the output parses; extraction also generalizes to classification. Third, text embeddings: the same model returns a vector for a sentence, so an app can search, match and route locally. Fourth, its sibling Whistle is a 16.9 MB speech-to-text model that shares Needle's container and engine, so one build gives Needle audio input: the clip goes in, the tool calls come out.

The entry barrier is low. After pip install cactus-needle, you decorate a Python function as a tool: the signature gives the argument types and the docstring is the tool description, and agent.run completes the loop by executing the function and returning results. Every turn returns one JSON object with function_calls, the model's reasoning and a calibrated confidence, so an app can choose to act, confirm or refuse by confidence. To deploy to a device, a command such as needle build --platform macos-arm64 fetches the prebuilt engine and puts the needle3.cact weights beside it. The repository lists macOS and Linux ARM64 targets and has notes for browser and WASI builds. The architecture is what is really different. Needle 3 is a Laddered Simple Attention Network: a Monarch Hadamard MLP in place of the feed-forward layer, grouped-query attention with causal convolution taps, an engram n-gram memory read by gather, and multi-lane hyper-connections, trained so that every depth from 2 to 20 layers is a deployable model. Most parameters sit in the engram, so the 121M model does the arithmetic of a 50M one. A byte-level grammar compiled from your schemas constrains every token, and every response carries a calibrated confidence score. The logic behind the design: on devices with only tens of megabytes, spending capacity on lookup-style memory beats spending it on general reasoning.

Fine-tuning is the other emphasis. The README says fine-tuning on DroidCall lifts every subnetwork by 18 to 36 points, and from 4 layers and 29M parameters the tuned subnetwork passes DeepSeek V4 Flash. Local fine-tuning trains LoRA adapters on the attention projections with the base frozen, merged at export, at 4-bit precision, on your own machine with JAX. Platform fine-tuning trains the full model at 2-bit, needs a Cactus API key, and can reinforce your data with Needle's original dataset so that nothing already learned is unlearned. For Whistle, the numbers are 8.6x smaller than Whisper base and 6.6x quicker to the first token, ahead on LibriSpeech, SPGISpeech, Earnings-22 and the FLEURS average, and behind on TED-LIUM, AMI and the MLS average, across English, German, French, Spanish, Italian, Dutch and Polish.

Our reading is that Needle turns a very specific need into a very small product. If your app already has a clear set of functions and needs to map language or speech to calls reliably, it is much cheaper than deploying a large model and works offline. The risks are specific too. All benchmarks come from the project's own README. Telemetry is on by default in the binary, so privacy-sensitive uses must set NEEDLE_TELEMETRY=0 and DO_NOT_TRACK=1. And it does not chat or do open-domain reasoning; if your function set changes often, weigh the cost of fine-tuning and data generation. Run a small evaluation with your real tool list and real spoken samples before replacing cloud calls.