> ## Content Index > Fetch the complete content index at: https://www.notatechguy.com/llms.txt > Use this file to discover other available public pages before exploring further. # Needle: 8 MB AI model runs on phones and microcontrollers - URL: https://www.notatechguy.com/needle-8-mb-ai-model-runs-on-phones-and-microcontrollers/ - Published: 2026-09-20T20:03:31.000Z - Updated: 2026-09-20T20:03:31.000Z - Description: An open-source 8-29 MB model claims to beat 10x larger models on tool calls, targeting phones, wearables and microcontrollers. - Author: Marcello Babbili - Tags: Technology & AI Cactus Compute built Needle, an AI model that packs tool-call capability into 8-29 MB, and the project just landed on GitHub's trending list with +234 stars and 11,736 total [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com)[P²](https://huggingface.co/Cactus-Compute/needle3?ref=notatechguy.com). The maintainers claim it outperforms models 10 times larger on mobile tool calls, but they name no competitors and publish no independent benchmarks [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com) — the 10x figure is the vendor's own, with no error bars and no released training code [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). **My read:** This is the most aggressive size-to-capability ratio I have seen for an on-device model. The architecture details, the "engram n-gram memory" and "Monarch Hadamard MLP," read like real engineering rather than a wrapper repo. But the "beats models 10x its size" claim cites relative sizes, not named models, and the only sources are the maintainers' own GitHub page [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com) and Hugging Face listing [P²](https://huggingface.co/Cactus-Compute/needle3?ref=notatechguy.com). I would want to see exact-match accuracy numbers against a named model like Phi-3 or Gemma before I took the 10x claim seriously. The star velocity, 234 in a day, tells you developers are curious, not that the benchmarks hold. The previous version, Needle 2, was a 45-million-parameter model in a 14 MB binary that ran in about 28 MB of RAM [P⁴](https://github.com/cactus-compute/needle/blob/main/README.md?ref=notatechguy.com). The current Needle 3 stretches the range from 8 to 29 MB depending on configuration, with 121 million parameters [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). The maintainers say most of those parameters sit in an "engram" n-gram memory bank that is looked up rather than multiplied, so the 121M model does the arithmetic of a 50-million-parameter network [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). ![Needle model parameters vs effective arithmetic](https://storage.ghost.io/c/6e/89/6e896869-22ef-4281-a213-b4c462c17cff/content/images/2026/09/chart_0a26c19f05aee24c36f2.png) ### Monarch Hadamard MLP and grammar-constrained outputs Needle 3 swaps the conventional feed-forward network for a Monarch Hadamard MLP and pairs grouped-query attention with causal convolution taps [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). Each layer depth from 2 to 20 functions as a standalone deployable model, letting a developer cut the network short to trade accuracy for speed without retraining [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). A byte-level grammar compiled from the user's own schemas constrains every output, so a tool call in JSON should be syntactically valid by construction [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). Each response also carries a calibrated confidence score from a learned head, allowing an app to fall back to a cloud model when the on-device result is uncertain [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). None of this has been peer-reviewed, no error bars are reported, and no training code has been released — the capability figures are the vendor's own [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com). The benchmarks are described as exact-match accuracy for tool calling and field micro-F1 for extraction on full test splits [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com), but no external party has run them. No phone, car, or wearable manufacturer has confirmed deployment. ### Smart-home and embedded developers get a local tool-call option A developer building a smart-home app that parses voice commands into thermostat actions without sending audio to the cloud could test Needle today. The schema-constrained generation would keep the output valid, and the confidence score would gate when to escalate to a server-side model. The model targets phones, wearables, smart homes, robots, cars, and microcontrollers [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com)[P²](https://huggingface.co/Cactus-Compute/needle3?ref=notatechguy.com), though the microcontroller claim, 8 MB on a device with 28 MB of RAM, puts it in the range of higher-end embedded chips rather than the smallest Arduino-class boards. For an embedded firmware engineer who currently ships a hand-maintained rule-based parser on a Cortex-M7 board, Needle would replace that state machine with a model that generates structured tool calls in roughly 28 MB of RAM, trading deterministic logic for probabilistic output that needs confidence-score gating [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com)[P⁴](https://github.com/cactus-compute/needle/blob/main/README.md?ref=notatechguy.com). This fits a broader compression trend. But pushing AI onto microcontrollers also expands the attack surface. An on-device model that makes tool calls could act on the device it runs on if compromised. The Hugging Face model page [P²](https://huggingface.co/Cactus-Compute/needle3?ref=notatechguy.com) and the GitHub repository [S¹](https://github.com/cactus-compute/needle?ref=notatechguy.com) are the places to check for benchmark tables and code. The maintainers have not announced a date for independently audited results. --- *Sources: [S1 — cactus-compute/needle: Automation foundation model for tiny devices: 2](https://github.com/cactus-compute/needle?ref=notatechguy.com) · [P2 — Cactus-Compute/needle3 · Hugging Face](https://huggingface.co/Cactus-Compute/needle3?ref=notatechguy.com) · [P3 — mariklolik/vibe](https://github.com/mariklolik/vibe?ref=notatechguy.com) · [P4 — README.md](https://github.com/cactus-compute/needle/blob/main/README.md?ref=notatechguy.com) · [P5 — MokshithRao/ai-research-assistant](https://github.com/MokshithRao/ai-research-assistant?ref=notatechguy.com)* --- *Written from 5 sourced items, 4 of them primary.* ## More from Not A Tech Guy - [OpenAI launches Australian Youth Safety Blueprint for teens](https://www.notatechguy.com/openai-launches-australian-youth-safety-blueprint-for-teens/) - [Claude Code TradingView bridge hits 6,528 GitHub stars](https://www.notatechguy.com/claude-code-tradingview-bridge-hits-6-528-github-stars/) - [Together AI score centering stabilizes RL for LLMs up to 30B](https://www.notatechguy.com/together-ai-score-centering-stabilizes-rl-for-llms-up-to-30b/)