Engineering · July 31, 2026

Generative inference on a $60 board: Gear-1M on the STM32H747

The smallest member of the Gear family generates text at 20 tokens per second on an ARM Cortex-M7 — no NPU, no external RAM, no cloud. This is how a language model fits where a language model isn't supposed to fit.

By the OptAI research team

Key points

20.0 tok/s measured on an STM32H747I-DISCO — Cortex-M7 @ 400 MHz, 50 ms per token.
1.59 MB of flash holds the entire binary — weights, runtime and application — with 449 KB peak RAM.
W4A32 quantization: 4-bit weights, full-precision activations, chosen for a core with no vector int8 path.
No KV-cache in the local path: the same ConvKV-Gated Mixer that gives the larger models their 64K window is what makes this budget possible.

Two horizontal budget bars. Flash: 1.59 megabytes used of 2.0 available. SRAM: 449 kilobytes peak of 1.0 megabyte available. — The whole deployment story in two bars: the binary and its working set fit the board with room left over.

The constraint

The STM32H747I-DISCO is a standard evaluation board: a Cortex-M7 at 400 MHz, 2 MB of flash, 1 MB of SRAM, and a small LCD. It costs about $60 and it is the kind of part that already sits inside appliances, meters, and industrial controllers by the million.

Nothing about that budget is negotiable. There is no swap, no external DRAM, and no accelerator to offload to. If the model, the runtime, and the application don't fit in 2 MB of flash and 1 MB of RAM together, the exercise is over.

What fits in a megabyte of weights

5 layers, 128 hidden — the same hybrid layout as Gear-1B, scaled to the budget.
A 2,048-entry BPE tokenizer — sized for command-and-control language, not Wikipedia.
W4A32 — 4-bit weights unpacked into full-precision math, because a Cortex-M7 has an FPU but no int8 vector unit worth building around.
L_conv = 4 — the convolutional state is one position longer than in the larger models, a cheap trade at this scale.

Why the KV-cache had to go

A conventional transformer spends RAM on its KV-cache in proportion to how long the conversation gets — which is exactly the resource this board doesn't have. Gear's local layers hold a fixed 4-position convolutional state instead, no matter the context length. Decode-time memory stops being a function of the prompt, which is the property that lets the same architecture stretch from this board up to a 64K-context flagship.

Per-layer decode state plotted against context length: global attention climbing, sliding-window attention flat at 1,024, and the ConvKV mixer flat along the baseline. — Per-layer decode state vs. context length. The violet baseline — the ConvKV-Gated Mixer — is the line the microcontroller lives on.

What it does

Gear-1M maps natural language to structured board-control intents. The demo below is the exact workload we profile — the board parses the request and emits typed JSON that the firmware executes:

$ gear run --model gear-1m-w4a32
> turn on the blue led and write hello on the lcd
{ "action": "led_set", "led": "blue", "state": "on" }
{ "action": "lcd_write", "text": "hello" }

At 20 tokens per second, a response like that completes in about a second and a half — fast enough to feel conversational on a device with no network stack at all.

Measured, not projected

20.0 tok/sdecode, Cortex-M7 @ 400 MHz

50 msper token, W4A32

1.59 MB.text — the whole binary

449 KBpeak RAM

All four numbers were read off the device, not estimated from a server run. The profiling methodology — and the same treatment for the 270M and 1B models on mobile NPUs — ships with the technical report.

Weights and the deployment binary are on Hugging Face, and the rest of the family is on the models page. If you have a board we haven't ported to, tell us about it.