{"id":7167,"date":"2026-10-20T09:52:35","date_gmt":"2026-10-20T07:52:35","guid":{"rendered":"https:\/\/www.pschatzmann.ch\/home\/?p=7167"},"modified":"2026-10-20T09:58:46","modified_gmt":"2026-10-20T07:58:46","slug":"the-cheapest-way-to-run-a-neural-tts","status":"publish","type":"post","link":"https:\/\/www.pschatzmann.ch\/home\/2026\/10\/20\/the-cheapest-way-to-run-a-neural-tts\/","title":{"rendered":"The Cheapest Way to Run a Neural TTS"},"content":{"rendered":"

Classic embedded text-to-speech (formant synthesizers like SAM or eSpeak) is tiny and fast, but it sounds like a robot from the 80s. Neural TTS sounds natural, but it is usually something you run on a PC or in the cloud. I wanted to know: what is the cheapest hardware that can run a neural text-to-speech model locally, faster than real time?<\/strong><\/p>\n

To find out I ported tiny-tts<\/a> to C++ as TinyTTS<\/a>. It is a VITS-style, end-to-end model with only ~1.6 million parameters, and the port is plain hand-written C++ with no inference runtime (no TFLite, no ONNX Runtime). That makes it easy to run the exact same code on very different hardware.<\/p>\n

Here is what “Hello world!” sounds like: \u25b6\ufe0f hello_world.wav<\/a><\/p>\n

Microcontrollers: enough memory, not enough speed<\/h2>\n

My first target was the ESP32. Memory turned out not to be a problem at all: the model, a slimmed pronunciation dictionary and a small neural G2P model for unknown words need about 4.2 MB of flash, and only about 743 KB of PSRAM after startup.<\/p>\n

Speed is a different story. Even after INT8 weight quantization, weight caching and SIMD dot products, the numbers for speak(\"Hello world!\")<\/code> (1.49 s of audio) are:<\/p>\n\n\n\n\n\n\n\n
Board<\/th>\nTime<\/th>\nReal-time factor<\/th>\n<\/tr>\n<\/thead>\n
ESP32-S3<\/td>\n~33.9 s<\/td>\n~23x slower than real time<\/td>\n<\/tr>\n
ESP32-P4<\/td>\n~19.6 s<\/td>\n~13x slower than real time<\/td>\n<\/tr>\n
ESP32-P4, model trained for 16 kHz<\/td>\n~9.2 s<\/td>\n~8x slower than real time<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

It works, and it is a nice proof of concept, but you would not want to wait 20 seconds for your doorbell to say “Hello”.<\/p>\n

Where does the time go?<\/h2>\n

Profiling shows the same picture on every CPU: the decoder<\/strong> (a HiFi-GAN-style vocoder that turns the spectral representation into audio samples) takes 87-90% of the time. Most of the rest goes to the flow<\/strong>. The text encoder, the duration predictor and the grapheme-to-phoneme conversion are cheap.<\/p>\n

So the question is not “which CPU is fast enough?” but “which cheap board has something that can do convolutions in parallel?” That points to a GPU or an NPU.<\/p>\n

The winner: Milk-V Duo 256M<\/h2>\n

The Milk-V Duo 256M<\/a> is a tiny Linux board around the size of a Raspberry Pi Pico. It sells for roughly ten dollars, and its SG2002 SoC has:<\/p>\n