{"id":7123,"date":"2026-10-06T18:08:35","date_gmt":"2026-10-06T16:08:35","guid":{"rendered":"https:\/\/www.pschatzmann.ch\/home\/?p=7123"},"modified":"2026-10-06T20:24:02","modified_gmt":"2026-10-06T18:24:02","slug":"neural-tts-on-a-microcontroller","status":"publish","type":"post","link":"https:\/\/www.pschatzmann.ch\/home\/2026\/10\/06\/neural-tts-on-a-microcontroller\/","title":{"rendered":"Neural TTS on a Microcontroller ?"},"content":{"rendered":"

TinyTTS<\/a> is a proof of concept: It asks whether a VITS-style neural text-to-speech model can run on a microcontroller at all. It is a header-only C++ port of tiny-tts<\/a>. The model has about 1.6M parameters. It has four stages: text encoder, flow, duration predictor and a HiFi-GAN-style vocoder. There is no inference runtime such as TFLite Micro. Everything is hand-written C++, available as an Arduino library, an ESP-IDF component and a desktop CLI.<\/p>\n

Memory is not the problem.<\/strong> Weights, a slimmed CMU dictionary and a neural G2P fallback total 4.2 MB. After begin()<\/code>, PSRAM usage was only about 743 KB.<\/p>\n

Speed is the problem.<\/strong> Synthesizing “Hello world!” (1.49 s of audio) took:<\/p>\n\n\n\n\n\n\n\n\n\n
Platform<\/th>\nTime<\/th>\nvs. real time<\/th>\n<\/tr>\n<\/thead>\n
ESP32-S3 (baseline)<\/td>\n~440 s<\/td>\n~296x slower<\/td>\n<\/tr>\n
ESP32-S3 (INT8, SIMD, tiling)<\/td>\n~34 s<\/td>\n~23x slower<\/td>\n<\/tr>\n
ESP32-P4 (optimized)<\/td>\n~28 s<\/td>\n~19x slower<\/td>\n<\/tr>\n
Raspberry Pi 4 (4 threads)<\/td>\n~1.55 s<\/td>\n~1x<\/td>\n<\/tr>\n
Desktop<\/td>\n~0.76 s<\/td>\nfaster than real time<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The model works properly on the desktop and on microcomputers, but even the fastest ESP32 is about 20 times too slow<\/strong><\/p>\n

Where the time goes<\/h2>\n

The vocoder needs about 440 million multiply-accumulates per second of audio, roughly 90% of the model’s work. The flow accounts for most of the rest. No microcontroller CPU can do that in real time.<\/p>\n

The fix: move the heavy part to an FPGA<\/h2>\n

The TangNanoVocoder <\/a>project splits the processing up to run on a:<\/p>\n