Remember the screeching of the dial-up modem? Turning data into sound is still one of the simplest ways to connect devices: you can use an audio cable, send it from a speaker to a microphone, put it on a telephone line or a radio, or just store it in a WAV file. No Wi-Fi, no pairing, no extra hardware. TinyModem is a header-only C++ library of software audio modems for Arduino and the desktop. It Read more
If you look for speech recognition examples for microcontrollers, you almost always end up with wake word detection or keyword spotting: the micro_speech example of TensorFlow Lite Micro, which recognizes “yes” and “no”; Espressif’s ESP-SR with WakeNet for the wake word and MultiNet for a list of commands; keyword spotting models trained with Edge Impulse; or Picovoice’s Porcupine and Rhino. They work well for what they do, but they all classify a small, fixed set Read more
If you have ever streamed video from an ESP32, you have most likely used the CameraWebServer example: the camera delivers JPEG images, which the web server pushes one after the other to the browser. This works, but there is no sound, the bandwidth is high, and the result is not a real video stream that you could record or play in VLC. A real stream needs a video codec such as H.264, an audio codec Read more
Classic embedded text-to-speech (formant synthesizers like SAM or eSpeak) is tiny and fast, but it sounds like a robot from the 80s. Neural TTS sounds natural, but it is usually something you run on a PC or in the cloud. I wanted to know: what is the cheapest hardware that can run a neural text-to-speech model locally, faster than real time? To find out I ported tiny-tts to C++ as TinyTTS. It is a VITS-style, Read more
Why a tiny FPGA? I spend most of my time making microcontrollers do audio, graphics and machine learning, and I keep hitting the same wall: the CPU is too slow for the heavy inner loop. So I bought a cheap Sipeed Tang Nano 20K to see how far a hobby FPGA can push past that wall. Four projects later, the answer is: surprisingly far. The board is about the size of a stick of gum Read more
To produce the most natural-sounding speech, modern Text-To-Speach (TTS) systems rely on neural networks. TinyTTS is a proof of concept: It asks whether a VITS-style neural TTS model can run on a microcontroller at all. It is a header-only C++ port of tiny-tts. The model has about 1.6M parameters. It has four stages: text encoder, flow, duration predictor and a HiFi-GAN-style vocoder. There is no inference runtime such as TFLite Micro. Everything is hand-written C++ Read more
A follow-up to AudioTools video support, which covers container/codec selection (AVI vs. MP4 vs. MPG, MJPEG vs. H.264). This post is the other half: the pipeline that actually plays a file, a minimal sketch, and every class involved. Why this exists On a desktop, “play this video” hands the problem to an OS: a media framework demuxes the container, decodes on a scheduler-managed thread, and a compositor times frames to the display’s refresh. An ESP32 Read more
I added video support to my AudioTools library: Different container formats can be used to store video on audio together in the same file. Here is a quick overivew of the provided functionality that is relevant for Microcontrollers: Container Formats The simplest and therefore most recommended container format is AVI (Audio Video Interleave). It is simple to both encode and decode and you can easily mix audio and video together. It supports both different audio Read more
Material Design is an open-source design system created by Google in 2014 to help developers and designers build consistent, high-quality digital interfaces across Android, iOS, the web, and other platforms. I did not find any existing Arduino Library which would allow me to easily implement some Material Design Applications on Arduino, so I decided to implement my own TinyMaterialDesign library. I also added a comprehensive tutorial which explains the library in detail. Further information can Read more
I started to work on extending the AudioTools library to support different video formats. There are quite a few boards with LDC display that have a built in support for audio output. The cool thing is that they also provide an SD drive, so that you have plenty of room to store audio and videos. In a first step, I tested the audio and the LCD output. Here is an overview of the boards that Read more