Overview
I was playing with the idea to implement a simple Synthesizer for the ESP32.which would play different instruments. The ESP32 should have enough power to generate interesting sounds. The limitation however is on the memory. So any approach which relies on sampled data and big sound buffers will just not work. Therefore we need to rely on playing the generated sounds as directly as possible.
A Synthesizer is an electronic musical instrument that generates audio signals.
There are plenty of different implementation approaches
- Hardware Devices (e.g. Roland JD-XA)
- Software applications (or plugins) with a powerful GUI (e.g. Garage Band)
- Audio Programming Languages (e.g. Sonic PI, see Wikipedia for a complete list)
- Web Browser based Synthesizers
To play the software instruments a MIDI keyboard is usually connected with a MIDI cable to the computer which runs the software. The WebMIDI API brings all the functionality of MIDI to the browser so that it is possible to provide many instruments without installing any software – I think this is pretty cool! Examples can be found at
The Evolution of Synthesizers
The first simple synthesizers were based on generators for shaped sounds which were adjusted by filters and envelopes were used to make the sound more dynamic. The next evolution were FM Synthesizers which were based on the concept of modulating different frequencies to generate interesting sounds.
Finally the concept of sampling was added to make it possible to add the recorded sound of real instruments on one hand or basically any type of sound on the other hand in order to generate music.
DSP Libraries
Naturally the libraries depend on your target programming language. Because I wanted to have the best flexibility and performance I concentrated on C++ based Digital Signal Processing (DSP) libraries. Here is my curated list:
- The Synthesis ToolKit in C++ STK is a set of open source audio signal processing and algorithmic synthesis classes written in the C++ programming language. STK was designed to facilitate rapid development of music synthesis and audio processing software, with an emphasis on cross-platform functionality, realtime control, ease of use, and educational example code.instruments. This library is lightweight and portable so it should be useful for our purpose.
- Maximilan: is an open source, MIT licensed C++ audio synthesis library. It’s designed to be cross platform and simple to use. The syntax and program structure are based on the popular ‘Processing’ environment. Maximilian provides standard waveforms, envelopes, sample playback, resonant filters, and delay lines. In addition, equal power stereo, quadraphonic and 8-channel ambisonic support is included. There’s also Granular synthesisers with time stretching, FFTs and some Music Information Retrieval stuff.
- gamma is a cross-platform, C++ library for doing generic synthesis and filtering of signals. It is oriented towards real-time sound and graphics applications, but is equally useful for non-real-time tasks. Gamma is designed to be “light-footed” in terms of memory and processing making it highly suitable for plug-in development or embedding in other C++ projects.
- yummyDSP is a simple digital sound processing library for the Arduino which can generate sounds. This library however is very limited in its functionality.
Musicdsp.org provides a collection of algorithms, thoughts and snippets, gathered for the music DSP community. If you target to develop some platform independent plugins you might also consider the following:
MIDI Arduino Libraries
Some of the described libraries provide MIDI functionality. Unfortunately this will not work in Arduino environment. Here is an overview of the relevant Arduino libraries which are available to build the solution:
- Fortunately there exists already a Midi library for the Arduino which helps in reading the MIDI input and an implementation
- RTP-MIDI provides Midi functionality over the network..
Playing Sound
Most of the described libraries are issuing the output with the help of an abstraction layer that is supported by multiple platforms (e.g RTAudio, PortAudio etc). This will be perfect for the Raspberry PI – Unfortunately none of these will be working on the ESP32 and the major challenge is to provide a way to use the the available I2S interface:
In one of my earlier posts I gave an overview of how to play music on a ESP32. The ESP32 can generate sound with the help of the I2S interface which is described in the ESP32 reference. So all we need to to is feed the I2S interface with sound data!
The sound data will be generated by the STK library based on the MIDI input. So in a nutshell this sounds to be pretty straight forward!
Next Steps
In my next posts I plan to provide an simple implementation of a Synthesizer for the Raspberry PI Zero and the ESP32.
3 Comments
Erich · 15. May 2021 at 13:15
Hi, that sounds all very well. In the meanwhile, another developer created already a lot of things, take a look at the development and videos of Marcel Licence – at Youtube and Github. He created something like a small series of Sound-Devices based on the ESP32 or ESP32 A1S Audio-Board. A Synth, a sampler, a looper, a drum-computer – and a very cool sounding Organ emulation – only the organ is not published on github, all the other things are well publieshed and the code is readable.
pschatzmann · 15. May 2021 at 14:42
Thanks for your feed back!
I think https://github.com/sensorium/Mozzi might be also of interest.
icouldbeyu · 15. September 2020 at 0:30
Any interest in resuming this series?! I’m interested in how I could build a synth with the ESP32 and this seems like a great series you were in the midst of putting together!