The Synthesis Toolkit (STK) Library for the Arduino ESP32 – Getting rid of Files

I made the Synthesis ToolKit (SKT) available as Arduino Library.

The Synthesis ToolKit in C++ (STK) is a set of open source audio signal processing and algorithmic synthesis classes written in the C++ programming language. STK was designed to facilitate rapid development of music synthesis and audio processing software, with an emphasis on cross-platform functionality, realtime control, ease of use, and educational example code.

One of the challenges using Microcontrollers is, that you usually don’t have a file system available. On the ESP32 we could us SPIFFS and store the files in Flash memory to make the following example work:

#include "FileLoop.h"
#include "ArdStreamOut.h"

using namespace stk;

FileLoop input;
ArdStreamOut output(Serial);

void setup() {
  Serial.begin(115200);

  input.openFile( "rawwaves/sinewave.raw", true );

  Stk::setSampleRate( 44100.0 );
  input.setFrequency( 440.0 );
}

void loop() {
  output.tick( input.tick() );
}

Some New Memory based Classes

To avoid this, I was making all raw files which are provided by the framework available as c arrays and I have added some additional classes for using them. Here is the example which uses the sound data provided as array:

#include "ArdStreamOut.h"
#include "MemoryLoop.h"
#include "MemoryFS.h"

using namespace stk;

extern const unsigned char sinewave_raw[];
extern const unsigned int sinewave_raw_len;

MemoryLoop input(new MemoryFS(sinewave_raw, sinewave_raw_len));
ArdStreamOut output(Serial);

void setup() {
  Serial.begin(115200);
  Stk::setSampleRate( 44100.0 );
  input.setFrequency( 440.0 );
}

void loop() {
  output.tick( input.tick() );
}

The MemoryFS wraps the binary data and needs a byte array and the length. This is then passed as input to the MemoryLoop class which actually is a child class of the FileLoop.

Here is the output from the Serial Plotter:

We provide the following new classes:

• MemoryWvIn similar to the FileWvIn but using an array
• MemoryLoop similar to the FileLoop but using an array
• MemoryFS Management of arrays

Sound C Files

The sound files have been generated with the help of xxd. Here is e.g. the content of the zz-raw-sinwave.c file:

 const unsigned char sinewave_raw[]  = {
  0x00, 0x00, 0x00, 0xc9, 0x01, 0x92, 0x02, 0x5b, 0x03, 0x24, 0x03, 0xed,
  0x04, 0xb6, 0x05, 0x7e, 0x06, 0x47, 0x07, 0x10, 0x07, 0xd9, 0x08, 0xa1,
  ...
  0xf8, 0x27, 0xf8, 0xf0, 0xf9, 0xb9, 0xfa, 0x82, 0xfb, 0x4a, 0xfc, 0x13,
  0xfc, 0xdc, 0xfd, 0xa5, 0xfe, 0x6e, 0xff, 0x37
};
unsigned int sinewave_raw_len = 2048;

It is a little bit tricky to process this data without the classes described above but here is a sketch that reads this data back into something meaningful:

#include "Stk.h"

using namespace stk;

extern const unsigned char sinewave_raw[];
extern const unsigned int sinewave_raw_len;

void setup() {
  Serial.begin(115200);
}

void loop() {
  int16_t value;
  for (int j=0;j<sinewave_raw_len;j+=2){
    memcpy_P(&value, &(sinewave_raw[j]), 2);
    Stk::swap16((unsigned char *)&value);
    Serial.println(value);
  }
}

We have a couple of challenges here:

• The data which is defined as const unsigned char sinewave_raw[] – should actually be represented as a int16_t array.
• Unfortunately, because the data is stored in program memory it must be read back in chunks of 4 bytes. The memcpy_P is helping us to do this!
• The ESP processor is big endian – however the data is stored as little endian, so we need to swap the bytes in the integer.

Here is the result from the Serial Plotter:

Behind the Scene

I have extended all Instruments that are using files to use this concept as well. Here is the example of the Rhodey class:

Rhodey :: Rhodey( void )
  : FM()
{
  #ifdef __VFS__
  // Concatenate the STK rawwave path to the rawwave files
  for ( unsigned int i=0; i<3; i++ )
    waves_[i] = new MemoryLoop( (Stk::rawwavePath() + "sinewave.raw").c_str(), sinewave_raw,sinewave_raw_len );
  waves_[3] = new MemoryLoop( (Stk::rawwavePath() + "fwavblnk.raw").c_str(), fwavblnk_raw,fwavblnk_raw_len );
  #else
  // Concatenate the STK rawwave path to the rawwave files
  for ( unsigned int i=0; i<3; i++ )
    waves_[i] = new FileLoop( (Stk::rawwavePath() + "sinewave.raw").c_str(), true );
  waves_[3] = new FileLoop( (Stk::rawwavePath() + "fwavblnk.raw").c_str(), true );
  #endif

If the __VFS__ has been defined in the ArdConfig.h configuration file then all instruments are using the memory based implementation and do not require any files! The option is automatically activated if you use the ESP32. You can comment out the #define __VFS__ in ArdConfig.h if you still want to work with files…

Adapting the Configuration

The configuration include file (ArdConfig.h) can be used to activate/deactivate functionality based on the environment. Here is the content for the ESP32:

#ifdef ESP32    
    #define __RTOS__ // supports Free RTOS
    #define __LWIP__  // use lwip socket implementation
    #define __ESP__   // functionality developped for esp32 and esp8266
    #define __VFS__   // raw files as compilable c arrays
    #define TASK_STACK_SIZE configMINIMAL_STACK_SIZE

    #ifdef __RTOS__  
        #include "FreeRTOS.h"
    #endif

#endif

Please note that this is pretty much work in process and some things might change here!

Conclusion

It was the goal to make also the file based instruments easily available to the ESP32. I think this goal has been achieved.

With this concept, the raw files are stored in flash memory. E.g the Wroom32 has 4MByte available and the raw files of the framework are only using around 397 KB!

In the next post I will give an overview of the new Midi functionality…