{"id":4577,"date":"2022-04-08T12:46:08","date_gmt":"2022-04-08T10:46:08","guid":{"rendered":"https:\/\/www.pschatzmann.ch\/home\/?p=4577"},"modified":"2022-04-08T13:26:53","modified_gmt":"2022-04-08T11:26:53","slug":"an-introduction-to-audio-output-with-tensorflow-lite","status":"publish","type":"post","link":"https:\/\/www.pschatzmann.ch\/home\/2022\/04\/08\/an-introduction-to-audio-output-with-tensorflow-lite\/","title":{"rendered":"An Introduction to Arduino Audio Generated by Tensorflow Lite"},"content":{"rendered":"

The goal of this blog is to give a quick introduction into using TensorFlow Lite For Microcontrollers to create Audio Output<\/strong> with the help of the audio-tools library<\/a>.<\/p>\n

Hallo World<\/h2>\n

The starting point is the good overview provided by the “Hallo World” example of Tensorflow Lite<\/a> which describes how to create, train and use a model which based on the sine function<\/strong>.<\/p>\n

Converting into Audio<\/h2>\n

We can use this model to output the result as a tone with the help o the following sketch:<\/p>\n

#include \"AudioTools.h\"\n#include \"AudioLibs\/TfLiteAudioStream.h\"\n#include \"model.h\"\n\nTfLiteSineReader tf_reader(20000,0.3);  \/\/ Audio generation logic \nTfLiteAudioStream tf_stream;            \/\/ Audio source -> no classification so N is 0\nI2SStream i2s;                          \/\/ Audio destination\nStreamCopy copier(i2s, tf_stream);      \/\/ copy tf_stream to i2s\nint channels = 1;\nint samples_per_second = 16000;\n\n\nvoid setup() {\n  Serial.begin(115200);\n  AudioLogger::instance().begin(Serial, AudioLogger::Warning);\n\n  \/\/ Setup tensorflow input\n  auto tcfg = tf_stream.defaultConfig();\n  tcfg.channels = channels;\n  tcfg.sample_rate = samples_per_second;\n  tcfg.kTensorArenaSize = 2 * 1024;\n  tcfg.model = g_model;\n  tcfg.input = &tf_reader;\n  tf_stream.begin(tcfg);\n\n  \/\/ setup Audioi2s output\n  auto cfg = i2s.defaultConfig(TX_MODE);\n  cfg.channels = channels;\n  cfg.sample_rate = samples_per_second;\n  i2s.begin(cfg);\n\n}\n\nvoid loop() { copier.copy(); }\n\n<\/code><\/pre>\n
Like in any other audio sketch we copy the audio data from the source (TfLiteAudioStream)<\/strong> to the sink (I2SStream)<\/strong>.<\/p>\n
The TfLiteSineReader class<\/h3>\n
The heart of the processing is the TfLiteSineReader<\/strong> class which is provided by the framework and has been defined as follows<\/p>\n
class TfLiteSineReader : public TfLiteReader {\n  public: TfLiteSineReader(int16_t range=32767, float increment=0.01 ){\n    this->increment = increment;\n    this->range = range;\n  }\n\n  virtual int read(TfLiteAudioStream *parent, int16_t*data, int sampleCount) {\n    int channels = parent->config().channels;\n    float two_pi = 2 * PI;\n    \/\/ setup on first call\n    if (p_interpreter==nullptr){\n      p_interpreter = parent->interpreter();\n      input = p_interpreter->input(0);\n      output = p_interpreter->output(0);\n    }\n    for (int j=0; j<sampleCount; j+=channels){\n      \/\/ Quantize the input from floating-point to integer\n      input->data.int8[0] = TfQuantizer::quantize(actX,input->params.scale, input->params.zero_point);\n\n      \/\/ Invoke TF Model\n      TfLiteStatus invoke_status = p_interpreter->Invoke();\n      \/\/ Dequantize the output and convert it to int32 range\n      data[j] = TfQuantizer::dequantizeToNewRange(output->data.int8[0], output->params.scale, output->params.zero_point, range);\n      for (int i=1;i<channels;i++){\n          data[j+i] = data[j];\n      }\n      \/\/ Increment X\n      actX += increment;\n      if (actX>two_pi){\n        actX-=two_pi;\n      }\n    }\n\n    return sampleCount;\n  }\n<\/code><\/pre>\n
As you can see, we just provide a array of int16_t data generated by the Tensorflow Model!<\/p>\n
Summary<\/h2>\n
This is a pretty bad way to generate a sine tone and the audio tools library provides better ways to do this. However the goal was to give an simple introduction as a stepping stone…<\/p>\n
Dependencies<\/h2>\n