{"id":5592,"date":"2023-04-28T16:42:55","date_gmt":"2023-04-28T14:42:55","guid":{"rendered":"https:\/\/www.pschatzmann.ch\/home\/?p=5592"},"modified":"2023-05-21T11:21:11","modified_gmt":"2023-05-21T09:21:11","slug":"arduino-audio-tools-codecs-and-containers","status":"publish","type":"post","link":"https:\/\/www.pschatzmann.ch\/home\/2023\/04\/28\/arduino-audio-tools-codecs-and-containers\/","title":{"rendered":"Arduino Audio Tools: Codecs and Containers"},"content":{"rendered":"

So far I have created quite a few posts about the supported codecs<\/a>, but I never provided any detailed information how things work behind the scene. Audio codecs are used to compress and decompress PCM audio data<\/a>.<\/p>\n

I tried to eliminate all the complexity but sometimes it helps to know how things are working, so from an API point of view you can just stream audio PCM data to an encoder and stream (compressed) encoded data to a decoder to get the data in PCM format.<\/p>\n

Fixed Length vs Variable Length Codecs<\/h2>\n

Many codecs have some fixed length<\/strong> for the encoded<\/strong> frames and the corresponding decoded frames<\/strong>. This length is usually dependent on additional parameters e.g. the sampling rate. Examples are SBC, AptX or all the voice codecs like GSM, ALaw, ULaw, G7XX etc.<\/p>\n

So to make things work easily, I just needed to split the received bytes up automatically to feed the codec with the right amount of data!<\/p>\n

This works if you can make sure that you don’t miss any bytes e.g. in the beginning. And then there are some codecs that might produce some variable length. To solve this we need some audio containers.<\/p>\n

Containers<\/h2>\n

To be able to reproduce the original segments you can use a container format. To keep things simple here as well, I just provide a simple and lean custom binary container format that consists of records that start with a header with a line-feed as separator, the length and the record type. Alternatively you can also use the Ogg container.<\/p>\n

Arduino Sketch<\/h2>\n

Here is a simple Arduino sketch that demonstrates the BinaryContainerDecoder<\/strong> and BinaryContainerDecoder<\/strong> classes that can be used e.g. with an Opus<\/strong> Encoder and Decoder.<\/p>\n

#include \"AudioTools.h\"\n#include \"AudioCodecs\/ContainerBinary.h\"\n#include \"AudioCodecs\/CodecOpus.h\"\n#include \"AudioLibs\/AudioKit.h\"\n\nAudioInfo info(8000,1,16);\nSineWaveGenerator<int16_t> sineWave( 32000);  \nGeneratedSoundStream<int16_t> sound( sineWave); \nAudioKitStream out; \/\/ or any other output e.g. I2SStream, CsvStream<int16>\nEncodedAudioStream decoder(&out, new BinaryContainerDecoder(new OpusAudioDecoder())); \nEncodedAudioStream encoder(&decoder, new BinaryContainerEncoder(new OpusAudioEncoder()));StreamCopy copier(encoder, sound);     \n\nvoid setup() {\n  Serial.begin(115200);\n  AudioLogger::instance().begin(Serial, AudioLogger::Warning);\n\n  \/\/ start I2S\n  Serial.println(\"starting I2S...\");\n  auto cfgi = out.defaultConfig(TX_MODE);\n  cfgi.copyFrom(info);\n  out.begin(cfgi);\n\n  \/\/ Setup sine wave\n  sineWave.begin(info, N_B4);\n\n  \/\/ start decoder\n  decoder.begin(info);\n\n  \/\/ start encoder\n  encoder.begin(info);\n\n  Serial.println(\"Test started...\");\n}\n\n\nvoid loop() { \n  copier.copy();\n}\n<\/code><\/pre>\n
Instead of providing a codec as argument to the EncodedAudioStream we provide the container and in the constructor of the container we provide the codec to be used:<\/p>\n
In this example, we just feed a sine signal to an OpusAudioDecoder which forwards the data to the BinaryContainerDecoder to make sure that we can reconstruct the encoded format. The packaged data is then forwarded to the BinaryContainerDecoder which submits the orignal packages created by the encoder to the OpusAudioDecoder.<\/p>\n
The decoded result is finally written to the AudioKitStream!<\/p>\n
Of cause this is a simplification and you usually have some communication stream between the encoder and decoder: e.g. TCP\/IP, UDP or ESP-Now…<\/p>\n
Extended Functionality of the Binary Container<\/h2>\n