So far I have created quite a few posts about the supported codecs, but I never provided any detailed information how things work behind the scene. Audio codecs are used to compress and decompress PCM audio data.
I tried to eliminate all the complexity but sometimes it helps to know how things are working, so from an API point of view you can just stream audio PCM data to an encoder and stream (compressed) encoded data to a decoder to get the data in PCM format.
Fixed Length vs Variable Length Codecs
Many codecs have some fixed length for the encoded frames and the corresponding decoded frames. This length is usually dependent on additional parameters e.g. the sampling rate. Examples are SBC, AptX or all the voice codecs like GSM, ALaw, ULaw, G7XX etc.
So to make things work easily, I just needed to split the received bytes up automatically to feed the codec with the right amount of data!
This works if you can make sure that you don’t miss any bytes e.g. in the beginning. And then there are some codecs that might produce some variable length. To solve this we need some audio containers.
Containers
To be able to reproduce the original segments you can use a container format. To keep things simple here as well, I just provide a simple and lean custom binary container format that consists of records that start with a header with a line-feed as separator, the length and the record type. Alternatively you can also use the Ogg container.
Arduino Sketch
Here is a simple Arduino sketch that demonstrates the BinaryContainerDecoder and BinaryContainerDecoder classes that can be used e.g. with an Opus Encoder and Decoder.
#include "AudioTools.h"
#include "AudioCodecs/ContainerBinary.h"
#include "AudioCodecs/CodecOpus.h"
#include "AudioLibs/AudioKit.h"
AudioInfo info(8000,1,16);
SineWaveGenerator<int16_t> sineWave( 32000);
GeneratedSoundStream<int16_t> sound( sineWave);
AudioKitStream out; // or any other output e.g. I2SStream, CsvStream<int16>
EncodedAudioStream decoder(&out, new BinaryContainerDecoder(new OpusAudioDecoder()));
EncodedAudioStream encoder(&decoder, new BinaryContainerEncoder(new OpusAudioEncoder()));StreamCopy copier(encoder, sound);
void setup() {
Serial.begin(115200);
AudioLogger::instance().begin(Serial, AudioLogger::Warning);
// start I2S
Serial.println("starting I2S...");
auto cfgi = out.defaultConfig(TX_MODE);
cfgi.copyFrom(info);
out.begin(cfgi);
// Setup sine wave
sineWave.begin(info, N_B4);
// start decoder
decoder.begin(info);
// start encoder
encoder.begin(info);
Serial.println("Test started...");
}
void loop() {
copier.copy();
}
Instead of providing a codec as argument to the EncodedAudioStream we provide the container and in the constructor of the container we provide the codec to be used:
In this example, we just feed a sine signal to an OpusAudioDecoder which forwards the data to the BinaryContainerDecoder to make sure that we can reconstruct the encoded format. The packaged data is then forwarded to the BinaryContainerDecoder which submits the orignal packages created by the encoder to the OpusAudioDecoder.
The decoded result is finally written to the AudioKitStream!
Of cause this is a simplification and you usually have some communication stream between the encoder and decoder: e.g. TCP/IP, UDP or ESP-Now…
Extended Functionality of the Binary Container
- You can write you own additional (meta) data in the encoder with writeMeta(const uint8_t *data, size_t len) and retrieve it in the decoder with a callback method. (See this test example)
- The binary container classes can also be used w/o any codec. This is useful if you want to distribute pure audio pcm data along with some additional custom (meta) information.
MP3, AAC Container Formats
MP3 and AAC contain their own (container) formats which are provided by the codec itself: That’s the reason why you can usually just start in the middle of an encoded stream and get perfect PCM from the decoders.
0 Comments