A Http Live Streaming (HLS) Player with the Arduino Audio Tools

In traditional http streaming we can just read a single audio stream and feed it to a decoder.

Unfortunatly Http Live Streaming (HLS) adds a lot of additional complexity and overhead. It works by splitting an audiostream into very short segments, which are just a few seconds in length, and then serving these segments over HTTP or HTTPS. This list of available segments is published in a playlist file (.m3u8), which contains a URL for each recent segment. The short segments can contain MP3, AAC or very often AAC wrapped in MPEG-TS (MTS). The playlist files needs to be reloaded repeatedly to get the new segements.

My Arduino AudioTools project contains all the astractions to put toghether a HLS Player in a single Arduino Sketch using the following concepts:

• HLSStream: Audio Source, which provides the audio data as a single Arduino Stream
• MultiDecoder: Support for decoding of different audio formats
• MTSDecoder: A decoder which extracts the AAC from MPEG-TS to be fed to an AAC decoder
• Buffers and Tasks to bridge the long delays added by the overhead introduced by having to constatinly reload the playlist file to retrieve the new urls.

HLS Player Sketch

Here is a short Arduino Sketch for an ESP32 which uses PSRAM to buffer enough audio to prevent that the http loading is not breaking up the playback:

#include "AudioTools.h"
#include "AudioTools/AudioCodecs/CodecHelix.h"
#include "AudioTools/AudioCodecs/CodecMTS.h"
#include "AudioTools/AudioLibs/HLSStream.h"
#include "AudioTools/Concurrency/RTOS.h"

I2SStream out;  // audio output
BufferRTOS<uint8_t> buffer(0);
QueueStream<uint8_t> queue(buffer);
HLSStream hls_stream("ssid", "password");  // audio data source
// decoder
MP3DecoderHelix mp3;
AACDecoderHelix aac;
MTSDecoder mts(aac);             // MPEG-TS (MTS) decoder
MultiDecoder multi(hls_stream);  // MultiDecoder using mime from hls_stream
EncodedAudioOutput dec(&out, &multi);
// 2 separate copy processes
StreamCopy copier_play(dec, queue);
StreamCopy copier_write_queue(queue, hls_stream);
Task writeTask("write", 1024 * 8, 10, 0);

// Arduino Setup
void setup(void) {
  Serial.begin(115200);
  AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Info);

  // https://streams.radiomast.io/ref-128k-mp3-stereo/hls.m3u8
  // https://streams.radiomast.io/ref-128k-aaclc-stereo/hls.m3u8
  // https://streams.radiomast.io/ref-64k-heaacv1-stereo/hls.m3u8
  if (!hls_stream.begin(
          "https://streams.radiomast.io/ref-128k-mp3-stereo/hls.m3u8"))
    stop();

  // register decoders with mime types
  multi.addDecoder(mp3, "audio/mpeg");
  multi.addDecoder(aac, "audio/aac");
  multi.addDecoder(mts, "video/MP2T");  // MPEG-TS

  // start output
  auto cfg = out.defaultConfig(TX_MODE);
  // add optional configuration here: e.g. pins
  out.begin(cfg);

  buffer.resize(10 * 1024);  // increase to 50k psram
  dec.begin();               // start decoder
  queue.begin(100);          // activate read when 100% full

  writeTask.begin([]() { copier_write_queue.copy(); });
}

// Arduino loop
void loop() { copier_play.copy(); }

The MultiDecoder is used to decode mp3, aac and MPEG-TS.
We use a write task, which just copies the data from the HLSStream source to a Queue and a play task which copies the queue to the EncodedAudioOutput which performs the decoding.

Dependencies

The following libraries need to be installed

• Arduino Audio Tools
• libhelix mp3 and aac decoder

Actual Source Code

The actual, potentially updated source code can be found on Github.

Conclusion

I think it is quite impressive that we can implement such a complex solution by just composing a couple of powerful abstractions.