Down the Rabbit Hole: Publishing MP3 Files via RTSP

 

Introduction

Last week, I was asking myself the question how easy or difficult it would be to publish mp3 files via a RTSP server.

Quite some time ago, I have extended the Mico-RTSP-Audio project to be integrated with the AudioTools, so all I theoretically needed to do was using an AudioPlayer to send mp3 files to the server.

The RTSP Server

I havent looked at this functionality for years, so the first thing was to check if the sketch that is publishing a sine tone was still working. To my dismay, it was not!

I never had the need to look at the internernals of that project, so to make it work again, I needed to understand the inner workings. Here are my findings:

• It used an ESP32 timer to publish data in defined intervals
• It used a low level socket API to build the server part tighly integrated into the ESP32 functionality
• It was using a FreeRTPS task API for the session management

Well, the AudioTools is providing a processor independent timer abstraction. Tasks are also abstracted by C++ classes and Arduino provides nice high level networking classes, so I decided to redesign the functionality and integrate it into the AudioTools to support a platform independent WiFi and Ethernet implementation.

The new implementation still relies on Tasks, but I think it would not be too difficult to provide some implementation w/o it.

Here is the link to the complete class documentation of the new functionality that is available with the 2.1.0 release of the AudioTools.

The MP3 Publishing Example

The major challenge in publishing MP3 files is, that we need to extract header information from the mp3 segments e.g. to determine the timing and to split it up into individual segements that start with a synch word. I am providing the new MP3ParserEncoder class to provide this functionality.

In order to avoid any disruption that could be caused by ID3 metadata, we will filter it out with the help of a MetaDataFilterEncoder.

So first we need to include the relevant functionality:

#include "AudioTools.h"
#include "AudioTools/Disk/AudioSourceSDMMC.h" // Access to Files
#include "AudioTools/AudioCodecs/MP3Parser.h" // MP3ParserEncoder
#define USE_RTSP_LOGIN // activate WiFi login support
#include "AudioTools/Communication/RTSP.h" // RTSP Server
Code language: PHP (php)

I decided to make the WiFi login optional, so to keep the example short, I will activate it via a define. Now we are ready to define the differnt objects and link them together.

int port = 554;
const char* wifi = "SSID";
const char* password = "password";

// rtsp
MP3ParserEncoder enc; // mp3 packaging
RTSPFormatMP3 mp3format(enc); // RTSP mp3
MetaDataFilterEncoder filter(enc);
RTSPOutput<RTSPPlatformWiFi> rtsp_out(mp3format, filter);
AudioSourceSDMMC source("/", ".mp3");
CopyDecoder dec; // no decoding, just copy
AudioPlayer player(source, rtsp_out, dec);
RTSPServer<RTSPPlatformWiFi> rtsp(rtsp_out.streamer(), port);
Code language: JavaScript (javascript)

Connecting the dots: We can use an AudioPlayer that sends the data ‘as-is’ to an RTSPOutput (using a CopyDecoder) and AudioSourceSDMMC as data source. The RTSPServer gets the data from the RTSPOutput using the RTSPFormatMP3 to provide the relevant rtsp setup information and a MP3ParserEncoder via a MetaDataFilterEncoder to make the mp3 segements available as required.

In the Arduino setup we start all relevant objects:

void setup() {
  Serial.begin(115200);
  AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Warning);

  // delay between mp3 files
  source.setTimeoutAutoNext(1000);

  // start the player
  player.begin();

  // Start Output Stream
  rtsp_out.begin();

  // Start Wifi & rtsp server
  rtsp.begin(wifi, password);
}
Code language: JavaScript (javascript)

And finally the loop processing is quite short:

  if (rtsp_out && rtsp) {
      player.copy();
  }

We just send some data from the player, when the server has a connection and is ready and when the RTSPOutput queue is not full yet.

When I was testing this functionality with VLC, it was working quite nicely: but with ffplay it did not work. It took me quite some time to figure out the solution and extend the server functionality:

• some clients do not expect UDP to receive the audio data but TCP/IP
• some clients expect a 4 byte RFC2250 header before the mp3 segment.

The RTSPServer currently determines if the RFC2250 header is needed based on the client identification, but you can define this yourself in the RTSPFormatMP3 by calling the setUseRfc2250Header(bool) method.

Soruce Code

The complete (potentially updated) example can be found in the example directory of the project.

Summary and Conclusion

It turned out to be much more work since I decided to redesign the whole RTSP server to make it Platform independent. In order to support mp3, I needed to implement/extend my MP3 parser and expose the functionality via the new encoder RTSPFormatMP3 class. I finally I also needed to tweak the RTSPFormatMP3 implementation to provide the information from the parser and RFC2250 header support.