{"id":4620,"date":"2022-04-25T09:30:30","date_gmt":"2022-04-25T07:30:30","guid":{"rendered":"https:\/\/www.pschatzmann.ch\/home\/?p=4620"},"modified":"2024-03-21T07:54:49","modified_gmt":"2024-03-21T06:54:49","slug":"low-latency-streaming-of-audio-data","status":"publish","type":"post","link":"https:\/\/www.pschatzmann.ch\/home\/2022\/04\/25\/low-latency-streaming-of-audio-data\/","title":{"rendered":"Arduino Low Latency Streaming of Audio Data – Using TCP\/IP ?"},"content":{"rendered":"

Introduction<\/h2>\n

Traditional streaming protocols, such as RTSP and RTMP<\/strong>, support low-latency streaming. These are quite complex to implement and I did not find any efficient library which would work properly on a microcontroller e.g. on an ESP32. I tried to migrate the live555<\/a> project, but so far I did not manage to have it working properly.<\/p>\n

We could use Bluetooth<\/strong> with my A2DP library<\/a> but the lag (of more then half a second) is too big to be useful if we want to use an instrument and live playing. If you do not care about latency, it is however still an good option.<\/p>\n

Finally you can also just implement a Webserver<\/strong> which just returns audio data as demonstrated in these examples<\/a><\/p>\n

So I thought it is best to make one step back and just use regular communication protocols<\/strong> to send Audio just as raw binary data over the wire.<\/p>\n

Data Synchronization<\/h2>\n

In theory, if we send the data out in one defined rate and we process output it on the receiving system with the same rate, we should be fine. In practice however if the clocks and speeds of both systems are not synchronized, we will run into buffer overflows or underflows. It is thus far easier to send the data as fast as possible<\/strong> and just block on the receiving<\/strong> system when the buffer is full. This approach works well with all recorded data or data which is created via DSP algorithms. This is by the way also how A2DP works!
\nLatency can be controlled with the buffer size: A small buffer size leads to a low latency.<\/p>\n

Data Representation<\/h2>\n

On Microcontrollers, audio data is usually just a stream of int16_t data<\/strong>. A problem is, that different processors represent numbers in different ways: Endianness<\/strong> is the order or sequence of bytes of a word of digital data in computer memory. It is primarily expressed as big-endian<\/strong> or little-endian<\/strong>. To simplify things we assume that we just exchange data between systems with the same endianness!<\/p>\n

Protocols<\/h2>\n

TCP\/IP<\/h3>\n

This is by far the simplest solution. We can use TCP\/IP to push audio data from the sender to a receiver. If we use blocking I2S writes, the protocol will make sure that the sending is stalled when the I2S buffer is full. So we do not need to provide any specific synchronization logic and we can control the latency<\/strong> with the I2S buffer size<\/strong>.<\/p>\n