{"id":6725,"date":"2025-06-05T12:38:47","date_gmt":"2025-06-05T10:38:47","guid":{"rendered":"https:\/\/www.pschatzmann.ch\/home\/?p=6725"},"modified":"2025-06-09T09:47:39","modified_gmt":"2025-06-09T07:47:39","slug":"vibe-coding-adding-m4a-audio-playback-support-to-the-audiotools","status":"publish","type":"post","link":"https:\/\/www.pschatzmann.ch\/home\/2025\/06\/05\/vibe-coding-adding-m4a-audio-playback-support-to-the-audiotools\/","title":{"rendered":"Vibe Coding: Adding M4A Audio Playback Support to the AudioTools"},"content":{"rendered":"

Last week I was asked, if my Arduino Audio Tools<\/a> project was supporting the playback of M4A files<\/strong>, which it didn’t: but I thought that would be quite a good opportunity to try out Vibe Coding<\/strong>,\u00a0 since this problem space is well documented and quite established it seemed to be quite a good fit. Needless to say that at that point of time I did not know anything about M4A<\/strong>. So I just asked Claude to generate me an audio file extractor class for AAC which follows my Container API.<\/p>\n

And it did generate quite a lot of detailed code<\/strong> that at first glace was looking very promising. But running some tests proved that it did not work and after looking at the code and the execution logic in the debugger I came to the concolusion that this is hopeless and impossible to fix<\/strong>!<\/p>\n

Then I had quite some detailed discussion with ChatGPT which was leading me to a better understanding what needs to be done:<\/p>\n

    \n
  1. Parse the MP4 container<\/strong> format and handle the relevant boxes only<\/li>\n
  2. Build a sample table<\/strong> which was giving the record sizes to be played back<\/li>\n
  3. For AAC<\/strong> get the profile, sample rate index and channel index and with this build and add an adts header before the samples<\/li>\n
  4. For ALAC<\/strong> data extract the magic cookie and provide it to the decoder<\/li>\n
  5. Return the audio data which is contained in the mdat box so that it can be sent to a decoder.<\/li>\n<\/ol>\n

    The M4A Audio Format<\/h3>\n

    The M4A audio format<\/strong> is based on the MP4 container<\/strong> structure, which organizes data into a hierarchy of boxes<\/em>. Each box begins with a 4-byte big-endian size field, followed by a 4-byte type identifier. Container boxes can nest other boxes and typically contain no data of their own, although exceptions exist. In contrast, non-container boxes hold actual data.<\/p>\n

    Audio samples are stored in the mdat<\/code> box, but to extract them correctly, information from the stsz<\/code> box is required. The stsz<\/code> box indicates the total number of samples and either a fixed sample size or, if the size is zero, a table of individual sample sizes. This data is essential to determine how many bytes to read from mdat<\/code> for each sample. Additionally, the child boxes within the stsd<\/code> box reveal the audio format\u2014such as alac<\/code> (Apple Lossless), mp4a<\/code> (AAC), or mp3<\/code>.<\/p>\n

     <\/p>\n

    How to eat an elephant: cut him up into little pieces!<\/em><\/strong><\/p>\n

     <\/p>\n

    So, I decided to split up the problem into different C++classes:<\/p>\n

    The MP4 Parser<\/h3>\n

    I expected to be able to find an existing simple, lean and tested MP4 parser <\/strong>which would run on a Microcontroller that provides a simple callback API where I can just plug in some callbacks with the relevant logic, and to my big surprise I found nothing<\/strong>!<\/p>\n

    So I was forced do this myself<\/strong> and I wrote and tested my MP4Parser<\/a> class which, to keep things simple, expects each box to fit into the actual RAM buffer. Here I had quite some surprises: In some rare cases it did not find a valid box at the expected location, so I added some fallback error handling that just scans for the next box and reports the skiped data under the last box type.<\/p>\n

    As a next step, I was extending this with a MP4ParserIncremental class which removes this restriction and can provide the box content incrementally<\/strong> when the buffer is too small. After that functionality was tested, I integrated it back into the MP4Parser.<\/p>\n

    This was quit simple to test: just execute the test sketch and check if you find the boxes at the indicated locations!<\/em><\/p>\n

    Extracting the Audio Data<\/h3>\n

    Next I decided to have a separate M4AAudioDemuxer<\/a> class which implements all the necessary parsing data callbacks<\/strong> and data extraction logic and provides the the result as frame<\/a> entries in a callback. This class basically<\/p>\n