英文:

Realtime conversion of MP3 to PCM using minimp3 produces noise/gaps at beginning of frame

问题

I'm using minimp3 (https://github.com/lieff/minimp3) to implement an audio pipeline where I stream MP3 data to an Unreal Engine 5 client using Web Sockets. Once received, I add the data to a TArray (acting as a buffer), and then decode as much as I can to PCM. Once decoded I add the new samples to an audio source. I then repeat this with all MP3 frames in the data stream.

我正在使用minimp3 (https://github.com/lieff/minimp3) 来实现一个音频流程，通过Web Sockets将MP3数据传送到虚幻引擎5客户端。一旦接收到数据，我将数据添加到一个TArray中（作为缓冲区），然后尽可能多地解码成PCM格式。解码后，我将新的样本添加到音频源。然后我会在数据流中的所有MP3帧上重复这个过程。

I've confirmed that the MP3 is being transmitted correctly, however, the converted PCM has gaps/noise at the beginning of chunks of MP3 frames that distort the audio during playback.

我已经确认MP3正在正确传输，但是转换后的PCM在MP3帧的开始处有间隙/噪音，在播放过程中扭曲了音频。

The noise/gaps seem to occur at the beginning of new chunks. My initial theory was that it's due to the byte reservoir system of MP3 (given that the gap is roughly 3 frames long every time), but I thought minimp3 is supposed to account for that? It's the same instance of mp3d between chunks.

噪音/间隙似乎出现在新块的开头。我的最初理论是这是由于MP3的字节储存系统引起的（考虑到每次间隙大约有3帧长），但我以为minimp3应该会处理这个问题？每个块之间都是同一个mp3d实例。

What do I need to do to fix the decoding issues at the beginning of the chunks?

我需要做什么来解决块的开头解码问题？

英文:

I'm using minimp3 (https://github.com/lieff/minimp3) to implement an audio pipeline where I stream MP3 data to an Unreal Engine 5 client using Web Sockets. Once received, I add the data to a TArray (acting as a buffer), and then decode as much as I can to PCM. Once decoded I add the new samples to an audio source. I then repeat this with all MP3 frames in the data stream.

I've confirmed that the MP3 is being transmitted correctly, however, the converted PCM has gaps/noise at the beginning of chunks of MP3 frames that distort the audio during playback.

Attached Code:

https://gist.github.com/DeveloperHarris/c3bd30d6cc689f22ec9e8fc839c80655

The top audio in Audacity is a test MP3 comprised of a 100Hz sine wave received by the Unreal Engine client.
The bottom track is the converted PCM stream.

The noise/gaps seem to occur at the beginning of new chunks. My initial theory was that it's due to the byte reservoir system of MP3 (given that the gap is roughly 3 frames long every time), but I thought minimp3 is supposed to account for that? It's the same instance of mp3d between chunks.

What do I need to do to fix the decoding issues at the beginning of the chunks?

答案1

得分: 1

Turns out, I needed to read the library docs more carefully:

The decoder will analyze the input buffer to properly sync with the MP3 stream, and will skip ID3 data, as well as any data which is not valid. Short buffers may cause false sync and can produce 'squealing' artefacts. The bigger the size of the input buffer, the more reliable the sync procedure. We recommend having as many as 10 consecutive MP3 frames (~16KB) in the input buffer at a time.

Adding a check at the beginning of my decode function that checks if it has 10 consecutive frames was enough to fix the issue.

英文:

Turns out, I needed to read the library docs more carefully:

The decoder will analyze the input buffer to properly sync with the MP3 stream, and will skip ID3 data, as well as any data which is not valid. Short buffers may cause false sync and can produce 'squealing' artefacts. The bigger the size of the input buffer, the more reliable the sync procedure. We recommend having as many as 10 consecutive MP3 frames (~16KB) in the input buffer at a time.

Adding a check at the beginning of my decode function that checks if it has 10 consecutive frames was enough to fix the issue.