音频示例在Android 11上未按预期播放。

huangapple go评论65阅读模式
英文:

Audio sample not playing as expected on android11

问题

我正在尝试从Android 11中的视频文件中提取并播放音频样本。然而,我已经按照官方文档中的一些教程进行了操作,但仍然无法正常工作。我得到的只是音频的较短持续时间内的噪音。请检查我用于实现此操作的代码:

// 这是我获取音频样本的方式
public ArrayList<byte[]> getAudio() {
    int index = 0;
    ArrayList<byte[]> audio = new ArrayList<>();
    ByteBuffer buffer = ByteBuffer.allocateDirect(format.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE));

    for (;;) {
        int size = extractor.readSampleData(buffer, 0);
        total_size += size;
        if (size < 0) { // 没有更多数据可读取
            break;
        }

        for (int i = 0; i < size / 2; i++) {
            byte[] sample = new byte[2];
            for (int j = 0; j < sample.length; j++) {
                sample[j] = buffer.get();
            }
            audio.add(sample);
        }
        extractor.advance();
    }
    return audio;
}

Log.d("AudioTest", "音频数据总大小(字节): " + total_size);

// 这是我播放音频的方式
int SIZE = AudioTrack.getMinBufferSize(48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
AudioTrack track = new AudioTrack(AudioManager.STREAM_MUSIC, 48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, SIZE, AudioTrack.MODE_STREAM);
ArrayList<byte[]> audio = getAudio();
long start = System.currentTimeMillis();

track.play();
byte[] data = new byte[audio.size() * 2];
for (int i = 0, k = 0; i < audio.size(); i++) {
    byte[] sample = audio.get(i);
    for (int j = 0; j < sample.length; j++, k++) {
        data[k] = sample[j];
    }
}
track.write(data, 0, data.length);

long duration = System.currentTimeMillis() - start;
Log.d("AudioTest", "耗时: " + (duration / 1000) + "秒");

我遇到的问题是,我的音频仅播放了5秒的随机噪音。getAudio() 方法仅从15秒的视频文件中检索到了487,831字节。以下是视频文件的详细信息:

帧:

  • has-sdtp: 1
  • track-id: 1
  • level: 2048
  • mime: video/avc
  • frame-count: 461
  • profile: 8
  • language: und
  • color-standard: 1
  • display-width: 1920
  • track-fourcc: 828601953
  • csd-1: java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]
  • color-transfer: 3
  • durationUs: 15,382,033
  • display-height: 1080
  • width: 1920
  • color-range: 2
  • max-input-size: 3,145,748
  • frame-rate: 30
  • height: 1080
  • csd-0: java.nio.HeapByteBuffer[pos=0 lim=33 cap=33]

样本:

  • max-bitrate: 262,664
  • isDMCMMExtractor: 1
  • sample-rate: 48,000
  • track-id: 2
  • mime: audio/mp4a-latm
  • profile: 2
  • language: und
  • aac-profile: 2
  • track-fourcc: -1
  • encoder-delay: 0
  • durationUs: 15,402,666
  • channel-count: 2
  • bits-per-sample: 16
  • encoder-padding: 48
  • max-input-size: 524,308
  • csd-0: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]

任何帮助将不胜感激。

谢谢。

英文:

I am trying to extract and play audio samples from a video file in Android 11. However, I have followed some tutorials from the official documentation, but it still doesn't work. All I get is noise for a shorter duration of the audio. Please check the code I am using to achieve this:


// This is how I get the audio samples
public ArrayList&lt;byte[]&gt; getAudio() {
int index = 0;
ArrayList&lt;byte[]&gt; audio = new ArrayList&lt;&gt;();
ByteBuffer buffer = ByteBuffer.allocateDirect(format.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE));
for (;;) {
int size = extractor.readSampleData(buffer, 0);
total_size += size;
if (size &lt; 0) { // no more data to read
break;
}
for (int i = 0; i &lt; size / 2; i++) {
byte[] sample = new byte[2];
for (int j = 0; j &lt; sample.length; j++) {
sample[j] = buffer.get();
}
audio.add(sample);
}
extractor.advance();
}
return audio;
}
Log.d(&quot;AudioTest&quot;, &quot;total audio data size in bytes: &quot; + total_size);
// This is how I play the audio
int SIZE = AudioTrack.getMinBufferSize(48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
AudioTrack track = new AudioTrack(AudioManager.STREAM_MUSIC, 48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, SIZE, AudioTrack.MODE_STREAM);
ArrayList&lt;byte[]&gt; audio = getAudio();
long start = System.currentTimeMillis();
track.play();
byte[] data = new byte[audio.size() * 2];
for (int i = 0, k = 0; i &lt; audio.size(); i++) {
byte[] sample = audio.get(i);
for (int j = 0; j &lt; sample.length; j++, k++) {
data[k] = sample[j];
}
}
track.write(data, 0, data.length);
long duration = System.currentTimeMillis() - start;
Log.d(&quot;AudioTest&quot;, &quot;that took: &quot; + (duration / 1000) + &quot;s&quot;);

I am facing an issue where my audio plays random noise for only 5 seconds. The getAudio() method retrieves only 487,831 bytes from the 15-second video file. Here are the details of the video file:

Frames:

  • has-sdtp: 1
  • track-id: 1
  • level: 2048
  • mime: video/avc
  • frame-count: 461
  • profile: 8
  • language: und
  • color-standard: 1
  • display-width: 1920
  • track-fourcc: 828601953
  • csd-1: java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]
  • color-transfer: 3
  • durationUs: 15,382,033
  • display-height: 1080
  • width: 1920
  • color-range: 2
  • max-input-size: 3,145,748
  • frame-rate: 30
  • height: 1080
  • csd-0: java.nio.HeapByteBuffer[pos=0 lim=33 cap=33]

Samples:

  • max-bitrate: 262,664
  • isDMCMMExtractor: 1
  • sample-rate: 48,000
  • track-id: 2
  • mime: audio/mp4a-latm
  • profile: 2
  • language: und
  • aac-profile: 2
  • track-fourcc: -1
  • encoder-delay: 0
  • durationUs: 15,402,666
  • channel-count: 2
  • bits-per-sample: 16
  • encoder-padding: 48
  • max-input-size: 524,308
  • csd-0: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]

Any help would be highly appreciated.

Thank you.

答案1

得分: 3

音频使用高级音频编码(AAC)进行了压缩。你可以从这一行看到:mime: audio/mp4a-latm

但你没有对其进行解码。MediaExtractor 读取压缩样本,而 AudioTrack 需要解码后的数据(在你的情况下是 AudioFormat.ENCODING_PCM_16BIT)。

你可以使用MediaCodec API来解码MediaExtractor 读取的音频缓冲区。

顺便说一下,你不需要将解码后的缓冲区复制到数组中,只需将它们原样传递给 AudioTrack 即可。

或者,你可以检测正确的编码并在设置 AudioTrack 时指定它。请参见这里这里

英文:

The audio is compressed using the Advanced Audio Coding (AAC) compression. You can see it from this line:
mime: audio/mp4a-latm.

But you are not decoding it. The MediaExtractor reads compressed samples while the AudioTrack expects decoded data (AudioFormat.ENCODING_PCM_16BIT in your case).

You can use the MediaCodec API to decode the audio buffers that MediaExtractor reads.

By the way, you will not need to copy the decoded buffers into an array, just pass them to the AudioTrack as is.

Alternatively, you could detect the correct encoding and specify it when you set the AudioTrack up. See here and here.

huangapple
  • 本文由 发表于 2023年7月13日 16:35:19
  • 转载请务必保留本文链接:https://go.coder-hub.com/76677416.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定