英文:
Audio sample not playing as expected on android11
问题
我正在尝试从Android 11中的视频文件中提取并播放音频样本。然而,我已经按照官方文档中的一些教程进行了操作,但仍然无法正常工作。我得到的只是音频的较短持续时间内的噪音。请检查我用于实现此操作的代码:
// 这是我获取音频样本的方式
public ArrayList<byte[]> getAudio() {
int index = 0;
ArrayList<byte[]> audio = new ArrayList<>();
ByteBuffer buffer = ByteBuffer.allocateDirect(format.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE));
for (;;) {
int size = extractor.readSampleData(buffer, 0);
total_size += size;
if (size < 0) { // 没有更多数据可读取
break;
}
for (int i = 0; i < size / 2; i++) {
byte[] sample = new byte[2];
for (int j = 0; j < sample.length; j++) {
sample[j] = buffer.get();
}
audio.add(sample);
}
extractor.advance();
}
return audio;
}
Log.d("AudioTest", "音频数据总大小(字节): " + total_size);
// 这是我播放音频的方式
int SIZE = AudioTrack.getMinBufferSize(48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
AudioTrack track = new AudioTrack(AudioManager.STREAM_MUSIC, 48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, SIZE, AudioTrack.MODE_STREAM);
ArrayList<byte[]> audio = getAudio();
long start = System.currentTimeMillis();
track.play();
byte[] data = new byte[audio.size() * 2];
for (int i = 0, k = 0; i < audio.size(); i++) {
byte[] sample = audio.get(i);
for (int j = 0; j < sample.length; j++, k++) {
data[k] = sample[j];
}
}
track.write(data, 0, data.length);
long duration = System.currentTimeMillis() - start;
Log.d("AudioTest", "耗时: " + (duration / 1000) + "秒");
我遇到的问题是,我的音频仅播放了5秒的随机噪音。getAudio()
方法仅从15秒的视频文件中检索到了487,831字节。以下是视频文件的详细信息:
帧:
- has-sdtp: 1
- track-id: 1
- level: 2048
- mime: video/avc
- frame-count: 461
- profile: 8
- language: und
- color-standard: 1
- display-width: 1920
- track-fourcc: 828601953
- csd-1: java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]
- color-transfer: 3
- durationUs: 15,382,033
- display-height: 1080
- width: 1920
- color-range: 2
- max-input-size: 3,145,748
- frame-rate: 30
- height: 1080
- csd-0: java.nio.HeapByteBuffer[pos=0 lim=33 cap=33]
样本:
- max-bitrate: 262,664
- isDMCMMExtractor: 1
- sample-rate: 48,000
- track-id: 2
- mime: audio/mp4a-latm
- profile: 2
- language: und
- aac-profile: 2
- track-fourcc: -1
- encoder-delay: 0
- durationUs: 15,402,666
- channel-count: 2
- bits-per-sample: 16
- encoder-padding: 48
- max-input-size: 524,308
- csd-0: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]
任何帮助将不胜感激。
谢谢。
英文:
I am trying to extract and play audio samples from a video file in Android 11. However, I have followed some tutorials from the official documentation, but it still doesn't work. All I get is noise for a shorter duration of the audio. Please check the code I am using to achieve this:
// This is how I get the audio samples
public ArrayList<byte[]> getAudio() {
int index = 0;
ArrayList<byte[]> audio = new ArrayList<>();
ByteBuffer buffer = ByteBuffer.allocateDirect(format.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE));
for (;;) {
int size = extractor.readSampleData(buffer, 0);
total_size += size;
if (size < 0) { // no more data to read
break;
}
for (int i = 0; i < size / 2; i++) {
byte[] sample = new byte[2];
for (int j = 0; j < sample.length; j++) {
sample[j] = buffer.get();
}
audio.add(sample);
}
extractor.advance();
}
return audio;
}
Log.d("AudioTest", "total audio data size in bytes: " + total_size);
// This is how I play the audio
int SIZE = AudioTrack.getMinBufferSize(48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
AudioTrack track = new AudioTrack(AudioManager.STREAM_MUSIC, 48000, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, SIZE, AudioTrack.MODE_STREAM);
ArrayList<byte[]> audio = getAudio();
long start = System.currentTimeMillis();
track.play();
byte[] data = new byte[audio.size() * 2];
for (int i = 0, k = 0; i < audio.size(); i++) {
byte[] sample = audio.get(i);
for (int j = 0; j < sample.length; j++, k++) {
data[k] = sample[j];
}
}
track.write(data, 0, data.length);
long duration = System.currentTimeMillis() - start;
Log.d("AudioTest", "that took: " + (duration / 1000) + "s");
I am facing an issue where my audio plays random noise for only 5 seconds. The getAudio()
method retrieves only 487,831 bytes from the 15-second video file. Here are the details of the video file:
Frames:
- has-sdtp: 1
- track-id: 1
- level: 2048
- mime: video/avc
- frame-count: 461
- profile: 8
- language: und
- color-standard: 1
- display-width: 1920
- track-fourcc: 828601953
- csd-1: java.nio.HeapByteBuffer[pos=0 lim=9 cap=9]
- color-transfer: 3
- durationUs: 15,382,033
- display-height: 1080
- width: 1920
- color-range: 2
- max-input-size: 3,145,748
- frame-rate: 30
- height: 1080
- csd-0: java.nio.HeapByteBuffer[pos=0 lim=33 cap=33]
Samples:
- max-bitrate: 262,664
- isDMCMMExtractor: 1
- sample-rate: 48,000
- track-id: 2
- mime: audio/mp4a-latm
- profile: 2
- language: und
- aac-profile: 2
- track-fourcc: -1
- encoder-delay: 0
- durationUs: 15,402,666
- channel-count: 2
- bits-per-sample: 16
- encoder-padding: 48
- max-input-size: 524,308
- csd-0: java.nio.HeapByteBuffer[pos=0 lim=2 cap=2]
Any help would be highly appreciated.
Thank you.
答案1
得分: 3
音频使用高级音频编码(AAC)进行了压缩。你可以从这一行看到:mime: audio/mp4a-latm
。
但你没有对其进行解码。MediaExtractor
读取压缩样本,而 AudioTrack
需要解码后的数据(在你的情况下是 AudioFormat.ENCODING_PCM_16BIT
)。
你可以使用MediaCodec API来解码MediaExtractor
读取的音频缓冲区。
顺便说一下,你不需要将解码后的缓冲区复制到数组中,只需将它们原样传递给 AudioTrack
即可。
或者,你可以检测正确的编码并在设置 AudioTrack
时指定它。请参见这里和这里。
英文:
The audio is compressed using the Advanced Audio Coding (AAC) compression. You can see it from this line:
mime: audio/mp4a-latm
.
But you are not decoding it. The MediaExtractor
reads compressed samples while the AudioTrack
expects decoded data (AudioFormat.ENCODING_PCM_16BIT
in your case).
You can use the MediaCodec API to decode the audio buffers that MediaExtractor
reads.
By the way, you will not need to copy the decoded buffers into an array, just pass them to the AudioTrack
as is.
Alternatively, you could detect the correct encoding and specify it when you set the AudioTrack
up. See here and here.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论