英文:
How many frame need to be decoded before output callback is invoked in webcodec(avc1.640028)
问题
当我在一个requestAnimationFrame的滴答中只调用VideoDecoder.decode(new EncodedVideoChunk(sample));一次时，我注意到output回调也大致在每个requestAnimationFrame中调用，但视频质量较差。
然而，当我在相同滴答中多次调用
VideoDecoder.decode(new EncodedVideoChunk(sample));
并且output回调可以产生良好质量的帧时，为什么会发生这种情况？这是否与解码器倾向于引用某些“后续”样本有关？只有当“后续”样本在相同滴答中入队时，它才能引用“后续”样本。如果样本在下一个滴答中入队，Web解码器将选择在不引用“延迟”样本的情况下发出帧。
如果我的猜测正确，那么在相同滴答中解码多少个样本才能产生良好质量的帧？以下数据可能提供更多提示：
当在不同滴答中解码和渲染（解码然后渲染）时，它是正常的：
decode:0 帧
decode:1 帧
decode:2 帧
...
render:95 帧
render:94 帧
render:96 帧
然而，当在相同滴答中解码和渲染（同时解码和渲染）时，帧质量较差：
decode:0 帧
decode:1 帧
decode:2 帧
...
render:85 帧
请注意，这些只是我为您提供的翻译。如果您有其他问题或需要进一步的解释，请告诉我。
英文:
When I invoked
VideoDecoder.decode(
new EncodedVideoChunk(sample)
);
once in a tick of requestAnimationFrame, I noticed the output callback is also invoked roughly in each requestAnimationFrame, but the video quality is poor
However, when I invoke
VideoDecoder.decode(
new EncodedVideoChunk(sample)
);
multiple times in the same tick and output callback can produce the frame of good quality
Why this happen? I there anything related to the fact that decoder tend to refer to some "after" sample? It can only refer to "after" samples when the "after" sample is enqueued in the same tick. If the samples are enqueued in the next tick, the web decoder will choose to emit the frame without referring to the "late" samples?
It my guess is true, what is a good number of samples to decode in the same tick to produce a frame with good quality
The following data may provide more hint:
When decode and render in different tick(decode then render), it is OK
decode:0 frame
decode:1 frame
decode:2 frame
decode:3 frame
decode:4 frame
decode:5 frame
decode:6 frame
decode:7 frame
decode:8 frame
decode:9 frame
decode:10 frame
decode:11 frame
decode:12 frame
decode:13 frame
decode:14 frame
decode:15 frame
decode:16 frame
decode:17 frame
decode:18 frame
decode:19 frame
decode:20 frame
decode:21 frame
decode:22 frame
decode:23 frame
decode:24 frame
decode:25 frame
decode:26 frame
decode:27 frame
decode:28 frame
decode:29 frame
decode:30 frame
decode:31 frame
decode:32 frame
decode:33 frame
decode:34 frame
decode:35 frame
decode:36 frame
decode:37 frame
decode:38 frame
decode:39 frame
decode:40 frame
decode:41 frame
decode:42 frame
decode:43 frame
decode:44 frame
decode:45 frame
decode:46 frame
decode:47 frame
decode:48 frame
decode:49 frame
decode:50 frame
decode:51 frame
decode:52 frame
decode:53 frame
decode:54 frame
decode:55 frame
decode:56 frame
decode:57 frame
decode:58 frame
decode:59 frame
decode:60 frame
decode:61 frame
decode:62 frame
decode:63 frame
decode:64 frame
decode:65 frame
decode:66 frame
decode:67 frame
decode:68 frame
decode:69 frame
decode:70 frame
decode:71 frame
decode:72 frame
decode:73 frame
decode:74 frame
decode:75 frame
decode:76 frame
decode:77 frame
decode:78 frame
decode:79 frame
decode:80 frame
decode:81 frame
decode:82 frame
decode:83 frame
decode:84 frame
decode:85 frame
decode:86 frame
decode:87 frame
decode:88 frame
decode:89 frame
decode:90 frame
decode:91 frame
decode:92 frame
decode:93 frame
decode:94 frame
decode:95 frame
decode:96 frame
decode:97 frame
decode:98 frame
decode:99 frame
render:0 frame
render:3 frame
render:2 frame
render:4 frame
render:1 frame
render:7 frame
render:6 frame
render:8 frame
render:5 frame
render:11 frame
render:10 frame
render:12 frame
render:9 frame
render:15 frame
render:14 frame
render:16 frame
render:13 frame
render:19 frame
render:18 frame
render:20 frame
render:17 frame
render:23 frame
render:22 frame
render:24 frame
render:21 frame
render:27 frame
render:26 frame
render:28 frame
render:25 frame
render:31 frame
render:30 frame
render:32 frame
render:29 frame
render:35 frame
render:34 frame
render:36 frame
render:33 frame
render:39 frame
render:38 frame
render:40 frame
render:37 frame
render:43 frame
render:42 frame
render:44 frame
render:41 frame
render:47 frame
render:46 frame
render:48 frame
render:45 frame
render:51 frame
render:50 frame
render:52 frame
render:49 frame
render:55 frame
render:54 frame
render:56 frame
render:53 frame
render:59 frame
render:58 frame
render:60 frame
render:57 frame
render:63 frame
render:62 frame
render:64 frame
render:61 frame
render:67 frame
render:66 frame
render:68 frame
render:65 frame
render:71 frame
render:70 frame
render:72 frame
render:69 frame
render:75 frame
render:74 frame
render:76 frame
render:73 frame
render:79 frame
render:78 frame
render:80 frame
render:77 frame
render:83 frame
render:82 frame
render:84 frame
render:81 frame
render:87 frame
render:86 frame
render:88 frame
render:85 frame
render:91 frame
render:90 frame
render:92 frame
render:89 frame
render:95 frame
render:94 frame
render:96 frame
However when decode and render at the same tick(decode and render at the same time) the frame quality is poor
decode:0 frame
decode:1 frame
decode:2 frame
decode:3 frame
decode:4 frame
render:0 frame
decode:5 frame
render:3 frame
decode:6 frame
render:2 frame
decode:7 frame
render:4 frame
decode:8 frame
render:1 frame
decode:9 frame
render:7 frame
decode:10 frame
render:6 frame
decode:11 frame
render:8 frame
decode:12 frame
render:5 frame
decode:13 frame
render:11 frame
decode:14 frame
render:10 frame
decode:15 frame
render:12 frame
decode:16 frame
render:9 frame
decode:17 frame
decode:18 frame
render:15 frame
render:14 frame
decode:19 frame
render:16 frame
decode:21 frame
render:13 frame
decode:22 frame
render:19 frame
decode:23 frame
render:18 frame
decode:24 frame
render:17 frame
decode:25 frame
render:23 frame
decode:26 frame
render:22 frame
decode:27 frame
render:24 frame
decode:28 frame
render:21 frame
decode:29 frame
render:27 frame
decode:30 frame
render:26 frame
decode:31 frame
render:28 frame
decode:32 frame
decode:33 frame
render:25 frame
render:31 frame
decode:34 frame
render:30 frame
decode:35 frame
render:32 frame
decode:36 frame
render:29 frame
decode:37 frame
render:35 frame
decode:38 frame
render:34 frame
decode:39 frame
render:36 frame
decode:40 frame
render:33 frame
decode:41 frame
render:39 frame
decode:43 frame
render:38 frame
decode:44 frame
render:40 frame
decode:45 frame
render:37 frame
decode:46 frame
render:43 frame
decode:47 frame
render:44 frame
decode:48 frame
render:41 frame
decode:49 frame
render:47 frame
decode:50 frame
render:46 frame
decode:51 frame
decode:52 frame
render:48 frame
decode:53 frame
render:45 frame
decode:54 frame
render:51 frame
decode:55 frame
render:50 frame
decode:56 frame
render:52 frame
render:49 frame
decode:57 frame
render:55 frame
decode:58 frame
render:54 frame
decode:59 frame
render:56 frame
decode:60 frame
render:53 frame
decode:61 frame
render:59 frame
decode:62 frame
render:58 frame
decode:63 frame
render:60 frame
decode:64 frame
render:57 frame
decode:65 frame
render:63 frame
decode:66 frame
render:62 frame
decode:67 frame
render:64 frame
decode:68 frame
render:61 frame
decode:69 frame
render:67 frame
decode:70 frame
render:66 frame
decode:71 frame
render:68 frame
decode:72 frame
render:65 frame
decode:73 frame
render:71 frame
decode:74 frame
render:70 frame
decode:75 frame
render:72 frame
decode:76 frame
render:69 frame
decode:77 frame
render:75 frame
decode:78 frame
render:74 frame
decode:79 frame
decode:80 frame
render:76 frame
render:73 frame
decode:81 frame
render:79 frame
decode:82 frame
render:78 frame
decode:83 frame
render:80 frame
decode:84 frame
render:77 frame
decode:85 frame
render:83 frame
答案1
得分: 1
To better pin-point your problem, I would start by making sure that requestAnimationFrame is in sync with the decoder. Strangely, the output function is not the right place to do it, since the frame building is asynchronous. For that very reason, the videoDecoder has an event called "ondequeue" which will fire only when the frame is ready.
So, my suggestion for you is controlling your animation tick with the videoDecoder.ondequeue event (so that the decoder and the requestAnimationFrame works in sync). It is also good practice to allow the decoder to buffer some frames before start rendering, like that:
videoDecoder.ondequeue = () => {
decodeFrames++
if(decodedFrames > ??){
requestAnimationFrame();
decodedFrames--
}
};
英文:
To better pin-point your problem, I would start by making sure that requestAnimation Frame is in sync with the decoder. Strangely, the output function is not the right place to do it, since the frame building is asynchronous. For that very reason, the videoDecoder has an event called "ondequeue" which will fire only when the frame is ready.
So, my suggestion for you is controlling your animation tick with de videoDecoder.ondequeue event (so that the decoder and the requestAnimationFrame works in sync). It is also good practice to allow the decoder to buffer some frames before start rendering, like that:
videoDecoder.ondequeue = () => {
decodeFrames++
if(decodedFrames>??){
requestAnimationFrame();
decodedFrames--
}
};
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论