可以使用HTML5 VideoEncoder编码为YUV422吗?

huangapple go评论74阅读模式
英文:

Is it possible to encode to yuv422 with html5 VideoEncoder?

问题

我正在尝试使用HTML5 VideoEncoder录制视频有许多AVC配置文件可以使用https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter#av1)。根据该页面,4:2:2配置文件从`7A`开始,如下:

```javascript
         let videoEncoder = new VideoEncoder({
            output: (encodedChunk, config) => {
              // 使用mp4box或mp4muxer记录块
            },
            error: (error) => {
              console.log("onCodecError ", error);
            },
          });

          videoEncoder.configure({
            codec: "avc1.7A1032", // 4:2:2配置文件
            width: 1920,
            height: 1280,
            hardwareAcceleration: "prefer-hardware",
            avc: { format: "avc" },
          });

不幸的是,这会返回DOMException: 不支持的编解码配置文件

我尝试了以下脚本以发现支持的7A配置文件:

for (let i = 0; i < 256*256; i++) { 
	try {
    		let config = {
            		codec: "avc1.7A" + i.toString(16), 
            		width: 1920,
            		height: 1280,
			        framerate: 25,
			        bitrate: 50_000_000,
            		avc: { format: "avc" },
          	}; 
		let response = await VideoEncoder.isConfigSupported(config);
    		if (response.supported) { console.log(config.codec); }
	} catch(e) {}
}

并且实际上找到了相当多的配置文件:
[![列表的一部分][1]][1]

例如,7A4032。不幸的是,尽管此配置文件运行良好,但会导致YUV420录制。而且在https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter#av1页面上找不到它,所以我担心这可能是一种故障。

因此,问题是,是否有任何方法可以使用YUV422配置文件记录视频?

更新:更奇怪的是,使用VP09编解码器也会出现相同的情况。它的格式是vp09.PP.LL.DD,其中PP定义了配置文件。因此,00和02用于420,而01和03用于422。但我也无法创建任何01或03配置文件。



<details>
<summary>英文:</summary>

I am trying to record a video using HTML5 VideoEncoder. There&#39;s a number of AVC profiles one can use (https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter#av1). According to that page, 4:2:2 profiles are starting from `7A`, like this:

     let videoEncoder = new VideoEncoder({
        output: (encodedChunk, config) =&gt; {
          // recording chunks using mp4box or mp4muxer
        },
        error: (error) =&gt; {
          console.log(&quot;onCodecError &quot;, error);
        },
      });

      videoEncoder.configure({
        codec: &quot;avc1.7A1032&quot;, // 4:2:2 profile
        width: 1920,
        height: 1280,
        hardwareAcceleration: &quot;prefer-hardware&quot;,
        avc: { format: &quot;avc&quot; },
      });

Unfortunately this returns `DOMException: Unsupported codec profile.`

I tried the following script to discover any supported `7A` profiles:

for (let i = 0; i < 256*256; i++) {
try {
let config = {
codec: "avc1.7A" + i.toString(16),
width: 1920,
height: 1280,
framerate: 25,
bitrate: 50_000_000,
avc: { format: "avc" },
};
let response = await VideoEncoder.isConfigSupported(config);
if (response.supported) { console.log(config.codec); }
} catch(e) {}
}


And found quite a few actually:
[![Part of the list][1]][1]

For example, `7A4032`. Unfortunately, despite this profile works well, it results in YUV420 recording. Also it&#39;s nowhere to be found on a https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter#av1 page, so I am afraid it&#39;s a kind of a glitch.

So, the question is, is there any way to record video with YUV422 profile?

UPD: even more weird that the same happens with VP09 codec. It&#39;s format is vp09.PP.LL.DD, where PP defines profile. So 00 and 02 are for 420, while 01 and 03 are for 422. And I can&#39;t create any 01 or 03 profile as well.


  [1]: https://i.stack.imgur.com/jOMOL.png

</details>


# 答案1
**得分**: 1

"现在的问题是:<br>是否有任何方法可以使用YUV422配置录制视频?"

不,目前无法使用WebCodecs API进行YUV422编码。

422是一种紧凑的YUV格式(例如:YUY2)。

420是一种平面YUV格式(例如:NV12)。

WebCodecs似乎将帧处理为8位平面YUV系统(例如:I420),而不是NV12。

您无法更改它,因为它似乎是硬编码的。请参见[Chromium源代码第172行](https://chromium.googlesource.com/chromium/src/+/d6814a3a0f4d8a45dddb5e44126d5e71439e8ee9/third_party/blink/renderer/modules/webcodecs/video_frame.cc#172):

```cpp
//# static
bool VideoFrame::IsSupportedPlanarFormat(media::VideoFrame* frame) 
{
    // For now only I420 in CPU memory is supported.
    return 	frame && frame->IsMappable() &&
    		frame->format() == media::PIXEL_FORMAT_I420 &&
    		frame->layout().num_planes() == 3;
}

查看预期的输入来源:
HTMLVideoElementHTMLCanvasElementImageBitmapOffscreenCanvasHTMLImageElementSVGImageElementVideoFrame

以上列出的项目是否未返回RGB平面输出?

此外,VideoFrame是将自定义YUY2数据放入的最近选项,但它将自动编码为420。

在Chrome中,<video>标签本身可以解码YUV422和YUV444,如在MP4文件中测试过的。我不明白为什么他们的H264代码库可以解码它们,但无法将像素编码为YUV422和YUV444。

"有多种AVC配置文件可供使用..."

显示的页面链接似乎是在讨论<video>标签的一般功能。我认为它在这里没有考虑(或反映)WebCodecs API本身的特定功能。

附注: 我在Windows Chrome上使用VideoEncoder.isConfigSupported测试了avc1.7A4032,结果返回false

英文:

>So, the question is: <br>
"Is there any way to record video with YUV422 profile?"

No, it's not possible to encode YUV422 with the WebCodecs API (at present).

  • 422 is a packed YUV format (eg: YUY2).

  • 420 is a planar YUV format (eg: NV12).

WebCodecs seems to process frames as 8-bit Planar YUV system (eg: I420 instead of NV12).

You cannot change it because it seems to be hardcoded. See Chromium source code line 172:

//# static
bool VideoFrame::IsSupportedPlanarFormat(media::VideoFrame* frame) 
{
	// For now only I420 in CPU memory is supported.
	return 	frame &amp;&amp; frame-&gt;IsMappable() &amp;&amp;
			frame-&gt;format() == media::PIXEL_FORMAT_I420 &amp;&amp;
			frame-&gt;layout().num_planes() == 3;
}

Look at the expected input sources:<br>
HTMLVideoElement, HTMLCanvasElement, ImageBitmap, OffscreenCanvas, HTMLImageElement, SVGImageElement and VideoFrame.

Are those above-listed items not giving back an RGB planar output?

Also VideoFrame is your nearest option to putting custom YUY2 data but it will auto encode to 420.

The &lt;video&gt; tag itself in Chrome can decode YUV422 and YUV444 as tested in MP4 files. I wonder why their H264 codebase can decode them but then also cannot encode pixels as YUV422 and YUV444.

> "There's a number of AVC profiles one can use..."

That shown page link seems to be talking about the general capabilities of the &lt;video&gt; tag.<br>

I don't think here it considers (or reflects) the specific features of the WebCodecs API itself.

PS: I tested avc1.7A4032 using VideoEncoder.isConfigSupported and got false on Windows Chrome.

huangapple
  • 本文由 发表于 2023年7月20日 15:25:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/76727560.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定