WebRTC – Firefox和Chrome上的多轨道流问题

huangapple go评论70阅读模式
英文:

WebRTC - multi-track streams issue on Firefox <> Chrome

问题

我创建了一个视频通话+多屏幕共享应用,在每种情况下都运行良好,除了一个例外情况。

主要流程如下:

  • 用户(P1)加入一个空房间,什么都不会发生。
  • 第二个用户(P2)加入房间。P1向P2发送一个邀请,P2回应P1。
  • 其他用户(PN)加入房间。房间中已经有的每个用户向PN用户发送邀请。PN用户对每个邀请都会回应。
  • 在主应用程序流程中,每个用户都可以共享他们的屏幕(多次)。当这发生时,用户会向已连接的同行发送邀请,并从他们那里收到回应。

摄像头流程始终运行良好,而在特定情况下,屏幕共享会出现问题:

  • P1是一个使用Chrome的用户,加入了房间。什么都不会发生。
  • P2是一个使用Firefox的用户,加入了房间。P1(Chrome)向P2发送他的邀请,摄像头连接像往常一样正常工作。
  • P2(Firefox)共享屏幕。邀请会引发错误:
Failed to execute &#39;setLocalDescription&#39; on &#39;RTCPeerConnection&#39;: Failed to parse SessionDescription. a=rtpmap:127 H264/90000 Duplicate  payload type with conflicting codec name or clock rate

查看邀请的SDP,我看到了a=rtpmap:127行的有效负载重复了:

a=rtpmap:127 H264/90000
...
a=rtpmap:127 rtx/90000

这个错误仅在指定的流程中发生(Chrome首先发送邀请,Firefox回应(摄像头正常工作),Firefox共享屏幕,我收到SDP错误)。如果第一个邀请由Firefox用户发送,一切都正常工作。如果第一个邀请由Chrome发送,只要首次屏幕共享由Chrome用户开始,一切都正常工作。

如果第一个邀请由Chrome用户发送,然后Firefox用户共享屏幕,它就会出现问题。只有在这种情况下。

所以,问题在于Firefox用户在第一次屏幕共享期间创建的邀请包含了这个有效负载冲突。

为什么会发生这种情况(只在这种情况下),以及如何防止这个错误?

英文:

I created a video-call + multiple screen sharing app that works well in every case except one.

The main flow is:

  • A user (P1) join an empty room, nothing happens
  • A second user (P2) join the room. P1 sends an offer to P2 which sends a response to P1.
  • Other users (PN) join the room. Every user already in the room send offers to the PN user. PN user send answers for every offer.
  • During the main app flow, every user can share their screen (multiple times). When this happens, the user send an offer to every already-connected peers and receive answers from them.

The webcam flow always works well, while the screen sharing breaks in a specific case:

  • P1 Is a Chrome user that joins the Room. Nothing happens.
  • P2 Is a Firefox user that joins the Room. P1 (Chrome) sends his offer to P2 Webcam connection works well as usual.
  • P2 (Firefox) share a screen. The Offer throws an error:
Failed to execute &#39;setLocalDescription&#39; on &#39;RTCPeerConnection&#39;: Failed to parse SessionDescription. a=rtpmap:127 H264/90000 Duplicate  payload type with conflicting codec name or clock rate

Looking at the offer's SDP, i see that the payload for the line a=rtpmap:127 is duplicated:

a=rtpmap:127 H264/90000
...
a=rtpmap:127 rtx/90000

This error happens ONLY in the specified flow (Chrome sends an offer first, Firefox answers (and camera works well), Firefox share a screen and i get the SDP error). If the first offer is sent by a Firefox user, everything works well. If the first offer is sent by a Chrome, everything works well if the first screen sharing is started by a Chrome user.

If the first offer is sent by a Chrome user and then a Firefox user shares a screen, it breaks. Only in this case.

So, the problem is that the offer created by the Firefox user during the first screen sharing contains that payload conflict.

Why does it happen (only in that case) and how can i prevent this error?

答案1

得分: 0

I also run into this bug and I didn't find a solution other than modifying the SDP description by myself.
Although this is a rough workaround, it fixes the codec collision bug.

  1. You want to remove the duplicate codec number from the video track (127 here):

m=video 9 UDP/TLS/RTP/SAVPF 100 101 127 127 ...

  1. then remove the payload of the RTX codec.

a=rtpmap:127 RTX/90000

Keep in mind that RTX is used to resend corrupted packages.

RTX stands for retransmission.

RTP retransmission is an effective packet loss recovery technique for
real-time applications with relaxed delay bounds.

So removing it from the payload may void this capability.

Here's my JavaScript (TypeScript) function:

removeDuplicateCodecs(sdp: string): string {
    // Split SDP for each video track
    const lines = sdp.split(/^(?=m=video)/gm);

    for (let i = 0, videosLength = lines.length; i < videosLength; i++) {
        if (lines[i].startsWith("m=video")) {

            // Split each line
            let rows = lines[i].split(/\r\n/gm);
            const codecDuplicated: string[] = [];

            if (rows.length) {
                // Take the first row and get all codecs
                const duplicates = rows[0].match(/(\b\d+\b)(?=.*\b\b)/g);

                if (duplicates?.length) {
                    duplicates.forEach(duplicate => {

                        // Remove duplicates from the row
                        rows[0] = rows[0].replace(` ${duplicate}`, '')
                        codecDuplicated.push(duplicate);
                    });
                }
            }

            // Join back all rows
            lines[i] = rows.join('\r\n');

            // Split by rtpmap
            rows = lines[i].split(/^(?=a=rtpmap:)/gm);
            if (rows) {
                codecDuplicated.forEach(duplicate => {

                    // Remove duplicate codec definition rows
                    rows = rows.filter(row => !row.startsWith(`a=rtpmap:${duplicate} rtx`));
                });
                lines[i] = rows.join('');
            }
        }
    }
    return lines.join('');
}
英文:

I also run into this bug and i didn't found a solution other than modifying the SDP description by myself.
Although this is a rough workaround, it fix the codec collision bug.

  1. You want to remove the duplicate codec number form the video track (127 here):

    m=video 9 UDP/TLS/RTP/SAVPF 100 101 127 127 ...

  2. then remove the payload of the RTX codec.

    a=rtpmap:127 RTX/90000

    ...

Keep in mind that RTX is used to resend corrupted packages

> RTX stands for retransmission. RTX

> RTP retransmission is an effective packet loss recovery technique for
real-time applications with relaxed delay bounds rfc4588

So removing it from payload may void this capability.

Here my JS (typescript) function

removeDuplicateCodecs(sdp: string): string {
    // split sdp for each video track
    const lines = sdp.split(/^(?=m=video)/gm);

    for (let i = 0, videosLength = lines.length; i &lt; videosLength; i++) {
        if (lines[i].startsWith(&quot;m=video&quot;)) {

            // split each line
            let rows = lines[i].split(/\r\n/gm);
            const codecDuplicated: string[]= [];

            if (rows.length) {
                // take first row and get all codecs
                const duplicates = rows[0].match(/(\b\d+\b)(?=.*\b\b)/g);

                if (duplicates?.length) {
                    duplicates.forEach(duplicate =&gt; {

                        // remove duplicates from row
                        rows[0] = rows[0].replace(` ${duplicate}`, &#39;&#39;)
                        codecDuplicated.push(duplicate);
                    });
                }
            }

            // join back all rows
            lines[i] = rows.join(&#39;\r\n&#39;);

            // split by rtpmap
            rows = lines[i].split(/^(?=a=rtpmap:)/gm);
            if (rows) {
                codecDuplicated.forEach(duplicate =&gt; {
                    
                // remove duplicate codec definitions rows
                    rows = rows.filter(row =&gt; !row.startsWith(`a=rtpmap:${duplicate} rtx`));
                });
                lines[i] = rows.join(&#39;&#39;);
            }
        }
    }
    return lines.join(&#39;&#39;);
}

huangapple
  • 本文由 发表于 2023年7月6日 18:46:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/76628006.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定