WebRTC ICE candidate 在 setRemoteDescription 后未生成。

huangapple
go评论80阅读模式
英文:

WebRTC ICE candidate not generating after setRemoteDescription

问题

I see that you have provided a detailed log of SDP offers, answers, and ICE candidates in your WebRTC implementation. Based on the information you've provided, here are some answers to your questions:

  1. SDP Offer/Answer and ICE Candidates:

    • In the SDP you've shown, it includes both SDP (Session Description Protocol) information and ICE (Interactive Connectivity Establishment) candidate information. SDP is used for session negotiation (offer and answer), while ICE candidates are used for establishing the network connection.
    • When receiving SDP from another user, you should treat it as SDP for session negotiation. Use peerConnection.setRemoteDescription() to set the remote description with the received SDP.
    • ICE candidates should be gathered and exchanged separately. In WebRTC, ICE candidates are gathered as part of the connection setup process. These ICE candidates should be sent to the remote peer using your signaling channel (e.g., via WebSocket) so that both peers can exchange their ICE candidates. This helps establish the network connection by finding the best possible route.

  2. Handling ICE Candidates:

    • The onIceCandidate callback is called multiple times as ICE candidates are gathered. You should indeed send these ICE candidates to the remote peer through your signaling channel each time this callback is triggered. This allows both peers to exchange their candidates and establish a connection.
    • Keep in mind that ICE candidates can change dynamically as the network conditions change, so it's essential to keep sending them until the ICE negotiation process is complete.

  3. Initiating the Call:

    • After setting the remote description and exchanging ICE candidates, you can initiate the call by creating an SDP answer using peerConnection.createAnswer(). Once the answer is created, you can set it as the local description using peerConnection.setLocalDescription().
    • Send the answer SDP to the remote peer through your signaling channel, and they should set it as their remote description using peerConnection.setRemoteDescription().
    • Continue exchanging ICE candidates as needed until both peers have gathered enough candidates, and the ICE negotiation process is complete.

Remember that WebRTC communication involves bidirectional communication of SDP offers, answers, and ICE candidates between both peers. Proper synchronization and handling of these components are crucial for a successful WebRTC call setup.

英文:

I am trying to setup WebRTC ( implementation 'org.webrtc:google-webrtc:1.0.32006' )for video and voice call. I can see the exchange of SDP via websocket by creating offer then receive the answer from other user. However, after successfully receiving SDP answer from user, I save it via peerConnection.setRemoteDescription().

SessionDescription sessionDescription = new SessionDescription(SessionDescription.Type.ANSWER,sdp_ice_VALUE);
peerConnection.setRemoteDescription(new SdpObserver() {
     @Override
     public void onCreateSuccess(SessionDescription sessionDescription) {
                 Log.d(TAG, "onCreateSuccess: ");
        }

       @Override
       public void onSetSuccess() {
          Log.d(TAG, "onSetSuccess: ");
       }

       @Override
       public void onCreateFailure(String s) {
          Log.d(TAG, "onCreateFailure: ");
       }

       @Override
       public void onSetFailure(String s) {
          Log.d(TAG, "onSetFailure: ");
       }}, sessionDescription);

I can see the onSetSuccess call in logcat, which means the remote description is set successfully. However, ICE candidate is not generating.

I have checked the code several times and I can't find any obvious errors. Could anyone help me to debug this issue?

Here is the complete code/fragment

public class Video_Voice_Call_Fragment extends Fragment {
@Override
public void onViewCreated(@NonNull View view, @Nullable Bundle savedInstanceState) {
super.onViewCreated(view, savedInstanceState);
greenCallButton = view.findViewById(R.id.call_connect);
redCallButton = view.findViewById(R.id.call_disconnect);
initializeWebRTC();
startCall();
}
private void initializeWebRTC() {
rootEglBase = EglBase.create();
// Initialize PeerConnectionFactory
PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(requireContext())
.setEnableInternalTracer(true)
.createInitializationOptions());
DefaultVideoEncoderFactory videoEncoderFactory = new DefaultVideoEncoderFactory(rootEglBase.getEglBaseContext(),true,true);
DefaultVideoDecoderFactory videoDecoderFactory = new DefaultVideoDecoderFactory(rootEglBase.getEglBaseContext());
PeerConnectionFactory.Options options = new PeerConnectionFactory.Options();
peerConnectionFactory = PeerConnectionFactory.builder()
.setVideoDecoderFactory(videoDecoderFactory)
.setVideoEncoderFactory(videoEncoderFactory)
.setOptions(options)
.createPeerConnectionFactory();
}
private void startCall() {
// Create an offer and set local description
peerConnection = generateICE();
peerConnection.createOffer(new SimpleSdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
Log.d(TAG, "onCreateSuccess: " + sessionDescription.description);
peerConnection.setLocalDescription(new SimpleSdpObserver(), sessionDescription);
Websocket.connectWebSocket(new WebSocketCallback() {
@Override
public void onConnectionSuccess() {
//Websocket Connected
Log.d(TAG, "onConnectionSuccess: Websocket Connected");
String message = generateJSON(sessionDescription.description,true,"offer");
Websocket.sendMessage(message);
}
@Override
public void onConnectionFailure(String message) {
Log.d(TAG, "onConnectionFailure: Message = "+message);
}
@Override
public void onConnectionClosed(String reason) {
Log.d(TAG, "onConnectionClosed: Reason = "+reason);
}
@Override
public void onConnectionsMessage(String message) {
Log.d(TAG, "onConnectionsMessage: Message = "+message);
// Here User will receive the message from the websocket
try{
JSONObject socketMessage = new JSONObject(message);
String type = socketMessage.getString("type");
String sdp_ice_VALUE = socketMessage.getString("SDP_ICE_info");
if (type.equals("SDP")){
Log.d(TAG, "onConnectionsMessage: Type SDP");
SessionDescription sessionDescription = new SessionDescription(SessionDescription.Type.ANSWER,sdp_ice_VALUE);
peerConnection.setRemoteDescription(new SdpObserver() {
@Override
public void onCreateSuccess(SessionDescription sessionDescription) {
Log.d(TAG, "onCreateSuccess: ");
}
@Override
public void onSetSuccess() {
Log.d(TAG, "onSetSuccess: ");
}
@Override
public void onCreateFailure(String s) {
Log.d(TAG, "onCreateFailure: ");
}
@Override
public void onSetFailure(String s) {
Log.d(TAG, "onSetFailure: ");
}
}, sessionDescription);
}else {
// Received ICE from server as answer, This is the last step in exchanging info
WebRTCUtils.addIceCandidate(peerConnection,sdp_ice_VALUE);
}
}catch (JSONException e){
Log.d(TAG, "onMessage: Error "+e.getMessage());
}
}
});
// Send the offer to the remote peer
// Example code for sending the offer
}
@Override
public void onCreateFailure(String s) {
Log.e(TAG, "onCreateFailure: " + s);
}
}, new MediaConstraints());
}
private PeerConnection generateICE (){
Log.d(TAG, "generateICE: ");
ArrayList<PeerConnection.IceServer> iceServers = new ArrayList<>();
List<String> STUNList= Arrays.asList(
"stun:stun.l.google.com:19302",
"stun:stun1.l.google.com:19302",
"stun:stun2.l.google.com:19302",
"stun:stun3.l.google.com:19302",
"stun:stun4.l.google.com:19302",
"stun:stun.vodafone.ro:3478",
"stun:stun.samsungsmartcam.com:3478",
"stun:stun.services.mozilla.com:3478"
);
for(String i:STUNList){
PeerConnection.IceServer.Builder iceServerBuilder = PeerConnection.IceServer.builder(i);
iceServerBuilder.setTlsCertPolicy(PeerConnection.TlsCertPolicy.TLS_CERT_POLICY_INSECURE_NO_CHECK); //this does the magic.
PeerConnection.IceServer iceServer =  iceServerBuilder.createIceServer();
iceServers.add(iceServer);
}
PeerConnection.RTCConfiguration rtcConfig = new PeerConnection.RTCConfiguration(iceServers);
PeerConnection.Observer observer = new PeerConnection.Observer() {
@Override
public void onSignalingChange(PeerConnection.SignalingState signalingState) {
Log.d(TAG, "onSignalingChange: "+signalingState);
}
@Override
public void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {
Log.d(TAG, "onIceConnectionChange: "+iceConnectionState);
}
@Override
public void onIceConnectionReceivingChange(boolean b) {
Log.d(TAG, "onIceConnectionReceivingChange: Boolean = "+b);
}
@Override
public void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {
Log.d(TAG, "onIceGatheringChange: "+iceGatheringState);
}
@Override
public void onIceCandidate(IceCandidate iceCandidate) {
Log.d(TAG, "onIceCandidate: "+iceCandidate);
}
@Override
public void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {
Log.d(TAG, "onIceCandidatesRemoved: "+iceCandidates);
}
@Override
public void onAddStream(MediaStream mediaStream) {
Log.d(TAG, "onAddStream: "+mediaStream);
}
@Override
public void onRemoveStream(MediaStream mediaStream) {
Log.d(TAG, "onRemoveStream: "+mediaStream);
}
@Override
public void onDataChannel(DataChannel dataChannel) {
Log.d(TAG, "onDataChannel: "+dataChannel);
}
@Override
public void onRenegotiationNeeded() {
Log.d(TAG, "onRenegotiationNeeded: ");
}
@Override
public void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {
Log.d(TAG, "onAddTrack: "+rtpReceiver+"--- "+mediaStreams);
}
};
Log.d(TAG, "generateICE: ");
return peerConnectionFactory.createPeerConnection(rtcConfig,observer);
}
}

Update:- With the suggestions from @PhilippHancke. I edit my code:-

private void setupLocalTracks() {
// Create video capturer
videoCapturer = createVideoCapturer();
// Create video source
videoSource = peerConnectionFactory.createVideoSource(false);
localVideoTrack = peerConnectionFactory.createVideoTrack("local_video", videoSource);
// Create audio source and track
AudioSource audioSource = peerConnectionFactory.createAudioSource(new MediaConstraints());
localAudioTrack = peerConnectionFactory.createAudioTrack("local_audio", audioSource);
// Create local media stream
localMediaStream = peerConnectionFactory.createLocalMediaStream("local_stream");
localMediaStream.addTrack(localVideoTrack);
localMediaStream.addTrack(localAudioTrack);
peerConnection.addStream(localMediaStream);
}

Now i can see much more details in log which has SDP offer/answer and ice too.I first call peerConnection.createOffer(new SimpleSdpObserver()which generate SessionDescription for User ID 4.

Message Received From User ID == 4 Message = v=0
o=- 5794866635417549132 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS local_stream
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:EXXZ
a=ice-pwd:NuOsEt2YVDiZkxFomvHPhUW+
a=ice-options:trickle renomination
a=fingerprint:sha-256 F2:1E:77:5D:21:00:15:65:00:CA:24:BE:65:66:95:09:B4:10:4C:80:C0:16:90:4A:22:E7:ED:BA:D4:A8:F5:56
a=setup:actpass
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:102 ILBC/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:473446091 cname:WQlLPNcKBxol0kL2
a=ssrc:473446091 msid:local_stream local_audio
a=ssrc:473446091 mslabel:local_stream
a=ssrc:473446091 label:local_audio
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 123 125
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:EXXZ
a=ice-pwd:NuOsEt2YVDiZkxFomvHPhUW+
a=ice-options:trickle renomination
a=fingerprint:sha-256 F2:1E:77:5D:21:00:15:65:00:CA:24:BE:65:66:95:09:B4:10:4C:80:C0:16:90:4A:22:E7:ED:BA:D4:A8:F5:56
a=setup:actpass
a=mid:video
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 H264/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:127 red/90000
a=rtpmap:123 rtx/90000
a=fmtp:123 apt=127
a=rtpmap:125 ulpfec/90000
a=ssrc-group:FID 1370797121 3619002051
a=ssrc:1370797121 cname:WQlLPNcKBxol0kL2
a=ssrc:1370797121 msid:local_stream local_video
a=ssrc:1370797121 mslabel:local_stream
a=ssrc:1370797121 label:local_video
a=ssrc:3619002051 cname:WQlLPNcKBxol0kL2
a=ssrc:3619002051 msid:local_stream local_video
a=ssrc:3619002051 mslabel:local_stream
a=ssrc:3619002051 label:local_video
Message Received From User ID == 206 Message = v=0
o=- 3655990368999828607 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS remote_stream
m=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 102 0 8 106 105 13 110 112 113 126
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:0O57
a=ice-pwd:FSyUVENoJglSvKVpeih/bzXA
a=ice-options:trickle renomination
a=fingerprint:sha-256 B8:43:6B:EE:38:08:19:85:DA:97:AA:45:2D:46:17:90:9E:8B:D2:D8:D7:9D:04:97:22:DE:C9:CB:FC:A1:DB:5B
a=setup:active
a=mid:audio
a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=sendrecv
a=rtcp-mux
a=rtpmap:111 opus/48000/2
a=rtcp-fb:111 transport-cc
a=fmtp:111 minptime=10;useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:102 ILBC/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:110 telephone-event/48000
a=rtpmap:112 telephone-event/32000
a=rtpmap:113 telephone-event/16000
a=rtpmap:126 telephone-event/8000
a=ssrc:624557633 cname:PYJF7YMAd+hgS7Zk
a=ssrc:624557633 msid:remote_stream remote_audio
a=ssrc:624557633 mslabel:remote_stream
a=ssrc:624557633 label:remote_audio
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 127 123 125
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:0O57
a=ice-pwd:FSyUVENoJglSvKVpeih/bzXA
a=ice-options:trickle renomination
a=fingerprint:sha-256 B8:43:6B:EE:38:08:19:85:DA:97:AA:45:2D:46:17:90:9E:8B:D2:D8:D7:9D:04:97:22:DE:C9:CB:FC:A1:DB:5B
a=setup:active
a=mid:video
a=extmap:14 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:13 urn:3gpp:video-orientation
a=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=sendrecv
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 H264/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:127 red/90000
a=rtpmap:123 rtx/90000
a=fmtp:123 apt=127
a=rtpmap:125 ulpfec/90000
a=ssrc-group:FID 2171330257 4072720398
a=ssrc:2171330257 cname:PYJF7YMAd+hgS7Zk
a=ssrc:2171330257 msid:remote_stream remote_video
a=ssrc:2171330257 mslabel:remote_stream
a=ssrc:2171330257 label:remote_video
a=ssrc:4072720398 cname:PYJF7YMAd+hgS7Zk
a=ssrc:4072720398 msid:remote_stream remote_video
a=ssrc:4072720398 mslabel:remote_stream
a=ssrc:4072720398 label:remote_video

In the SDP above i can ICE candidates too which creates confusion for me because when other receive this SDP then how user would save it, as SDP (setRemoteDescription) or treat it as ICE candidates beucase it contains both.

Also i can see in log that onIceCandidate calls multiples times with these kinds of values

audio:0:candidate:1587269880 1 udp 2122262783 2406:b400:52:820d:70e0:90ff:fe12:3352 35364 typ host generation 0 ufrag B/Tc network-id 4 network-cost 10::UNKNOWN
audio:0:candidate:393917512 1 udp 2122194687 192.168.0.123 42552 typ host generation 0 ufrag B/Tc network-id 3 network-cost 10::UNKNOWN
audio:0:candidate:559267639 1 udp 2122136831 ::1 50036 typ host generation 0 ufrag B/Tc network-id 2::UNKNOWN

Do i need to send these information to other user as ICE (it calls around 10 times approx) every time it calls onIceCandidate?

After these exchange which method should i call to initiate the call?

答案1

得分: 1

ICE candidates are only generated after you call setLocalDescription, see https://w3c.github.io/webrtc-pc/#set-the-session-description step 4 substep 2.

(please also note that org.webrtc:google-webrtc:1.0.32006 is an outdated version that has known security vulnerabilities)

英文:

ICE candidates are only generated after you call setLocalDescription, see https://w3c.github.io/webrtc-pc/#set-the-session-description step 4 substep 2.

(please also note that org.webrtc:google-webrtc:1.0.32006 is an outdated version that has known security vulnerabilities)

huangapple
  • 本文由 发表于 2023年6月26日 23:15:28
  • 转载请务必保留本文链接：https://go.coder-hub.com/76558014.html
  • android
  • java
  • webrtc
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定