Since WebRTC is encrypted, it’s hard to verify that the incoming RTP streams are formatted correctly and timestamped correctly.
My second question is about general debugging the RTP incoming streams in WebRTC. Is that correct? From looking at the source code that’s what I concluded, but I can’t find any documentation discussing a/v sync.
The first question I have is about our assumption that synchronization in webrtc is based off RTPC packet timestamps. My guess from that observation is that it’s possible that webrtc is trying to sync the streams, but is trying to sync them to an incorrect offset. Meaning that most of the time audo and video fall out of sync the same amount, having the audio ahead by about 150ms. One interesting observation I had was most of the time when I reproduce the bad a/v sync, the a/v sync seems to always have the same incorrect offset. If you switch to a transcode on the test channel you will see that the audio and video fall out of sync. If you then select the gear icon it the bottom right of the video, you will see quality options for transcodes. On, when if you open the test channel above you will see the source stream by default, it should be in sync. Other streams on the website won’t have the same behavior) (This test channel is running on new test ingest bits that produce the streams as described above. Our video transcode keeps the same input fps, so one frame in results in one frame out. But the timestamps in both the RTP and RTPC packet are the exact same as source. In fact, the audio and RTPC streams are copies of the source streams with the SSRC updated. The only differences are that we change the SSRC id on the transcode streams as well as scale the video. Our transcodes produce two RTP streams and two RTPC streams almost identical to the streams in source. We have been seeing a problem with a/v sync on our live transcodes.
Thus, we send along with the RTP video and audio streams a RTPC message for each stream every 5 seconds. Instead, RTPC messages must be sent for each audio and video stream to give an absolute time for the streams to be synced to. From our current understanding of WebRTC, we assume that the RTP timestamps can be seeded with a random value (which we don’t do today) and thus they can and shouldn’t be used to sync the streams. We have a considerably basic stream - we have one RTP video stream and one RTP audio stream. Recently, we have been having trouble debugging audio / video synchronization on our channels and have some questions. We use WebRTC on all our clients for video playback to keep the latency as low as possible. Mixer is a website that provides low latency game streaming simular to YouTube and Twitch.
0 kommentar(er)
