WebRTC connections randomly without audio output

user_5021504 0
Viewed 692

I stumbled on a weird issue in a WebRTC webapp. Here's the setup: Client A and client B send audio via a send-only WebRTC connections to a SFU. Client C receives via two receive-only connections to that same SFU the audio streams from client A and B and adds them to two different "audio" elements. The routing between these send and receive connections work properly.

Here's the problem: On refreshing the page, sometimes client C hears audio from both client A and B. But most of the time client C only hears audio from randomly A or B. It's happening in both firefox and chrome.

Both connections are receiving data (see graph "bitsReceivedPerSecond") but only one connection is outputting audio. Here an example where C could hear A but not B:

Connection Client A -> C: Connection with audio output

Connection Client B -> C: Connection without audio output

My understanding of these graphs is that the raw WebRTC connection is working fine (data is sent and received) but somehow a connection does not output audio randomly.

Does anyone have a clue how this can happen?

Here is the "ontrack" callback for adding the streams to the audio elements. The Logs do appear correctly for each connection.

gotRemoteStream(e) {
    Logger.log("Remote Streams: #"+e.streams.length);

    if (this.audioElement.srcObject !== e.streams[0]) {
        Logger.log("Received remote Stream with tracks to audio: " + this.audioElement.id);
        this.audioElement.srcObject = e.streams[0];
    }
}
webrtc audio javascript
1 Answers

A single audio element can only play one audio track at a time.

You said there were two incoming audio tracks, so if this.audioElement is the same element, then each call to gotRemoteStream will race setting srcObject, one overwriting the other.

This is most likely why you only hear one or the other.

The simplest solution might be to key off the stream associations sent, since they'll tend to differ:

const elements = {};

gotRemoteStream({streams: [stream]}) {
    if (!elements[stream.id]) {
      elements[stream.id] = this.audioElementA.srcObject ? this.audioElementB
                                                         : this.audioElementA; 
    }
    elements[stream.id].srcObject = stream;
}

This should work for two incoming connections. More than that is left as an exercise.

user_9189100
Related