HTML5 & Web audio api: Streaming microphone data from browser to server. Ideal transports and data compression

user_1868465 0
Viewed 9297

I am looking to take the audio input from the browser and stream it to multiple listeners. The intended use is for music, so the quality must mp3 standard or thereabouts.

I have attempted two ways, both yielding unsuccessful results:

WebRTC

  • Streaming audio directly between browsers works fine, but the audio quality seems to be non-customisable though what I have seen. (I have seen that it is using the Opus audio codec, but seems to not expose any controls).
  • Does anyone have any insight into how to increase the audio quality in WebRTC streams?

Websockets

  • The issue is the transportation from the browser to the server. The PCM audio data I can acquiring via the method below has proven too large to repeatedly stream to the server via websockets. The stream works perfectly in high speed internet environments, but on slower wifi it is un-usable.

    var context = new webkitAudioContext()
navigator.webkitGetUserMedia({audio:true}, gotStream)

function gotStream (stream)
{
    var source = context.createMediaStreamSource(stream)
    var proc = context.createScriptProcessor(2048, 2, 2)

    source.connect(proc)
    proc.connect(context.destination)
    proc.onaudioprocess = function(event)
    {
        var audio_data = event.inputBuffer.getChannelData(0)|| new Float32Array(2048)
        console.log(audio_data)
        // send audio_data to server
    }
}

So the main question is, is there any way to compress the PCM data in order to make it easier to stream to the server? Or perhaps there is an easier way to go about this?

getusermedia web-audio-api webrtc websocket html
3 Answers

WebRTC seems to default to one mono channel around 42 kb/s, it seems to be primarily designed for voice.

You can disable the audio processing features using constraints to get a more consistent input from the browser using:

navigator.mediaDevices.getUserMedia({ audio: { autoGainControl: false, channelCount: 2, echoCancellation: false, latency: 0, noiseSuppression: false, sampleRate: 48000, sampleSize: 16, volume: 1.0 } });

Then you also should set stereo and maxaveragebitrate params on the SDP:

let answer = await peer.conn.createAnswer(offerOptions);
answer.sdp = answer.sdp.replace('useinbandfec=1', 'useinbandfec=1; stereo=1; maxaveragebitrate=510000');
await peer.conn.setLocalDescription(answer);

This should output a string which looks like this:

a=fmtp:111 minptime=10;useinbandfec=1; stereo=1; maxaveragebitrate=510000

This could increase the bitrate up to 520kb/s for stereo, which is 260kps per channel. Actual bitrate depends on the speed of your network and strength of your signal tho.

user_20687660
Related