Improving the sound quality in socket io sending audio data from client to server back to client

user_5327617 0
Viewed 1103

This code actually works. Every 300 miliseconds a chunk of audio data is sent to the server and back to the clients in the socket room to be played. There is only one issue. It is extremely bad quality of audio. Every 300 second interval there is a short little static noise that is gone almost instantly. I believe that is because of the time between when the audio chunks are being sent to the server and back to the sockets in the room. I am not using a webRTC like socket io p2p or peerjs because they are really complicated and I'm a beginner, so is there anything in this code I can do to play out the audio more smoothly? I have tried different things such as changing the milliseconds in the setInterval function to 60 and raised it to 5000. The lower the interval the choppier but faster the playback but higher is better quality but 5 seconds of delay.

Client:

var constraints = { audio: true };
    navigator.mediaDevices.getUserMedia(constraints).then(function(mediaStream) {
        var mediaRecorder = new MediaRecorder(mediaStream);
        mediaRecorder.onstart = function(e) {
            this.chunks = [];
        };

        mediaRecorder.ondataavailable = function(e) {
            this.chunks.push(e.data);
        };
        mediaRecorder.onstop = function(e) {
            var blob = new Blob(this.chunks);
            var url = <%- JSON.stringify(url) %>;
            socket.emit('radio', {blob : blob, url : url});
        };

        mediaRecorder.start();

        setInterval(function() {
            mediaRecorder.stop()
            mediaRecorder.start();
        }, 300);
    });

    socket.on('voice', function(arrayBuffer) {
        var blob = new Blob([arrayBuffer], { 'type' : 'audio/webm;codecs=opus' });
        var audio = document.createElement('audio');
        audio.src = window.URL.createObjectURL(blob);
        audio.play();
    });

Server:

socket.on('radio', function(data) {
    socket.broadcast.to(data.url).emit('voice', data.blob);
    socket.join(data.url);
});
getusermedia socket.io node.js audio javascript
1 Answers

Your audio quality is fine. Your problem is gaps in the playback between your chunks of audio. Good audio looks like this, sort of. Each audio sample is a *. You probably have 44K samples per second or something like that.

 ***********************************************

The samples you feed your audio elements look like this

 ******* ******* ******* *******   ******** *******   ****

Notice the gaps. Those are the pops. Notice that the audio takes longer to play back than it does to capture. That's because of the gaps.

Your mission, should you decide to accept it, is to figure out how to put your chunks of audio into a queue so your audio players can play them continuously, starting the next one precisely when the last one left off. You need some sort of "fire and forget" mechanism.

Explaining this completely is a lot for a Stack Overflow question, so I will offer some hints.

  1. Don't start and stop your mediaRecorder. Leave it running.
  2. Put a small number of milliseconds in mediaRecorder.start(nn), like 10
  3. Send your blobs--your sample buffers--over your socket to your server from the ondataavailable event handler, not the onstop handler. That makes them basically stream to the server.
  4. Use the Web Audio API to queue your sample buffers to your user's audio device. Big topic. Look it up.
user_2056080
Related