Is AudioContext / getChannelData deterministic?

user_10578234 0
Viewed 135

I'm analysing an audio file in order to use the channelData to drive another part of my webapp (basically draw graphics based on the audio file). The callback function for the playback looks something like this:

successCallback(mediaStream) {
  var audioContext = new (window.AudioContext ||
    window.webkitAudioContext)();
  source = audioContext.createMediaStreamSource(mediaStream);

  node = audioContext.createScriptProcessor(256, 1, 1);
  node.onaudioprocess = function(data) {
    var monoChannel = data.inputBuffer.getChannelData(0);

    ..
  };

Somehow I thought if I run the above code with the same file it would yield the same results all the time. But that's not the case. The same audio file would trigger the onaudioprocess function sometimes 70, sometimes 72 times for instance, yielding different data all the time. Is there a way to get consistent data of that sort in the browser?

EDIT: I'm getting the audio from a recording function on the same page. When the recording is finished the resulting file gets set as the src of an <audio> element. recorder is my MediaRecorder.

  recorder.addEventListener("dataavailable", function(e) {
    fileurl = URL.createObjectURL(e.data);
    document.querySelector("#localaudio").src = fileurl;
    ..
html5-audio html javascript
1 Answers

To answer your original question: getChannelData is deterministic, i.e. it will yield the same Float32Array from the same AudioBuffer for the same channel (unless you happen to transfer the backing ArrayBuffer to another thread, in which case it will return an empty Float32Array with a detached backing buffer from then on).

I presume the problem you are encountering here is a threading issue (my guess is that the MediaStream is already playing before you start processing the audio stream from it), but it's hard to tell exactly without debugging your complete app (there are at least 3 threads at work here: an audio processing thread for the MediaStream, an audio processing thread for the AudioContext you are using, and the main thread that runs your code).

Is there a way to get consistent data of that sort in the browser?

Yes.

Instead of processing through a real-time audio stream for real-time analysis, you could just take the recording result (e.data), read it as an ArrayBuffer, and then decode it as an AudioBuffer, something like:

recorder.addEventListener("dataavailable", function (e) {
    let reader = new FileReader();
    
    reader.onload = function (e) {
        audioContext.decodeAudioData(e.target.result).then(function (audioBuffer) {
            var monoChannel = audioBuffer.getChannelData(0);
            // monoChannel contains the entire first channel of your recording as a Float32Array
            // ...
        });
    };
    
    reader.readAsArrayBuffer(e.data);
}

Note: this code would become a lot simpler with async functions and Promises, but it should give a general idea of how to read the entire completed recording.

Also note: the ScriptProcessorNode is deprecated due to performance issues inherent in cross-thread data copy, especially involving the JS main thread. The preferred alternative is the much more advanced AudioWorklet, but this is a fairly new way to do things on the web and requires a solid understanding of worklets in general.

user_27888720
Related