The pros and cons of different ways to serve and play wav audio files, from Node to the browser

Viewed 330

As I see it we can use native HTMLAudioElement with src, serve a file or stream from Node.js and have a native player consuming that and producing sound.

We can also split it into chunks (HLS/RTMP) and consume them with a JavaScript player that can use it like hls.js or mediaelement. This would help in cases of long audio files (and possibly heavy) so we can start to play before we have all the content. I saw also an interesting project using websockets.

What other ways are there? what are the pros and cons of each? does any of these as the "pro" of being able to encrypt the content being served?

2 Answers

Encrypting audio/video can be done manually by your server. Ideally, using streams.

const crypto = require("crypto")
const fs = require("fs")

const browser = getClientConnectionSomehow()

const encryptor = crypto.createCipheriv("aes256", "32 character long string", Buffer.alloc(16))

fs.createReadStream("path/to/video.mp4").pipe(encryptor).pipe(browser)

You can then send this to the client in a big stream. Another, probably better way to do this, is have the client establish a WebSocket, with the client requesting chunks of video, and the server using writeStreams to pipe the selected portions down to the client.

I ran a quick test on my browser with a 1-minute video and localhost. It just pipes the video to the client, no throttling, and works very well. I didn't implement throttling, but in pseudo-code terms, the way I would do that is:

Client:

<video src="/path/to/video">

</video>
let video = document.getElement("video")
WebSocket.send((currentVideoBuffer - video.currentTime) >= 10)
// say, we wait until there are 10 seconds left in the video to load the next chunk.

Server:

http.createServer((req,res) => {
    let stream = fs.createReadStream("path/to/video.mp4")
    WebSocket.on((shouldSend) => {
        if (shouldSend) {
            res.write(stream.read(500000)
            // just a dummy number of bytes. change as needed.
        }
    })
})

This is grossly oversimplifying how it really would work. You'd need to improve on the pseudo code a lot before it would actually be a viable option. But for livestreaming audio/video, this basically describes all the backend needs. I suggest using this option if you need encryption, too, but I have no idea how the browser would decrypt it and then shove it in an audio element.

Streaming from nodejs to the client is fairly simple (with express, just hook up res.write to the read function of a readStream from FS.)

Node is built for streaming content, and I suggest looking into their native tools. I have not looked into the libraries you've linked, but they seem like viable options.

I think it all comes down to how much tooling you'd want. With your own custom implementation, you'd get to add and remove features to make your dream video/audio player. (I'm sure there is a way, like with Opus streams, to downgrade quality of some streams, making some take less kb/s).

I suppose using a pre-made library is best for getting off the ground, but if your project revolves around this streaming, or you require something like AES-256 or anything unorthodox, I suggest making your own implementation. As I've demonstrated above, it's complex, but not too hard, thanks to nodejs.


Edit

I've found this NodeJS project. It's a video parser, which lets you split a video into chunks of buffers (and get the buffer's length!). This should help you split and stream the buffers.

https://github.com/gkozlenko/node-video-lib

I worked on a music streaming platform. We had to serve upto 1-2 GB audio files as stream. So I did this. Used ffmpeg library for formatting the uploaded audio. ffmpeg should be installed on your machine.

require('child_process').exec(`ffmpeg -y -i ${dir_str} -codec copy -bsf:v h264_mp4toannexb -map 0 -f segment -segment_time 10 -segment_format mpegts -segment_list "${m3u8_path}/${soundId}.m3u8" -segment_list_type m3u8 "${m3u8_path}/ts%d.ts"`)

I think this has the pro of encyrption.

Related