Downloading large files with axios

user_1918669 0
Viewed 997

I am currently downloading large files to my client code using the axios lib. Files are around 600MB. But during the download process the page crashes, like it runs out of memory or similar.

I need to hold the file in the memmory because the content is encrypted and I need to decrypt it before passing it to the user.

I use the REST GET Http request like this:

  axios.get(url, {
            headers: { 
                "Authorization": authHeader().Authorization,
                "Accept" : "application/octet-stream, application/json, text/plain, */*"            
            },   responseType: 'arraybuffer'

          })
          .then(function(response) {
          console.log(response);

Are there any common workaround around the problem. So far I wasn't able to find any.

axios javascript
5 Answers

Open the url in the new tab on the client side using window.open(url)

Let the browser handle the document automatically, If you want to decrypt the data, please try to decrypt on server side since you'll be giving out decryption key on the client side, which can have security issues.

user_193565910

Do you actually need to do it with axios? There is the Fetch API, which can serve the purpose. Here's how I do it for files in the same size range as yours (media files and ZIPs of up to 1 GB):

        fetch(url, {
            mode: 'no-cors', // to allow any accessible resource
            method: 'GET',
        })
            .then((response) => {
                console.debug('LOAD_FROM_URL::response', response);
                //NOTE: response URL is possibly redirected

See https://github.com/suterma/replayer-pwa/blob/main/src/store/actions.ts for more context.

It's working flawless so far for me.

user_794850

Can you please give a try by setting maxContentLength and maxBodyLength to Infinity in the axios call.

axios.get(url, {
  headers: { 
    "Authorization": authHeader().Authorization,
    "Accept" : "application/octet-stream, application/json, text/plain, */*"            
  },
  responseType: 'arraybuffer',
  maxContentLength: Infinity,
  maxBodyLength: Infinity
})
  .then(function(response) {
  console.log(response);
}

You can also have a look into this axios issue forum for the same.

user_41163000

HTTP Protocol: Range Parameter

Use the 'Range' header for getting big files part by part.

Example Curl HTTP Request : https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

curl http://i.imgur.com/z4d4kWk.jpg -i -H "Range: bytes=0-1023"

Example JS

axios.get(url, {
  headers: { 
    "Authorization": authHeader().Authorization,
    "Accept" : "application/octet-stream, application/json, text/plain, */*",
    //--------------------------|
    "Range" : "bytes=0-1023"    // <------ ADD HERE 
    //--------------------------|
  },
  responseType: 'arraybuffer'
}).then(function(response) {
  console.log(response);
}
user_71465520

You need to revisit the design of your frontend. Either its ReactJs or Angular or plain JS, just Offload such large downloads as well as other long running scritps to a concept of JavaScript Web Workers. This is latest way to offloading javascript tasks to the background so the page responsiveness remains intact.

Otherwise JavaScript is single threaded so there is no way to bypass the issue. WebWorkers are only thing to make exceptions here

It is as of today now supported on all latest browsers.

More on this: link

user_26484900
Related