■Background
In Google Cloud Run using AXIOS with node.js for file downloading from atlassian Confluence, when the content-length of the response is greater than 100M, there is a high probability that the download will not succeed.
■What I did
Read response data in chunks. It found that a request oborted exception occurred when it reached the end. The response data was not read completely. In the local environment (out of Google Cloud Run), a 100M file can be downloaded successfully.
const writer = fs.createWriteStream(filepath);
const response = await Axios({
url: fileUrl,
method: "GET",
responseType: "stream",
validateStatus : null,
httpAgent: new http.Agent({ keepAlive: true }),
httpsAgent: new https.Agent({ keepAlive: true })
}).catch(err => {
console.log("axios error: "+ err);
});
const contentLength = response.headers['content-length'];
console.log("contentLength: " + contentLength);
let chunklength = 0;
response.data.on("data", (chunk) => {
chunklength = chunklength + chunk.length;
console.log("chunk-length-sum: "+ chunklength);
writer.write(chunk);
});
response.data.on("aborted", () => {
});
response.data.on("close", () => {
});
response.data.on("end", () => {
});
writer.on("finish", async () => {
})
■What I expect
Used the AXIOS with node.js can download files.
■Environment
node.js 18.7 AXIOS 0.26.2 Google Cloud Run
■Question
1)Are there some limits in the Google Cloud Run?
Best regards