A simple problem that I am facing. I am using docker to build a simple image which downloads a file using curl or wget after container is loaded. The file that I am downloading is about 7GB and need 7GB more to extract it in the same container. Below is the code for the image I am building.
FROM node:alpine3.15 as builder
RUN apk update
RUN apk add wget
RUN apk add unzip
COPY package.json ./
COPY index.js ./
ADD startCensusLoad.sh .
RUN npm install
ENTRYPOINT ["/bin/sh","startProcess.sh"]
And this is my startProcess.sh file code.
wget --continue zip_file_download_link_goes_here;
unzip downloaded_zip_file.zip;
npm start;
When I build the image and start a container, the download starts as expected but stops at multiple points like 2 GB or 4 GB or 1.8 GB and so on. I have not been able to download the full file in the container yet. I have also tried curl and seeing the same behavior. Any suggestion on how to solve this issue. The container would run in a kube environment later on so that I can assign an ephemeral volume but for now I am trying to download full file in the container itself.