Using the RUN instruction in a Dockerfile with 'source' does not work

Viewed 360909

I have a Dockerfile that I am putting together to install a vanilla python environment (into which I will be installing an app, but at a later date).

FROM ubuntu:12.04

# required to build certain python libraries
RUN apt-get install python-dev -y

# install pip - canonical installation instructions from pip-installer.org
# http://www.pip-installer.org/en/latest/installing.html
ADD https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py /tmp/ez_setup.py
ADD https://raw.github.com/pypa/pip/master/contrib/get-pip.py /tmp/get-pip.py
RUN python /tmp/ez_setup.py
RUN python /tmp/get-pip.py
RUN pip install --upgrade pip 

# install and configure virtualenv
RUN pip install virtualenv 
RUN pip install virtualenvwrapper
ENV WORKON_HOME ~/.virtualenvs
RUN mkdir -p $WORKON_HOME
RUN source /usr/local/bin/virtualenvwrapper.sh

The build runs ok until the last line, where I get the following exception:

[previous steps 1-9 removed for clarity]
...
Successfully installed virtualenvwrapper virtualenv-clone stevedore
Cleaning up...
 ---> 1fc253a8f860
Step 10 : ENV WORKON_HOME ~/.virtualenvs
 ---> Running in 8b0145d2c80d
 ---> 0f91a5d96013
Step 11 : RUN mkdir -p $WORKON_HOME
 ---> Running in 9d2552712ddf
 ---> 3a87364c7b45
Step 12 : RUN source /usr/local/bin/virtualenvwrapper.sh
 ---> Running in c13a187261ec
/bin/sh: 1: source: not found

If I ls into that directory (just to test that the previous steps were committed) I can see that the files exist as expected:

$ docker run 3a87 ls /usr/local/bin
easy_install
easy_install-2.7
pip
pip-2.7
virtualenv
virtualenv-2.7
virtualenv-clone
virtualenvwrapper.sh
virtualenvwrapper_lazy.sh

If I try just running the source command I get the same 'not found' error as above. If I RUN an interactive shell session however, source does work:

$ docker run 3a87 bash
source
bash: line 1: source: filename argument required
source: usage: source filename [arguments]

I can run the script from here, and then happily access workon, mkvirtualenv etc.

I've done some digging, and initially it looked as if the problem might lie in the difference between bash as the Ubuntu login shell, and dash as the Ubuntu system shell, dash not supporting the source command.

However, the answer to this appears to be to use '.' instead of source, but this just causes the Docker runtime to blow up with a go panic exception.

What is the best way to run a shell script from a Dockerfile RUN instruction to get around this (am running off the default base image for Ubuntu 12.04 LTS).

18 Answers

The default shell for the RUN instruction is ["/bin/sh", "-c"].

RUN "source file"      # translates to: RUN /bin/sh -c "source file"

Using SHELL instruction, you can change default shell for subsequent RUN instructions in Dockerfile:

SHELL ["/bin/bash", "-c"] 

Now, default shell has changed and you don't need to explicitly define it in every RUN instruction

RUN "source file"    # now translates to: RUN /bin/bash -c "source file"

Additional Note: You could also add --login option which would start a login shell. This means ~/.bashrc for example would be read and you don't need to source it explicitly before your command

If you have SHELL available you should go with this answer -- don't use the accepted one, which forces you to put the rest of the dockerfile in one command per this comment.

If you are using an old Docker version and don't have access to SHELL, this will work so long as you don't need anything from .bashrc (which is a rare case in Dockerfiles):

ENTRYPOINT ["bash", "--rcfile", "/usr/local/bin/virtualenvwrapper.sh", "-ci"]

Note the -i is needed to make bash read the rcfile at all.

This is my solution on "Ubuntu 20.04"

RUN apt -y update
RUN apt -y install curl
SHELL ["/bin/bash", "-c"]
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
RUN source /root/.bashrc
RUN bash -c ". /root/.nvm/nvm.sh && nvm install v16 && nvm alias default v16 && nvm use default"

Here is an example Dockerfile leveraging several clever techniques to all you to run a full conda environment for every RUN stanza. You can use a similar approach to execute any arbitrary prep in a script file.

Note: there is a lot of nuance when it comes to login/interactive vs nonlogin/noninteractive shells, signals, exec, the way multiple args are handled, quoting, how CMD and ENTRYPOINT interact, and a million other things, so don't be discouraged if when hacking around with these things, stuff goes sideways. I've spent many frustrating hours digging through all manner of literature and I still don't quite get how it all clicks.

## Conda with custom entrypoint from base ubuntu image
## Build with e.g. `docker build -t monoconda .`
## Run with `docker run --rm -it monoconda bash` to drop right into
## the environment `foo` !
FROM ubuntu:18.04

## Install things we need to install more things
RUN apt-get update -qq &&\
    apt-get install -qq curl wget git &&\
    apt-get install -qq --no-install-recommends \
        libssl-dev \
        software-properties-common \
    && rm -rf /var/lib/apt/lists/*

## Install miniconda
RUN wget -nv https://repo.anaconda.com/miniconda/Miniconda3-4.7.12-Linux-x86_64.sh -O ~/miniconda.sh && \
    /bin/bash ~/miniconda.sh -b -p /opt/conda && \
    rm ~/miniconda.sh && \
    /opt/conda/bin/conda clean -tipsy && \
    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh

## add conda to the path so we can execute it by name
ENV PATH=/opt/conda/bin:$PATH

## Create /entry.sh which will be our new shell entry point. This performs actions to configure the environment
## before starting a new shell (which inherits the env).
## The exec is important! This allows signals to pass
RUN     (echo '#!/bin/bash' \
    &&   echo '__conda_setup="$(/opt/conda/bin/conda shell.bash hook 2> /dev/null)"' \
    &&   echo 'eval "$__conda_setup"' \
    &&   echo 'conda activate "${CONDA_TARGET_ENV:-base}"' \
    &&   echo '>&2 echo "ENTRYPOINT: CONDA_DEFAULT_ENV=${CONDA_DEFAULT_ENV}"' \
    &&   echo 'exec "$@"'\
        ) >> /entry.sh && chmod +x /entry.sh

## Tell the docker build process to use this for RUN.
## The default shell on Linux is ["/bin/sh", "-c"], and on Windows is ["cmd", "/S", "/C"]
SHELL ["/entry.sh", "/bin/bash", "-c"]
## Now, every following invocation of RUN will start with the entry script
RUN     conda update conda -y

## Create a dummy env
RUN     conda create --name foo

## I added this variable such that I have the entry script activate a specific env
ENV CONDA_TARGET_ENV=foo

## This will get installed in the env foo since it gets activated at the start of the RUN stanza
RUN  conda install pip

## Configure .bashrc to drop into a conda env and immediately activate our TARGET env
RUN conda init && echo 'conda activate "${CONDA_TARGET_ENV:-base}"' >>  ~/.bashrc
ENTRYPOINT ["/entry.sh"]

I've dealing with a similar scenario for an application developed with Django web web framework and these are the steps that worked perfectly for me:

  • content of my Dockerfile
[mlazo@srvjenkins project_textile]$ cat docker/Dockerfile.debug 
FROM malazo/project_textile_ubuntu:latest 

ENV PROJECT_DIR=/proyectos/project_textile PROJECT_NAME=project_textile WRAPPER_PATH=/usr/share/virtualenvwrapper/virtualenvwrapper.sh

COPY . ${PROJECT_DIR}/
WORKDIR ${PROJECT_DIR}

RUN echo "source ${WRAPPER_PATH}" > ~/.bashrc
SHELL ["/bin/bash","-c","-l"]
RUN     mkvirtualenv -p $(which python3) ${PROJECT_NAME} && \
        workon ${PROJECT_NAME} && \
        pip3 install -r requirements.txt 

EXPOSE 8000

ENTRYPOINT ["tests/container_entrypoint.sh"]
CMD ["public/manage.py","runserver","0:8000"]

  • content of the ENTRYPOINT file "tests/container_entrypoint.sh":
[mlazo@srvjenkins project_textile]$ cat tests/container_entrypoint.sh
#!/bin/bash
# *-* encoding : UTF-8 *-*
sh tests/deliver_env.sh
source ~/.virtualenvs/project_textile/bin/activate 
exec python "$@"

  • finally, the way I deploy the container was :
[mlazo@srvjenkins project_textile]$ cat ./tests/container_deployment.sh 
#!/bin/bash

CONT_NAME="cont_app_server"
IMG_NAME="malazo/project_textile_app"
[ $(docker ps -a |grep -i ${CONT_NAME} |wc -l) -gt 0 ] && docker rm -f ${CONT_NAME} 
docker run --name ${CONT_NAME} -p 8000:8000 -e DEBUG=${DEBUG} -e MYSQL_USER=${MYSQL_USER} -e MYSQL_PASSWORD=${MYSQL_PASSWORD} -e MYSQL_HOST=${MYSQL_HOST} -e MYSQL_DATABASE=${MYSQL_DATABASE} -e MYSQL_PORT=${MYSQL_PORT}  -d ${IMG_NAME}

I really hope this would be helpful for somebody else.

Greetings,

I had the same issue. If you also use a python base image you can change the shebang line in your shell script to #!/bin/bash. See for example the container_entrypoint.sh from Manuel Lazo.

Related