Good morning.
I'm a newbie working with docker and I was recently asked to move 2 containers to use a non-default user.I created a new user within the Dockerfile and successfully created the images. However when I execute docker-compose run workflow coverage run --append src/workflows/batch_inference/producer.py I get the following error:
"Running child workflow"
docker-compose run workflow coverage run --append src/workflows/batch_inference/producer.py
Creating service_training_1 ... done
Creating service_workflow_run ... done
Couldn't use data file '/mnt/test-results/.coverage': unable to open database file
However, doing the same thing with another service ( docker-compose run training coverage run --append src/workflows/tag_mapping/mapping.py) doesn't suffer from the same issue.
My question is, what could I be doing wrong or missing?
Here are the files that I'm using:
Parent image (training image/Dockerfile.training):
#initial image named training
FROM private_image
ARG USER=developer ### added this line
ARG PW=developer ### added this line
ARG UID=1001 ### added this line
ARG GID=1001 ### added this line
WORKDIR /app
RUN useradd -ms /bin/zsh ${USER} && usermod -aG sudo ${USER} ### added this line
COPY requirements.txt /app
COPY .coveragerc /app
RUN python3 -m pip install --no-cache-dir -r requirements.txt
COPY src /app/src
COPY datasets /app/datasets
RUN chown -R ${USER} /app /mnt ### added this line
USER ${USER} ### added this line
Child image (workflow image/Dockerfile.workflow):
#child image named workflow
FROM training
ARG USER=developer ### added this line
ARG PW=developer ### added this line
ARG UID=1001 ### added this line
ARG GID=1001 ### added this line
COPY src/workflows /app/src/workflows
RUN chown -R ${USER} /mnt ### added this line
USER ${USER} ### added this line
docker-compose.yaml:
version: "3.8"
services:
training:
image: training
build:
context: .
dockerfile: ./Dockerfile.training
args:
UID: 1001
GID: 1001
environment:
# use minio instead of S3
- MINIO_ENDPOINT=minio:9000
- USE_MINIO=true
workflow:
image: workflow
build:
context: .
dockerfile: ././Dockerfile.workflow
args:
UID: 1001
GID: 1001
depends_on:
- training
environment:
- WORKFLOW_ID=test1
- S3_DATA_FOLDER=fictional_folder
- MINIO_ENDPOINT=minio:9000
- USE_MINIO=true
- PARTITION_SIZE=100
volumes:
- type: volume
source: artifacts-vol
target: /mnt/artifacts
# needed for measuring coverage
- type: volume
source: test-results-vol
target: /mnt/test-results
zookeeper:
image: wurstmeister/zookeeper
hostname: zookeeper
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka
hostname: kafka
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'test'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
KAFKA_LISTENERS: INSIDE://:9094,OUTSIDE://:9092
KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9094,OUTSIDE://localhost:9092
KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
depends_on:
- zookeeper
ports:
- "9092:9092"
minio:
image: minio/minio:RELEASE.2022-05-19T18-20-59Z
ports:
- "9000:9000"
- "9001:9001"
volumes:
- type: volume
source: minio-vol
target: /mnt/minio
entrypoint: minio server --console-address ":9001" /mnt/minio
volumes:
test-results-vol:
minio-vol:
artifacts-vol:
Unfortunately, I can't share the entire code as it is too long and/or the python files to run the coverage (apologies in advance). I am aware that by not putting the complete code it may be impossible to help me, however, I would like to know if you could guide me in identifying in which part the error might be so that I can make the necessary changes.
I would also appreciate any comments or suggestions on how I can improve or make the Dockerfile more succinct with the non-root user changes and improve my understanding of how docker works.