AWS Elasticbeanstalk post deploy script problem

Question

I'm trying to run the celery worker after my application has been deployed to AWS Elasticbeanstalk.

99_celery_start.sh

#!/usr/bin/env bash

#make dirs
sudo mkdir -p /usr/etc/
sudo chmod 755 /usr/etc/
sudo touch /usr/etc/celery.conf
sudo touch /usr/etc/supervisord.conf

# Get django environment variables
celeryenv=`cat /var/app/rootfolder/myprject/.env | tr '
' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g' | sed 's/%/%%/g'`
celeryenv=${celeryenv%?}

# Create celery configuration script
celeryconf="[program:celery-worker]
user=root
directory=/var/app/rootfolder
; Set full path to celery program if using virtualenv
command=/var/app/venv/*/bin/celery -A myprject worker -P solo --loglevel=INFO        
.
.
.
.

celery.config

container_commands:
  01_celery_configure:
    command: "mkdir -p /.platform/hooks/postdeploy/ && cp .ebextensions/99_celery_start.sh /.platform/hooks/postdeploy/ && chmod 774 /.platform/hooks/postdeploy/99_celery_start.sh"
  02_run_celery:
    command: "sudo /.platform/hooks/postdeploy/99_celery_start.sh"

So what I'm trying is to copy the celery-worker script in the .ebextension folder and paste it into the post-deploy hook folder so that the script runs after the application is deployed on the instance.But the command 02_run_celery is executing before the application is extracted and deployed on the instance. Since the script requires the application folder /var/app/rootfolder/myprjct/.env, the deployment process gives an error cat: /var/app/rootfolder/myprjct/.env: No such file or directory.

user_1011253 · Answer

If your assumption is right and this is a race condition (the second command execute before the application deployed) - what about waiting for the application deployment? or waiting for that specific file you're interested in?

...

# Wait for the app deployment
while [ ! -f /var/app/rootfolder/myprject/.env ]
do
  sleep 1
  echo "Waiting for the application deployment"
done


# Get django environment variables
celeryenv=`cat /var/app/rootfolder/myprject/.env | tr '
' ',' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g' | sed 's/%/%%/g'`
celeryenv=${celeryenv%?}
...

user_11559079 · Answer

If anyone still looking for the answer then this is how I did it:

Folder structure:

|-- .ebextensions/
|   |-- celery.config        # Option settings
|   `-- cloudwatch.config     # Other .ebextensions sections, for example files and container commands
`-- .platform/
    |-- nginx/                # Proxy configuration
    |   |-- nginx.conf
    |   `-- conf.d/
    |       `-- custom.conf
    |-- hooks/                # Application deployment hooks
    |   `-- postdeploy/
    |       `-- 99_celery_start.sh

Now add permissions for 99_celery_start.script in celery.config:

01_celery_perm:
    command: "sudo chmod +x .platform/hooks/postdeploy/99_celery_start.sh"
02_dos2unix:
    command: "perl -i -pe's/\r$//;' .platform/hooks/postdeploy/99_celery_start.sh"

IMPORTANT: Make sure the script should be saved in LF line endings instead of CRLF.

AWS Elasticbeanstalk post deploy script problem

2 Answers