Elastic Beanstalk with CloudWatch logs and Nginx

You can use the following .ebextensions/awslog.config:

container_commands:
  touch_logs:
    command: touch /var/log/nodejs/nodejs.log /var/log/nginx/access.log /var/log/nginx/error.log /var/log/httpd/access_log /var/log/httpd/error_log

This ensures you never see the "No file is found with given path" for any of these files.

Note: If you're using a container other than node, you can replace /var/log/nodejs.log with whatever is found in beanstalklogs.conf...

The awslogs config file is /etc/awslogs/config/beanstalklogs.conf, the default (for Node):

[/var/log/nodejs/nodejs.log]
log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/nodejs/nodejs.log
log_stream_name={instance_id}
file=/var/log/nodejs/nodejs.log*

[/var/log/nginx/error.log]
log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/nginx/error.log
log_stream_name={instance_id}
file=/var/log/nginx/error.log*

[/var/log/nginx/access.log]
log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/nginx/access.log
log_stream_name={instance_id}
file=/var/log/nginx/access.log*

[/var/log/httpd/error_log]
log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/httpd/error_log
log_stream_name={instance_id}
file=/var/log/httpd/error_log*

[/var/log/httpd/access_log]
log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/httpd/access_log
log_stream_name={instance_id}
file=/var/log/httpd/access_log*

[/var/log/eb-activity.log]
log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/eb-activity.log
log_stream_name={instance_id}
file=/var/log/eb-activity.log*

As you can see this has both nginx and httpd lines.

One could either: 1. remove these httpd blocks. 2. touch these files.

Touching these files seems less invasive and suppresses the error messages so nothing is posted to cloudwatch logs:

sudo touch /var/log/httpd/error_log /var/log/httpd/access_log

Note: When EB first boots up you get errors for httpd, nginx and nodejs.log! So it seems easier to touch all of these files at boot time, which is what the awslog.config file above does.

Another way to solve this problem is to change /etc/awslogs/config/beanstalklogs.conf file.

You can create an .ebextensions/myconfig.config file that contains the following contents:

files:
  "/etc/awslogs/config/beanstalklogs.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      [/var/log/nodejs/nodejs.log]
      log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/nodejs/nodejs.log
      log_stream_name={instance_id}
      file=/var/log/nodejs/nodejs.log*

      [/var/log/nginx/error.log]
      log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/nginx/error.log
      log_stream_name={instance_id}
      file=/var/log/nginx/error.log*

      [/var/log/nginx/access.log]
      log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/nginx/access.log
      log_stream_name={instance_id}
      file=/var/log/nginx/access.log*

      # [/var/log/httpd/error_log]
      # log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/httpd/error_log
      # log_stream_name={instance_id}
      # file=/var/log/httpd/error_log*

      # [/var/log/httpd/access_log]
      # log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/httpd/access_log
      # log_stream_name={instance_id}
      # file=/var/log/httpd/access_log*

      [/var/log/eb-activity.log]
      log_group_name=/aws/elasticbeanstalk/YOUR_ENV/var/log/eb-activity.log
      log_stream_name={instance_id}
      file=/var/log/eb-activity.log*
commands:
  01-remove-default-beanstalklogs:
    command: rm /etc/awslogs/config/beanstalklogs.conf.bak
  02-restart-awslogs:
    command: service awslogs restart

Note that I have commented out the lines for uploading /var/log/httpd/error.log* and /var/log/httpd/access.log* files. So CloudWatch agent will stop looking for these files.

Awslogs uses the default beanstalklogs.conf mentioned by @AndyHayden before running the commands in myconfig.config. That's why I ask awslogs to restart after beanstalklogs.conf file is changed.

Place .ebextensions/myconfig.config in the *.zip file that you uploaded to elastic beanstalk. Then elastic beanstalk will execute the commands in myconfig.config.

Here is the document for the usage of commands: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#linux-commands

I also had this issue in one of my environment and I was facing this issue with the environment where there was autoscaling of instances configured and that too in the environment where there was too frequent scaling happening.

Looked at the awslogs in /var/logs dir and it had this error

cwlogs.push.stream - WARNING - 3317 - Thread-1 - No file is found with given path '/var/log/production.log*'.

I tried creating a file using the command

sudo touch /var/log/production.log

But that stopped the errors but that did not stream the logs even after restarting.

And finally looking at the other instances where there was a log streaming properly, it was symbolic link to the file where the actual log was being returned and so created a symbolic link from the ebextension through container command

commands:
  "01":
    command: ln -sf /var/app/current/log/production.log /var/log/production.log

This worked for me finally.