How do I make cloud-init startup scripts run every time my EC2 instance boots?

user_12171 0
Viewed 91922

I have an EC2 instance running an AMI based on the Amazon Linux AMI. Like all such AMIs, it supports the cloud-init system for running startup scripts based on the User Data passed into every instance. In this particular case, my User Data input happens to be an Include file that sources several other startup scripts:

#include
http://s3.amazonaws.com/path/to/script/1
http://s3.amazonaws.com/path/to/script/2

The first time I boot my instance, the cloud-init startup script runs correctly. However, if I do a soft reboot of the instance (by running sudo shutdown -r now, for instance), the instance comes back up without running the startup script the second time around. If I go into the system logs, I can see:

Running cloud-init user-scripts
user-scripts already ran once-per-instance
[  OK  ]

This is not what I want -- I can see the utility of having startup scripts that only run once per instance lifetime, but in my case these should run every time the instance starts up, like normal startup scripts.

I realize that one possible solution is to manually have my scripts insert themselves into rc.local after running the first time. This seems burdensome, however, since the cloud-init and rc.d environments are subtly different and I would now have to debug scripts on first launch and all subsequent launches separately.

Does anyone know how I can tell cloud-init to always run my scripts? This certainly sounds like something the designers of cloud-init would have considered.

cloud-init cloud amazon-ec2
8 Answers

I struggled with this issue for almost two days, tried all of the solutions I could find and finally, combining several approaches, came up with the following:

MyResource:
  Type: AWS::EC2::Instance
  Metadata:
    AWS::CloudFormation::Init:
      configSets:
        setup_process:
          - "prepare"
          - "run_for_instance"
      prepare:
        commands:
          01_apt_update:
            command: "apt-get update"
          02_clone_project:
            command: "mkdir -p /replication && rm -rf /replication/* && git clone https://github.com/awslabs/dynamodb-cross-region-library.git /replication/dynamodb-cross-region-library/"
          03_build_project:
            command: "mvn install -DskipTests=true"
            cwd: "/replication/dynamodb-cross-region-library"
          04_prepare_for_apac:
            command: "mkdir -p /replication/replication-west && rm -rf /replication/replication-west/* && cp /replication/dynamodb-cross-region-library/target/dynamodb-cross-region-replication-1.2.1.jar /replication/replication-west/replication-runner.jar"
      run_for_instance:
        commands:
          01_run:
            command: !Sub "java -jar replication-runner.jar --sourceRegion us-east-1 --sourceTable ${TableName} --destinationRegion ap-southeast-1 --destinationTable ${TableName} --taskName -us-ap >/dev/null 2>&1 &"
            cwd: "/replication/replication-west"
  Properties:
    UserData:
      Fn::Base64:
        !Sub |
          #cloud-config
          cloud_final_modules:
           - [scripts-user, always]
          runcmd:
           - /usr/local/bin/cfn-init -v -c setup_process --stack ${AWS::StackName} --resource MyResource --region ${AWS::Region}
           - /usr/local/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource MyResource --region ${AWS::Region}

This is the setup for DynamoDb cross-region replication process.

user_51515750

please use the below script above your bash script.

example: here m printing hello world to my file

stop instance before adding to userdata

script

Content-Type: multipart/mixed; boundary="//"
MIME-Version: 1.0

--//
Content-Type: text/cloud-config; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="cloud-config.txt"

#cloud-config
cloud_final_modules:
- [scripts-user, always]

--//
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="userdata.txt"

#!/bin/bash
/bin/echo "Hello World." >> /var/tmp/sdksdfjsdlf
--//
user_104906480

If someone wants to do this on CDK, here's a python example.

For Windows, user data has a special persist tag, but for Linux, you need to use MultiPart User data to setup cloud-init first. This Linux example worked with cloud-config (see ref blog) part type instead of cloud-boothook which requires a cloud-init-per (see also bootcmd) call I couldn't test out (eg: cloud-init-pre always).

Linux example:

    # Create some userdata commands
    instance_userdata = ec2.UserData.for_linux()
    instance_userdata.add_commands("apt update")
    # ...
    # Now create the first part to make cloud-init run it always
    cinit_conf = ec2.UserData.for_linux();
    cinit_conf .add_commands('#cloud-config');
    cinit_conf .add_commands('cloud_final_modules:');
    cinit_conf .add_commands('- [scripts-user, always]');
    multipart_ud = ec2.MultipartUserData()
    #### Setup to run every time instance starts
    multipart_ud.add_part(ec2.MultipartBody.from_user_data(cinit_conf , content_type='text/cloud-config'))
    #### Add the commands desired to run every time
    multipart_ud.add_part(ec2.MultipartBody.from_user_data(instance_userdata));

    ec2.Instance(
        self, "myec2",
        userdata=multipart_ud,
        #other required config...
    )

Windows example:

    instance_userdata = ec2.UserData.for_windows()
    # Bootstrap
    instance_userdata.add_commands("Write-Output 'Run some commands'")
    # ...
    # Making all the commands persistent - ie: running on each instance start
    data_script = instance_userdata.render()
    data_script += "<persist>true</persist>"
    ud = ec2.UserData.custom(data_script)
    ec2.Instance(
        self, "myWinec2",
        userdata=ud,
        #other required config...
    )
user_17366790

Another approach is to use #cloud-boothook in your user data script. From the docs:

Cloud Boothook

  • Begins with #cloud-boothook or Content-Type: text/cloud-boothook.
  • This content is boothook data. It is stored in a file under /var/lib/cloud and then executed immediately.
  • This is the earliest "hook" available. There is no mechanism provided for running it only one time. The boothook must take care of this itself. It is provided with the instance ID in the environment variable INSTANCE_ID. Use this variable to provide a once-per-instance set of boothook data.
user_18665960
Related