Cloudformation init - install docker

user_7013795 0
Viewed 1516

I am trying to use AWS Cloudformation init to install docker and docker-compose on some ec2 instances.

Before trying this I have just used user data successfully

I am using this template

# Use public Systems Manager Parameter
Parameters:
    LatestAmiId:
    Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
    host1:
    Type: AWS::EC2::Instance
    Metadata:
        AWS::CloudFormation::Init:
            configSets:
                ec2_bootstrap:
                    - install_docker
                    # - install_compose
            install_docker:
                packages:
                    yum:
                        docker: []
                services:
                    sysvinit:
                        docker:
                            enabled: "true"
                            ensureRunning: "true"
                commands:
                    docker_for_ec2_user:
                        command: usermod -G docker ec2-user
            # install_compose:
            #     commands:
            #         compose_for_ec2_user:
            #             command: 
            #               - curl -L https://github.com/docker/compose/releases/download/1.20.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
            #               - chmod +x /usr/local/bin/docker-compose
            #               - ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
    CreationPolicy:
        ResourceSignal:
            Timeout: PT5M
    Properties:
        InstanceType: t2.micro
        KeyName: NVirginia
        # Note we use the pseudo parameter AWS::Region
        ImageId: !Ref LatestAmiId
        AvailabilityZone: !Sub ${AWS::Region}a
        Tags: 
        - Key: "Name"
        Value: "host1"
        SecurityGroupIds:
        - !GetAtt SSHSecurityGroupStack.Outputs.SSHGroupId
        UserData:
        "Fn::Base64":
            !Sub |

        #!/bin/bash -xe
        # Get the latest CloudFormation package
        yum update -y aws-cfn-bootstrap
        # Start cfn-init
        /opt/aws/bin/cfn-init -s ${AWS::StackId} -r host1 --region ${AWS::Region}a || error_exit 'Failed to run cfn-init'
        # Start up the cfn-hup daemon to listen for changes to the EC2 instance metadata
        /opt/aws/bin/cfn-hup || error_exit 'Failed to start cfn-hup'
        # All done so signal success
        /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource host1 --region ${AWS::Region}a
            
            ## change the hostname
            sudo hostname host1
            ## restart docker
            sudo service docker restart
    

    SSHSecurityGroupStack:
    Type: AWS::CloudFormation::Stack
    Properties:
        TemplateURL: https://cloudformation-bruvio-templates.s3.amazonaws.com/ssh-security-group.yaml
        TimeoutInMinutes: 5

the template is supposed to create a security group with a couple of ports open and then an ec2 instance and use init to install docker and docker-compose

I actually copied the init part from this post

The stack fails when trying to create the instance as does not receive the success signal

I wonder what is wrong?

amazon-cloudformation amazon-web-services
2 Answers

There are many issues in your template. Some of them are:

  • missing -configsets
  • wrong region ${AWS::StackId}
  • no hup defined
  • indentation problems
  • missing error_exit

I fixed all of them, and it should work now:

# Use public Systems Manager Parameter
Parameters:
  LatestAmiId:
    Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
  host1:
    Type: AWS::EC2::Instance
    Metadata:
        AWS::CloudFormation::Init:
            configSets:
                ec2_bootstrap:
                    - install_docker
                    # - install_compose
            install_docker:
                packages:
                    yum:
                        docker: []
                services:
                    sysvinit:
                        docker:
                            enabled: "true"
                            ensureRunning: "true"
                commands:
                    docker_for_ec2_user:
                        command: usermod -G docker ec2-user
            # install_compose:
            #     commands:
            #         compose_for_ec2_user:
            #             command: 
            #               - curl -L https://github.com/docker/compose/releases/download/1.20.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
            #               - chmod +x /usr/local/bin/docker-compose
            #               - ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
    CreationPolicy:
        ResourceSignal:
            Timeout: PT5M
    Properties:
        InstanceType: t2.micro
        KeyName: NVirginia
        # Note we use the pseudo parameter AWS::Region
        ImageId: !Ref LatestAmiId
        AvailabilityZone: !Sub ${AWS::Region}a
        Tags: 
        - Key: "Name"
          Value: "host1"
        SecurityGroupIds:
          - !GetAtt SSHSecurityGroupStack.Outputs.SSHGroupId
        UserData:
          "Fn::Base64":
              !Sub |
                #!/bin/bash -xe
                # Get the latest CloudFormation package
                yum update -y aws-cfn-bootstrap
                # Start cfn-init
                /opt/aws/bin/cfn-init -s ${AWS::StackId} -r host1 --configsets ec2_bootstrap --region ${AWS::Region} 
                # All done so signal success
                /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource host1 --region ${AWS::Region}
user_2488230

this is the code I used. I applied @marcin advice.

I create 3 instances in this case (as I want to do some tests with a docker swarm).. only thing is that I also try to install compose. cloudformation does not fail but compose is not installed

    # Use public Systems Manager Parameter
Parameters:
  LatestAmiId:
    Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
  host1:
    Type: AWS::EC2::Instance
    Metadata:
        AWS::CloudFormation::Init:
            configSets:
                ec2_bootstrap:
                    - install_docker
                    - install_compose
            install_docker:
                packages:
                    yum:
                        docker: []
                services:
                    sysvinit:
                        docker:
                            enabled: "true"
                            ensureRunning: "true"
                files:
                  "/etc/cfn/cfn-hup.conf":
                    content: !Sub |
                      [main]
                      stack=${AWS::StackId}
                      region=${AWS::Region}
                    mode: "000400"
                    owner: "root"
                    group: "root"
                  "/etc/cfn/hooks.d/cfn-auto-reloader.conf":
                    content: !Sub |
                        [cfn-auto-reloader-hook]
                        triggers=post.update
                        path=Resources.host1.Metadata.AWS::CloudFormation::Init
                        action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource host1 --region ${AWS::Region}
                    mode: "000400"
                    owner: "root"
                    group: "root"

                commands:
                    docker_for_ec2_user:
                        command: usermod -G docker ec2-user
            install_compose:
                commands:
                    compose_for_ec2_user1:
                        command: 
                          curl -L https://github.com/docker/compose/releases/download/1.20.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
                    compose_for_ec2_user2:
                        command:
                          chmod +x /usr/local/bin/docker-compose
                          
                    compose_for_ec2_user3:
                        command:
                          ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

    CreationPolicy:
        ResourceSignal:
            Timeout: PT2M
    Properties:
        InstanceType: t2.micro
        KeyName: NVirginia
        # Note we use the pseudo parameter AWS::Region
        ImageId: !Ref LatestAmiId
        AvailabilityZone: !Sub ${AWS::Region}a
        Tags: 
        - Key: "Name"
          Value: "host1"
        SecurityGroupIds:
          - !GetAtt SSHSecurityGroupStack.Outputs.SSHGroupId
        UserData:
          "Fn::Base64":
              !Sub |
                #!/bin/bash -xe
                # Get the latest CloudFormation package
                yum update -y aws-cfn-bootstrap
                # Start cfn-init
                /opt/aws/bin/cfn-init -s ${AWS::StackId} -r host1 --configsets ec2_bootstrap --region ${AWS::Region} 
                # All done so signal success
                /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource host1 --region ${AWS::Region}      
                # Start up the cfn-hup daemon to listen for changes to the EC2 instance metadata
                /opt/aws/bin/cfn-hup              

  host2:
    Type: AWS::EC2::Instance
    Metadata:
        AWS::CloudFormation::Init:
            configSets:
                ec2_bootstrap:
                    - install_docker
                    - install_compose
            install_docker:
                packages:
                    yum:
                        docker: []
                services:
                    sysvinit:
                        docker:
                            enabled: "true"
                            ensureRunning: "true"
                files:
                  "/etc/cfn/cfn-hup.conf":
                    content: !Sub |
                      [main]
                      stack=${AWS::StackId}
                      region=${AWS::Region}
                    mode: "000400"
                    owner: "root"
                    group: "root"
                  "/etc/cfn/hooks.d/cfn-auto-reloader.conf":
                    content: !Sub |
                        [cfn-auto-reloader-hook]
                        triggers=post.update
                        path=Resources.host2.Metadata.AWS::CloudFormation::Init
                        action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource host2 --region ${AWS::Region}
                    mode: "000400"
                    owner: "root"
                    group: "root"

                commands:
                    docker_for_ec2_user:
                        command: usermod -G docker ec2-user
            install_compose:
                commands:
                    compose_for_ec2_user1:
                        command: 
                          curl -L https://github.com/docker/compose/releases/download/1.20.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
                    compose_for_ec2_user2:
                        command:
                          chmod +x /usr/local/bin/docker-compose
                          
                    compose_for_ec2_user3:
                        command:
                          ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

    CreationPolicy:
        ResourceSignal:
            Timeout: PT2M
    Properties:
        InstanceType: t2.micro
        KeyName: NVirginia
        # Note we use the pseudo parameter AWS::Region
        ImageId: !Ref LatestAmiId
        AvailabilityZone: !Sub ${AWS::Region}a
        Tags: 
        - Key: "Name"
          Value: "host2"
        SecurityGroupIds:
          - !GetAtt SSHSecurityGroupStack.Outputs.SSHGroupId
        UserData:
          "Fn::Base64":
              !Sub |
                #!/bin/bash -xe
                # Get the latest CloudFormation package
                yum update -y aws-cfn-bootstrap
                # Start cfn-init
                /opt/aws/bin/cfn-init -s ${AWS::StackId} -r host2 --configsets ec2_bootstrap --region ${AWS::Region} 
                # All done so signal success
                /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource host2 --region ${AWS::Region}      
                # Start up the cfn-hup daemon to listen for changes to the EC2 instance metadata
                /opt/aws/bin/cfn-hup


  host3:
    Type: AWS::EC2::Instance
    Metadata:
        AWS::CloudFormation::Init:
            configSets:
                ec2_bootstrap:
                    - install_docker
                    - install_compose
            install_docker:
                packages:
                    yum:
                        docker: []
                services:
                    sysvinit:
                        docker:
                            enabled: "true"
                            ensureRunning: "true"
                files:
                  "/etc/cfn/cfn-hup.conf":
                    content: !Sub |
                      [main]
                      stack=${AWS::StackId}
                      region=${AWS::Region}
                    mode: "000400"
                    owner: "root"
                    group: "root"
                  "/etc/cfn/hooks.d/cfn-auto-reloader.conf":
                    content: !Sub |
                        [cfn-auto-reloader-hook]
                        triggers=post.update
                        path=Resources.host3.Metadata.AWS::CloudFormation::Init
                        action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource host3 --region ${AWS::Region}
                    mode: "000400"
                    owner: "root"
                    group: "root"

                commands:
                    docker_for_ec2_user:
                        command: usermod -G docker ec2-user
            install_compose:
                    compose_for_ec2_user1:
                        command: 
                          curl -L https://github.com/docker/compose/releases/download/1.20.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
                    compose_for_ec2_user2:
                        command:
                          chmod +x /usr/local/bin/docker-compose
                          
                    compose_for_ec2_user3:
                        command:
                          ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

    CreationPolicy:
        ResourceSignal:
            Timeout: PT2M
    Properties:
        InstanceType: t2.micro
        KeyName: NVirginia
        # Note we use the pseudo parameter AWS::Region
        ImageId: !Ref LatestAmiId
        AvailabilityZone: !Sub ${AWS::Region}a
        Tags: 
        - Key: "Name"
          Value: "host3"
        SecurityGroupIds:
          - !GetAtt SSHSecurityGroupStack.Outputs.SSHGroupId
        UserData:
          "Fn::Base64":
              !Sub |
                #!/bin/bash -xe
                # Get the latest CloudFormation package
                yum update -y aws-cfn-bootstrap
                # Start cfn-init
                /opt/aws/bin/cfn-init -s ${AWS::StackId} -r host3 --configsets ec2_bootstrap --region ${AWS::Region} 
                # All done so signal success
                /opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackId} --resource host3 --region ${AWS::Region}      
                # Start up the cfn-hup daemon to listen for changes to the EC2 instance metadata
                /opt/aws/bin/cfn-hup              

  SSHSecurityGroupStack:
    Type: AWS::CloudFormation::Stack
    Properties:
      TemplateURL: https://cloudformation-bruvio-templates.s3.amazonaws.com/ssh-security-group.yaml
      TimeoutInMinutes: 5
user_70137950
Related