How to run AWS ECS Task with CloudFormation overriding container environment variables

user_4665092 0
Viewed 3298

I was searching a way to run ecs task. I already have a cluster and task definition settings. I just wanted to trigger a task using CloudFormation template. I know that I can run a task by clicking on the console and it works fine. For cfn, approach needs to be define properly.

Check the attached screenshots. I wanted to run that task using CloudFormation and pass container override environment variables. As per my current templates, it is not allowing me to do same like I can do using console. Using console I just need to select the following options

  1. Launch type 
  2. Task Definition 
              Family
              Revision 
  3. VPC and security groups 
  4. Environment variable overrides rest of the things automatically selected

It starts working with console but with cloudformaton template how can we do that. Is it possible to do or there is no such feature?

      "taskdefinition": {
    "Type" : "AWS::ECS::TaskDefinition",
    "DependsOn": "DatabaseMaster",
    "Properties" : {
        "ContainerDefinitions" : [{
            "Environment" : [
              {
                 "Name" : "TARGET_DATABASE",
                 "Value" : {"Ref":"DBName"}
              },
              {
                 "Name" : "TARGET_HOST",
                 "Value" : {"Fn::GetAtt": ["DatabaseMaster", "Endpoint.Address"]}
              }
            ]
          }],
        "ExecutionRoleArn" : "arn:aws:iam::xxxxxxxxxx:role/ecsTaskExecutionRole",
        "Family" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
        "TaskRoleArn" : "arn:aws:iam::xxxxxxxxxxxxxxx:role/xxxxxxxxxxxxxxx-XXXXXXXXX"
    }
  },
  "EcsService": {
    "Type" : "AWS::ECS::Service",
    "Properties" : {
        "Cluster" : "xxxxxxxxxxxxxxxxx",
        "LaunchType" : "FARGATE",
        "NetworkConfiguration" : {
               "AwsvpcConfiguration" : {
                      "SecurityGroups" : ["sg-xxxxxxxxxxx"],
                      "Subnets" : ["subnet-xxxxxxxxxxxxxx"]
                      }
           },
        "TaskDefinition" : "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      }
  }

enter image description here

ECS Run Task by using aws console

There is no validity error in the code however, I am talking about the approach. I added image name container name but now it is asking for memory and cpu, it should not ask as it is already defined we just need to run a task.

Edited

I wanted to run a task after creation of my database and wanted to pass those database values to the task to run and complete a job.

enter image description here

amazon-cloudformation amazon-ecs amazon-web-services json
2 Answers

Here is the working example of what you can do if you wanted to pass variable and run a task. In my case, I wanted to run a task after creation of my database but with environment variables, directly AWS does not provide any feature to do so, this is the solution which can help to trigger you ecs task.

"IAMRole": {
    "Type": "AWS::IAM::Role",
    "Properties": {
      "AssumeRolePolicyDocument": {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Principal": {
              "Service": [
                "events.amazonaws.com"
              ]
            },
            "Action": [
              "sts:AssumeRole"
            ]
          }
        ]
      },
      "Description": "Allow CloudWatch Events to trigger ECS task",
      "Policies": [
        {
          "PolicyName": "Allow-ECS-Access",
          "PolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
              {
                "Effect": "Allow",
                "Action": [
                  "ecs:*",
                  "iam:PassRole",
                  "logs:CreateLogStream",
                  "logs:PutLogEvents"
                ],
                "Resource": "*"
              }
            ]
          }
        }
      ],
      "RoleName": { "Fn::Join": [ "", ["CloudWatchTriggerECSRole-", { "Ref": "DBInstanceIdentifier" }]]}
    }
  },
  "DummyParameter": {
      "Type" : "AWS::SSM::Parameter",
      "Properties" : {
          "Name" : {"Fn::Sub": "${AWS::StackName}-${DatabaseMaster}-EndpointAddress"},
          "Type" : "String",
          "Value" : {"Fn::GetAtt": "DatabaseMaster.Endpoint.Address"}
      },
      "DependsOn": "TaskSchedule"
  },
  "TaskSchedule": {
    "Type": "AWS::Events::Rule",
    "Properties": {
      "Description": "Trigger ECS task upon creation of DB instance",
      "Name": { "Fn::Join": [ "", ["ECSTaskTrigger-", { "Ref": "DBName" }]]},
      "RoleArn": {"Fn::GetAtt": "IAMRole.Arn"},
      "EventPattern": {
          "source": [ "aws.ssm" ],
          "detail-type": ["Parameter Store Change"] ,
          "resources": [{"Fn::Sub":"arn:aws:ssm:eu-west-1:XXXXXXX:parameter/${AWS::StackName}-${DatabaseMaster}-EndpointAddress"}],
          "detail": {
              "operation": ["Create"],
              "name": [{"Fn::Sub": "${AWS::StackName}-${DatabaseMaster}-EndpointAddress"}],
              "type": ["String"]
          }
      },
      "State": "ENABLED",
      "Targets": [
        {
          "Arn": "arn:aws:ecs:eu-west-1:xxxxxxxx:cluster/NameOf-demo",
          "Id": "NameOf-demo",
          "RoleArn": {"Fn::GetAtt": "IAMRole.Arn"},
          "EcsParameters": {
            "LaunchType": "FARGATE",
            "NetworkConfiguration": {
              "AwsVpcConfiguration": {
                "SecurityGroups": {"Ref":"VPCSecurityGroups"},
                "Subnets": {"Ref":"DBSubnetName"}
              }
            },
            "PlatformVersion": "LATEST",
            "TaskDefinitionArn": "arn:aws:ecs:eu-west-1:XXXXXXXX:task-definition/NameXXXXXXXXX:1"
          },
          "Input": {"Fn::Sub": [
              "{\"containerOverrides\":[{\"name\":\"MyContainerName\",\"environment\":[{\"name\":\"VAR1\",\"value\":\"${TargetDatabase}\"},{\"name\":\"VAR2\",\"value\":\"${TargetHost}\"},{\"name\":\"VAR3\",\"value\":\"${TargetHostPassword}\"},{\"name\":\"VAR4\",\"value\":\"${TargetPort}\"},{\"name\":\"VAR5\",\"value\":\"${TargetUser}\"},{\"name\":\"VAR6\",\"value\":\"${TargetLocation}\"},{\"name\":\"VAR7\",\"value\":\"${TargetRegion}\"}]}]}",
              {
                  "VAR1":      {"Ref":"DBName"},
                  "VAR2":          {"Fn::GetAtt": ["DatabaseMaster", "Endpoint.Address"]},
                  "VAR3":  {"Ref":"DBPassword"},
                  "VAR4":          "5432",
                  "VAR5":          {"Ref":"DBUser"},
                  "VAR6":      "value6",
                  "VAR7":        "eu-west-2"
              }
          ]}
        }
      ]
    }
  }
user_46650920

For Fargate task, we need to specify in CPU in Task Definition. and memory or memory reservation in either task or container definition.

and environment variables should be passed to each container as ContainerDefinitions and overrided when task is run from ecs task-run from console or cli.

{
   "ContainerTaskdefinition": {
      "Type": "AWS::ECS::TaskDefinition",
      "Properties": {
         "Family": "SomeFamily",
         "ExecutionRoleArn": !Ref RoleArn,
         "TaskRoleArn": !Ref TaskRoleArn,
         "Cpu": "256",
         "Memory": "1GB",
         "NetworkMode": "awsvpc",
         "RequiresCompatibilities": [
            "EC2",
            "FARGATE"
         ],
         "ContainerDefinitions": [
            {
               "Name": "container name",
               "Cpu": 256,
               "Essential": "true",
               "Image": !Ref EcsImage,
               "Memory": "1024",
               "LogConfiguration": {
                  "LogDriver": "awslogs",
                  "Options": {
                     "awslogs-group": null,
                     "awslogs-region": null,
                     "awslogs-stream-prefix": "ecs"
                  }
               },
               "Environment": [
                  {
                     "Name": "ENV_ONE_KEY",
                     "Value": "Valu1"
                  },
                  {
                     "Name": "ENV_TWO_KEY",
                     "Value": "Valu2"
                  }
               ]
            }
         ]
      }
   }
}

EDIT(from discussion in comments): ECS Task Run is not a cloud-formation resource, it can only be run from console or CLI. But if we choose to run from a cloudformation resource, it can be done using cloudformation custom resource. But once task ends, we now have a resource in cloudformation without an actual resource behind. So, custom resource needs to do:

  • on create: run the task.
  • on delete: do nothing.
  • on update: re-run the task

Force an update by changing an attribute or logical id, every time we need to run the task.

user_101868080
Related