how to pass bucket/key name to fargate job via a cloudwatch event trigger on s3 object creation event?

Viewed 42

I have create a fargate task and trying to trigger it via s3 object creation event ( see sample below) via cloudformation.as it cannot trigger it directly, i have created a cloudwatchevent. I am trying to pass the bucket and obj name to my fargate task code . doing some research, i came across -> InputTransformer, but i'm not sure how to pass the value of my bucket and key name and how to read it in my python code. any help will be appreciated.

AWSTemplateFormatVersion: 2010-09-09
Description: An example CloudFormation template for Fargate.
Parameters:
  VPC:
    Type: AWS::EC2::VPC::Id
  SubnetA:
    Type: AWS::EC2::Subnet::Id
  SubnetB:
    Type: AWS::EC2::Subnet::Id
  Image:
    Type: String
    Default: 123456789012.dkr.ecr.region.amazonaws.com/image:tag
Resources:
  mybucket:
    Properties:
      BucketName: 'mytestbucket-us'

  cloudwatchEvent:
    Type: AWS::Events::Rule
    Properties:
      EventPattern:
        source: 
          - aws.s3
        detail:
          eventSource: 
            - s3.amazonaws.com
          eventName: 
            - PutObject
            - CompleteMultipartUpload
          requestParameters:
            bucketName:
              - !Ref mybucket
    Targets:
      - Id: my-fargate-task
        Arn: myclusterArn
        RoleArn: myinvocationrolearn
        Input:
          'Fn::Sub':
             - >-
               {"containerOverrides": [{"name":"somecontainer"]}
        EcsParameters:
          TaskDefinition:
            LaunchType: 'FARGATE'
            ...
          NetworkConfiguration:
            ...
  
  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Join ['', [!Ref ServiceName, Cluster]]

  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    DependsOn: LogGroup
    Properties:
      Family: !Join ['', [!Ref ServiceName, TaskDefinition]]
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      Cpu: 256
      Memory: 2GB
      ExecutionRoleArn: !Ref ExecutionRole
      TaskRoleArn: !Ref TaskRole
      ContainerDefinitions:
        - Name: !Ref ServiceName
          Image: !Ref Image

  # A role needed by ECS
  ExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Join ['', [!Ref ServiceName, ExecutionRole]]
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy'
  # A role for the containers
  TaskRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Join ['', [!Ref ServiceName, TaskRole]]
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: ecs-tasks.amazonaws.com
            Action: 'sts:AssumeRole'
2 Answers

You can't. CloudWatch events do not pass data to ECS jobs. You need to develop your own mechanism for that. For example, trigger lambda first, store event in S3 Parameter Store or DynamoDB, and then invoke your ECS job which will get stored data.

Related