I am trying to get logging working with EMR on EKS (a.k.a. emr-containers).
I am using this configuration:
"monitoringConfiguration": {
"cloudWatchMonitoringConfiguration": {
"logGroupName": "/emr-containers/jobs",
"logStreamNamePrefix": "demo"
},
"s3MonitoringConfiguration": {
"logUri": "s3://my-bucket/my-prefix"
}
And my execution role is correct:
policy = jsonencode({
"Version": "2012-10-17",
"Statement": [
{
Effect: "Allow",
Action: [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket"
],
Resource: "*"
},
{
Effect: "Allow",
Action: [
"logs:PutLogEvents",
"logs:CreateLogStream",
"logs:DescribeLogGroups",
"logs:DescribeLogStreams"
],
Resource: [
"arn:aws:logs:*:*:*"
]
}
]
})
However, when I look at cloudwatch, there is nothing in this log group, and similarly when I look at S3, there is nothing there.
It doesn't matter whether the job succeeds or fails -- in all cases there are no logs persisted either to cloudwatch or s3.
Any suggestions?