IAM permissions to create a new Redshift cluster from another cluster's snapshot

user_2126910 0
Viewed 455

I want to create a user that can, in order:

  1. Create a Redshift snapshot on <old-cluster>
  2. Create a new Redshift cluster from this snapshot on a <new-cluster>
  3. Be able to resume / pause the <new-cluster>
  4. Delete the <new-cluster>

For the user I've created, I've created a new policy and listed the following IAM permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "redshift:RestoreFromClusterSnapshot",
                "redshift:DeleteCluster",
                "redshift:CreateCluster",
                "redshift:PauseCluster",
                "redshift:ResumeCluster"
            ],
            "Resource": [
                "arn:aws:redshift:<region>:<account>:snapshot:*/*",
                "arn:aws:redshift:<region>:<account>:cluster:<new-cluster>"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "redshift:DescribeClusters",
                "redshift:ExecuteQuery"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "redshift:CreateClusterSnapshot",
            "Resource": "arn:aws:redshift:<region>:<account>:snapshot:<old-cluster>/*"
        }
    ]
}

This allows the user to create the snapshot fine. However, when I try to create a new cluster from the snapshot using the CLI, I get an (UnauthorizedOperation) error.

Command (with set $WAREHOUSE_NAME and $SNAPSHOT_IDENTIFIER, <user> refers to the user I've created):

aws redshift restore-from-cluster-snapshot \
    --cluster-identifier $WAREHOUSE_NAME \
    --snapshot-identifier $SNAPSHOT_IDENTIFIER \
    --port 5439 \
    --availability-zone <region> \
    --cluster-subnet-group-name <subnet-group> \
    --no-publicly-accessible \
    --cluster-parameter-group <param-group> \
    --vpc-security-group-ids <security-group> \
    --automated-snapshot-retention-period 1 \
    --manual-snapshot-retention-period 1 \
    --number-of-nodes 2 \
    --aqua-configuration-status disabled \
    --no-availability-zone-relocation \
    --profile <user>

I get the following error:

An error occurred (UnauthorizedOperation) when calling the RestoreFromClusterSnapshot operation: Access Denied. Please ensure that your IAM Permissions allow this operation.

Anyone come across this before?

Update

I found this post about Redshift permissions that included a bunch of required EC2 permissions. I have now updated the permissions of the aforementioned policy to the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "redshift:RestoreFromClusterSnapshot",
                "redshift:DeleteCluster",
                "redshift:CopyClusterSnapshot",
                "redshift:CreateCluster",
                "redshift:AuthorizeSnapshotAccess",
                "redshift:PauseCluster",
                "redshift:RevokeSnapshotAccess",
                "redshift:ResumeCluster"
            ],
            "Resource": [
                "arn:aws:redshift:<region>:<account>:cluster:<new-cluster>",
                "arn:aws:redshift:<region>:<account>:snapshot:*/<new-cluster>"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInternetGateways",
                "ec2:DescribeAddresses",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeVpcs",
                "redshift:DescribeClusterSnapshots",
                "redshift:DescribeClusters",
                "ec2:DescribeAccountAttributes",
                "redshift:DescribeClusterParameterGroups",
                "redshift:ExecuteQuery",
                "ec2:DescribeSubnets",
                "ec2:DescribeSecurityGroups",
                "redshift:DescribeClusterSubnetGroups"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "redshift:CreateClusterSnapshot",
            "Resource": "arn:aws:redshift:<region>:<account>:snapshot:<old-cluster>/*"
        }
    ]
}

I'm now running into the following error code when I try the same command as before:

An error occurred (InvalidParameterValue) when calling the RestoreFromClusterSnapshot operation: Unable to restore cluster. The key 'arn:aws:kms:<region>:<account>:key/<key-id>' is inaccessible.

That key ID refers to the original KMS key for the <old-cluster> encryption.

I think it has something to do with --kms-key-id that's a parameter for the restore-from-cluster-snapshot CLI command?

amazon-redshift amazon-iam amazon-web-services
1 Answers

Managed to solve this myself.

I was missing two key bits of configuration:

  1. The EC2 permissions in the IAM policy for the user I created
  2. Adding the user to the KMS key used by <old-cluster> for encryption

Solving 1. was done by adding the the EC2 permissions to the policy I had created. The final permissions JSON looked like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "redshift:RestoreFromClusterSnapshot",
                "redshift:DeleteCluster",
                "kms:GetPublicKey",
                "redshift:CopyClusterSnapshot",
                "redshift:CreateCluster",
                "redshift:AuthorizeSnapshotAccess",
                "redshift:PauseCluster",
                "redshift:RevokeSnapshotAccess",
                "redshift:ResumeCluster"
            ],
            "Resource": [
                "arn:aws:redshift:<region>:<account>:cluster:<new-cluster>",
                "arn:aws:redshift:<region>:<account>:snapshot:*/<new-cluster>",
                "arn:aws:kms:<region>:<account>:key/<key-id>"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInternetGateways",
                "ec2:DescribeAddresses",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeVpcs",
                "redshift:DescribeClusterSnapshots",
                "redshift:DescribeClusters",
                "ec2:DescribeAccountAttributes",
                "redshift:DescribeClusterParameterGroups",
                "redshift:ExecuteQuery",
                "ec2:DescribeSubnets",
                "ec2:DescribeSecurityGroups",
                "redshift:DescribeClusterSubnetGroups"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "redshift:CreateClusterSnapshot",
            "Resource": "arn:aws:redshift:<region>:<account>:snapshot:<old-cluster>/*"
        }
    ]
}

Solving 2. was done by adding the user I had created to the KMS key used to encrypt the <old-cluster>. The KMS key permissions file now looks like this (where <user> is the user I created):

{
    "Version": "2012-10-17",
    "Id": "redshift-default-key-1",
    "Statement": [
        {
            "Sid": "Allow administration of the key",
            "Effect": "Allow",
            "Principal": {
                "AWS": [
                    "arn:aws:iam::<account>:user/<user>",
                    "arn:aws:iam::<account>:root"
                ]
            },
            "Action": [
                "kms:Create*",
                "kms:Describe*",
                "kms:Enable*",
                "kms:List*",
                "kms:Put*",
                "kms:Update*",
                "kms:Revoke*",
                "kms:Disable*",
                "kms:Get*",
                "kms:Delete*",
                "kms:ScheduleKeyDeletion",
                "kms:CancelKeyDeletion",
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "*"
        }
    ]
}

And finally, the command I used to restore from cluster snapshot was as follows:

aws redshift restore-from-cluster-snapshot \
                --cluster-identifier $WAREHOUSE_NAME \
                --snapshot-identifier $SNAPSHOT_IDENTIFIER \
                --snapshot-cluster-identifier <old-cluster> \
                --port 5439 \
                --availability-zone <region> \
                --cluster-subnet-group-name <subnet-group> \
                --no-publicly-accessible \
                --cluster-parameter-group <param-group> \
                --vpc-security-group-ids <security-group> \
                --automated-snapshot-retention-period 1 \
                --manual-snapshot-retention-period 1 \
                --number-of-nodes 2 \
                --aqua-configuration-status disabled \
                --no-availability-zone-relocation \
                --profile <user>

And it works! Hope this helps if you're coming across a similar issue :)

user_21269100
Related