How can I use automation in AWS to replicate a github repo to an S3 bucket (quickstart-git2s3)?

Viewed 167

I'd like to try and automate an S3 bucket replication of a Github repo (for the sole reason that Cloudformation modules must reference templates in S3).

This quickstart I tried to use looked like it could do it, but it doesn't result in success for me, even though github reports success in pushing via the webhook for my repository.

https://aws-quickstart.github.io/quickstart-git2s3/

I configured these parameters.

I am not sure what to configure for allowed IP's, so I tested fully open.

AllowedIps  0.0.0.0/0   -
ApiSecret   ****    -
CustomDomainName    -   -
ExcludeGit  True    -
OutputBucketName    -   -
QSS3BucketName  aws-quickstart  -
QSS3BucketRegion    us-east-1   -
QSS3KeyPrefix   quickstart-git2s3/  -
ScmHostnameOverride -   -
SubnetIds   subnet-124j124  -
VPCCidrRange    172.31.0.0/16   -
VPCId   vpc-l1kj4lk2j1l2k4j

I tried manually executing the code build as well but got this error:

COMMAND_EXECUTION_ERROR: Error while executing command: python3 - << "EOF" from boto3 import client import os s3 = client('s3') kms = client('kms') enckey = s3.get_object(Bucket=os.getenv('KeyBucket'), Key=os.getenv('KeyObject'))['Body'].read() privkey = kms.decrypt(CiphertextBlob=enckey)['Plaintext'] with open('enc_key.pem', 'w') as f: print(privkey.decode("utf-8"), file=f) EOF . Reason: exit status 1

The github webhook page reports this response:

Headers
Content-Length: 0
Content-Type: application/json
Date: Thu, 24 Jun 2021 21:33:47 GMT
Via: 1.1 9b097dfab92228268a37145aac5629c1.cloudfront.net (CloudFront)
X-Amz-Apigw-Id: 1l4kkn14l14n=
X-Amz-Cf-Id: 1l43k135ln13lj1n3l1kn414==
X-Amz-Cf-Pop: IAD89-C1
X-Amzn-Requestid: 32kjh235-d470-1l412-bafa-l144l1
X-Amzn-Trace-Id: Root=1-60d4fa3b-73d7403073276ca306853b49;Sampled=0
X-Cache: Miss from cloudfront
Body
{}
2 Answers

From the following link:

https://aws-quickstart.github.io/quickstart-git2s3/

You can see the following excerpts I have included:

Allowed IP addresses (AllowedIps) 18.205.93.0/25,18.234.32.128/25,13.52.5.0/25 Comma-separated list of allowed IP CIDR blocks. The default addresses listed are BitBucket Cloud IP ranges.

As such, since you said you're using GitHub, I believe you should use this URL to determine the IP range:

https://api.github.com/meta

As that API will respond with JSON, you should search for the attribute hooks since I believe that it described using hooks.

Why don't you copy/checkout the file you want before you run your cloudformation commands, no reason to get too fancy.

git checkout -- path/to/some/file
aws cloudformation ...

Otherwise why not fork the repo and add your Cloudformation stuff, and then it's all there. You could also delete everything you didn't need and merge/pull changes in the future. That way your deploys will be reproducible, and you can rollback easily from one commit to the other.

Related