How to change CodePipeline artifacts behavior? (If possible)

user_7438853 0
Viewed 663

I am designing a CI/CD system and looking for complete isolation between frontend and backend components. They will use separate CodeCommit Repositories, CodeBuild Projects, and deployment mechanisms. I've even created separate S3 buckets to house the artifacts. Furthermore, I'm using lengthy descriptive names for the pipelines, and related services, to ensure there's no confusion.

However, I am hitting an annoying issue where CodePipeline seems to be creating a Folder inside my artifacts S3 bucket with a truncated version of the pipeline name. I feel this is irrelevant and will only add to the confusion I'm trying to avoid. The entire S3 bucket is dedicated to those pipeline artifacts so I don't want a Folder containing everything within. I don't see anyway to stop CodePipeline from behaving this way.

Example:

Pipeline Name: my-clients-pipeline-for-frontend

S3 Artifacts: my-clients-pipeline-artifacts-for-frontend

----> my-clients-pipeline-f

--------> SourceArtifact

--------> BuildAritfact



Pipeline Name: my-clients-pipeline-for-backend

S3 Artifacts: my-clients-pipeline-artifacts-for-backend

----> my-clients-pipeline-f

--------> SourceArtifact

--------> BuildAritfact

The documentation states,

"Every time you use the console to create another pipeline in that Region, CodePipeline creates a folder for that pipeline in the bucket. It uses that folder to store artifacts for your pipeline as the automated release process runs."

Although, I am using CloudFormation to build the pipelines and not sure if this still applies. The pipelines are working... can I remove this truncated folder somehow?

https://docs.aws.amazon.com/codepipeline/latest/userguide/welcome-introducing-artifacts.html

aws-codepipeline amazon-web-services
1 Answers

This response is 19 months late so I hope you were able to find a solution in the meantime. ^-^

I generally try and use aws CLI for anything related to AWS Env modification when my pipelines are running. So for example in your case, if you have manually set your output artifacts to go in the buckets with appropriate names i.e: "my-clients-pipeline-artifacts-for-backend", then you can add a stage after deploy stage to call AWS Lambda that uses aws CLI to remove any folders created by AWS CodePipeline for example:

aws s3 rm s3://my-clients-pipeline-for-backend/my-clients-pipeline-f
--recursive

This command recursively deletes everything in the folder: "my-clients-pipeline-f" and then deletes the folder itself as well.

I know this isn't a super straightforward approach but it is a workaround/solution that works for me in most cases. I have not tested this to delete build artifact s3 folders but it should work fine nonetheless.

REFERENCES

Integrate AWS CLI in Lambda: https://bezdelev.com/hacking/aws-cli-inside-lambda-layer-aws-s3-sync/

Invoke Lambda in CodePipeline: https://docs.aws.amazon.com/codepipeline/latest/userguide/actions-invoke-lambda-function.html#actions-invoke-lambda-function-add-action

user_187607640
Related