I'm having an issue with AWS CLI where if I do an s3 cp or s3 sync with smaller files (<10GB) it works fine but when I try same command with files (>10gb) it returns an exit status of 1 which means some files failed to copy.
Is there a way to show/list the files that failed to copy/sync? Note: All copies are between s3 to s3 ( same bucket, same region, same ACL, just different "folders paths")
Looking at the official docs, when work with files more than 10gb you need to specify in CLI commands an argument --expected-size or the upload may fail when it reaches the default part limit of 10,000.
Example for a file larger than 50GB:
aws s3 cp - s3://mybucket/stream.txt --expected-size 54760833024
I hope that works for you. Keep rocking!.
Faced a different problem that led me to the same question: is there a way to see which uploads failed?
Per AWS docs, you can use the --only-show-errors flag to get only failing uploads in the output of the command. Works for both s3 cp and s3 sync.
