How to check if a given Amazon S3 key exists from shell script

user_9777493 0
Viewed 719

I would like to check given s3 key exists from shell script.

S3 Path:
========
s3://Bucket/test/names/12345/names-ca/one.txt
s3://Bucket/test/names/12346/names-ec/two.txt

I would like to check s3://Bucket/test/names/12345 is exists or not.what could be better approach to check this one from shell.

amazon-web-services amazon-s3 shell
1 Answers

You can use the awscli to test for the presence of an object in S3. For example:

aws s3api head-object --bucket mybucket --key dogs/snoopy.png

You will get a JSON response and a zero returncode if the object exists (and you have permission to issue HeadObject against it).

Here's an example bash script:

#!/bin/bash

aws s3api head-object --bucket mybucket --key dogs/snoopy.png 1>/dev/null 2>&1

if [[ $? -eq 0 ]]; then
    echo "S3 object exists"
else
    echo "S3 object does not exist"
fi

To test if there are objects under a given S3 prefix such as dogs/, you can issue:

aws s3 ls mybucket/dogs/ --recursive | grep -v "\/$" | wc -l

The output will be the count of S3 objects, for example 0 or 2. You can capture this in the normal way, for example:

COUNT=$(aws s3 ls mybucket/dogs/ --recursive | grep -v "\/$" | wc -l | tr -d ' ')

if [[ $COUNT -eq "0" ]]; then
    echo "No objects under dogs/"
else
    echo "Some ($COUNT) objects under dogs/"
fi

If using an AWS credentials profile other than your default, append --profile myprofile to the awscli command.

user_2714150
Related