Determine if an object exists in a S3 bucket based on wildcard

user_166291 0
Viewed 56785

Can someone please show me how to determine if a certain file/object exists in a S3 bucket and display a message if it exists or if it does not exist.

Basically I want it to:

1) Check a bucket on my S3 account such as testbucket

2) Inside of that bucket, look to see if there is a file with the prefix test_ (test_file.txt or test_data.txt).

3) If that file exists, then display a MessageBox (or Console message) that the file exists, or that the file does not exist.

Can someone please show me how to do this?

amazon-s3 c#
13 Answers

Not sure if this applies to .NET Framework, but the .NET Core version of AWS SDK (v3) only supports async requests, so I had to use a slightly different solution:

/// <summary>
/// Determines whether a file exists within the specified bucket
/// </summary>
/// <param name="bucket">The name of the bucket to search</param>
/// <param name="filePrefix">Match files that begin with this prefix</param>
/// <returns>True if the file exists</returns>
public async Task<bool> FileExists(string bucket, string filePrefix)
{
    // Set this to your S3 region (of course)
    var region = Amazon.RegionEndpoint.USEast1;

    using (var client = new AmazonS3Client(region))
    {
        var request = new ListObjectsRequest {
            BucketName = bucket,
            Prefix = filePrefix,
            MaxKeys = 1
        };

        var response = await client.ListObjectsAsync(request, CancellationToken.None);

        return response.S3Objects.Any();
    }
}

And, if you want to search a folder:

/// <summary>
/// Determines whether a file exists within the specified folder
/// </summary>
/// <param name="bucket">The name of the bucket to search</param>
/// <param name="folder">The name of the folder to search</param>
/// <param name="filePrefix">Match files that begin with this prefix</param>
/// <returns>True if the file exists</returns>
public async Task<bool> FileExists(string bucket, string folder, string filePrefix)
{
    return await FileExists(bucket, $"{folder}/{filePrefix}");
}

Usage:

var testExists = await FileExists("testBucket", "test_");
// or...
var testExistsInFolder = await FileExists("testBucket", "testFolder/testSubFolder", "test_");
user_10507840

I know this question is a few years old but the new SDK nowdays handles this in an easier manner.

  public async Task<bool> ObjectExistsAsync(string prefix)
  {
     var response = await _amazonS3.GetAllObjectKeysAsync(_awsS3Configuration.BucketName, prefix, null);
     return response.Count > 0;
  }

Where _amazonS3 is your IAmazonS3 instance and _awsS3Configuration.BucketName is your bucket name.

You can use your complete key as a prefix.

user_22048770

try this one:

    NameValueCollection appConfig = ConfigurationManager.AppSettings;

        AmazonS3 s3Client = AWSClientFactory.CreateAmazonS3Client(
                appConfig["AWSAccessKey"],
                appConfig["AWSSecretKey"],
                Amazon.RegionEndpoint.USEast1
                );

S3DirectoryInfo source = new S3DirectoryInfo(s3Client, "BUCKET_NAME", "Key");
if(source.Exist)
{
   //do ur stuff
}
user_50997550 
using Amazon;
using Amazon.S3;
using Amazon.S3.IO;
using Amazon.S3.Model;

string accessKey = "xxxxx";
string secretKey = "xxxxx";
string regionEndpoint = "EU-WEST-1";
string bucketName = "Bucket1";
string filePath = "https://Bucket1/users/delivery/file.json"

public bool FileExistsOnS3(string filePath)
{
   try
   {
      Uri myUri = new Uri(filePath);
      string absolutePath = myUri.AbsolutePath; // /users/delivery/file.json
      string key = absolutePath.Substring(1); // users/delivery/file.json
      using(var client = AWSClientFactory.CreateAmazonS3Client(accessKey, secretKey, regionEndpoint))
      {
         S3FileInfo file = new S3FileInfo(client, bucketName, key);
         if (file.Exists)
         {
            return true;
            // custom logic
         }
         else
         {
            return false;
            // custom logic
         }
      }
   }
   catch(AmazonS3Exception ex)
   {
      return false;
   }
}
user_97583880

There is an overload for GetFileSystemInfos Notice this line has filename.*

var files= s3DirectoryInfo.GetFileSystemInfos("filename.*");

public bool Check()
{
    var awsCredentials = new Amazon.Runtime.BasicAWSCredentials("AccessKey", "SecretKey");

      using (var client = new AmazonS3Client(awsCredentials, Amazon.RegionEndpoint.USEast1))
       {
       S3DirectoryInfo s3DirectoryInfo = new S3DirectoryInfo(client, bucketName, "YourFilePath");
                var files= s3DirectoryInfo.GetFileSystemInfos("filename.*");
                if(files.Any())
                {
                    //fles exists
                }
            }
        }
user_22603450

my 2 cents

    public async Task<bool> DoesKeyExistsAsync(string key)
    {
        ListObjectsResponse response = null;
        try
        {
            response = await _s3Client.ListObjectsAsync(new ListObjectsRequest { BucketName = _settings.BucketName, Prefix = key });
        }
        catch (Exception ex)
        {
            _logger.LogError($"Error while checking key {key}");
            return false;
        }
        return (response?.S3Objects?.Count > 0);
    }
user_26192440
Related