Get count of files in a folder in an s3 bucket from a python lambda function

Amazon S3 Storage Lens "provides a single view of object storage usage and activity across your entire Amazon S3 storage. It includes drilldown options to generate insights at the organization, account, Region, bucket, or even prefix level." However, the ability to obtain metrics at a Prefix level requires Advanced metrics, which is priced at $0.20 per million objects monitored per month. There is a boto3 library that gives access to Storage Lens, but that seems to be about configuration rather than retrieving the actual metrics. (I haven't used it, so I'm not sure what's involved.)

Alternatively, you could call list_objects_v2() for the desired Prefix. However, it only returns a maximum of 1000 objects, so you would need to keep calling it while NextContinuationToken is not null. Each call returns a KeyCount, which is the number of keys returned with the request.

Alternatively, if you use the resource-based call of bucket.objects.all(), then boto3 will perform the loop for you and will present back a list of s3.ObjectSummary objects. You can simply use len() on the list to obtain the count.

Both methods will be quite slow for buckets/folders with a large number of objects. Therefore, another option is to use Amazon S3 Inventory, which can provide a daily or weekly CSV file listing all objects. It might not be 'current', but it is a very easy way to count the objects without having to loop through calls.