I am working with the Amazon S3 API, specifically with ListObjects (v2), the GetBucket command. Frustratingly, I can only get back 1000 objects. I think that you specify a Delimeter or nextMarker is set to induce pagination and get back everything. Anyone have an example of how this is accomplished? The final result from curl or a GET is actually getting all the things back, not just 1000 objects.
1 Answers
If the list_objects() response has IsTruncated set to True, then you can make a subsequent call, passing NextContinuationToken from the previous response to the ContinuationToken field on the subsequent call. This will return the next 1000 objects.
Or, you can use the provided Paginators to do this for you.
From Paginators — Boto 3 documentation:
import boto3
# Create a client
client = boto3.client('s3', region_name='us-west-2')
# Create a reusable Paginator
paginator = client.get_paginator('list_objects')
# Create a PageIterator from the Paginator
page_iterator = paginator.paginate(Bucket='my-bucket')
for page in page_iterator:
print(page['Contents'])
Related