Does anyone know if there is a limit to the number of objects I can put in an S3 bucket? can I put a million, 10 million etc.. all in a single bucket?
7 Answers
There are no limits to the number of objects you can store in your S3 bucket. AWS claims it to have unlimited storage. However, there are some limitations -
- By default, customers can provision up to 100 buckets per AWS account. However, you can increase your Amazon S3 bucket limit by visiting AWS Service Limits.
- An object can be 0 bytes to 5TB.
- The largest object that can be uploaded in a single PUT is 5 gigabytes
- For objects larger than 100 megabytes, customers should consider using the Multipart Upload capability.
That being said if you really have a lot of objects to be stored in S3 bucket consider randomizing your object name prefix to improve performance.
When your workload is a mix of request types, introduce some randomness to key names by adding a hash string as a prefix to the key name. By introducing randomness to your key names the I/O load will be distributed across multiple index partitions. For example, you can compute an MD5 hash of the character sequence that you plan to assign as the key and add 3 or 4 characters from the hash as a prefix to the key name.
More details - https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-performance-improve/
-- As of June 2018
Related