Update Multiple Items in DynamoDB without exceeding Maximum Throughput

dev1398 0
Viewed 83

I'm trying to perform update_item to my DynamoDB for each entry in a list. In the list the columns are id and total_sales where the total sales updates every hour. The idea is to parse the list and update each item (associated with the id in the list) and update the total_sales value but there are more than 5000 entries in the list and it exceeded my maximum throughput. Is there a more efficient way than simply doing this


def write_to_dynamo(id, total_sales):
    response = table.update_item(
            Key={
                'id': id
            },
            UpdateExpression="set stats.revenue = :r",
            ExpressionAttributeValues={
                ':r': total_sales
            },
            ReturnValues="UPDATED_NEW"
        )
        return response

def main():
    for item in lst:
        write_to_dynamo(item.id, item.total_sales)
throughput amazon-dynamodb-index boto3 amazon-dynamodb python
1 Answers

In DynamoDB itself - no.

If you want to use UpdateItem, you need to make individual API calls.

If you want to replace an item, there is BatchWriteItem, which bundles requests, but even that consumes the same throughput.

You could write your changes to an SQS queue and then have a Lambda Function respond to that Queue and update items. This setup takes care of the retries when you exceed the throughput and can make your setup more resilient, but it will still consume the same throughput.

userBidr0
Related