How to delete X% of entries from a DynamoDB Table?

mourningraptor3 0
Viewed 25

I want to remove 10% of entries from a DDB table every time a script is ran. So far, I have created a Python script using boto3 that will delete all items from a DDB table:

import boto3
import sys

src_region = sys.argv[1]
src_profile_name = sys.argv[2]
src_ddb_table = sys.argv[3] 

# Create source session.
src_session = boto3.session.Session(profile_name=src_profile_name)
dynamoclient = src_session.client('dynamodb', region_name=src_region)

dynamoresponse = dynamoclient.get_paginator('scan').paginate(
    TableName=src_ddb_table,
    Select='ALL_ATTRIBUTES',
    ReturnConsumedCapacity='NONE',
    ConsistentRead=True
)

for page in dynamoresponse:
    for item in page['Items']:
        dynamoclient.delete_item(
            Key={'testTableDest': item['testTableDest']}, 
            TableName=src_ddb_table)

How can I modify this script to allow the user to select a percentage of entries they want to delete?

Thank you for any help!

foreach boto3 amazon-dynamodb python
1 Answers

If you want to delete them at random and are ok with a non-exact percentage you can do this easily with the python random package.

import boto3
import sys
import random

src_region = sys.argv[1]
src_profile_name = sys.argv[2]
src_ddb_table = sys.argv[3] 
percent_delete = int(sys.argv[4]) # 20

# Create source session.
src_session = boto3.session.Session(profile_name=src_profile_name)
dynamoclient = src_session.client('dynamodb', region_name=src_region)

dynamoresponse = dynamoclient.get_paginator('scan').paginate(
    TableName=src_ddb_table,
    Select='ALL_ATTRIBUTES',
    ReturnConsumedCapacity='NONE',
    ConsistentRead=True
)

for page in dynamoresponse:
    for item in page['Items']:
        if random.random() * 100 < percent_delete:
            dynamoclient.delete_item(
                Key={'testTableDest': item['testTableDest']}, 
                TableName=src_ddb_table)

This isn't "perfectly random" but will suffice.

Coin Graham0
Related