AWS BotoCore Error - An AttributeValue may not contain an empty string

user_8326382 0
Viewed 8652

I'm trying to fill a DynamoDB database with data from an old event store made out of a PostgreSQL database. After it ran through a good portion of the db entries, this error was thrown when attempting to call the put_item function.

botocore.exceptions.ClientError:-

An error occurred (ValidationException) when calling the PutItem operation: One or more parameter values were invalid: An AttributeValue may not contain an empty string

I decided to rerun the code and see what was happening by dumping out all of the table attributes right before it was inserted.

I can see the only "empty string" is in the answer_string attribute of a dictionary, called details, see below:-

Importing event type 5 completed by user: 1933
1933 5 {'answer': {'difficulty': 1, 'answer_string': ''}, 'card_id': n 
'13448', 'review_id': '153339', 'time_spent': 2431}
62 153339
2017-01-18 00:46:48.373009+00:00 2017-01-18 00:46:48.364217+00:00

I'm pretty certain this is what's causing the error to be thrown, as none of the other table attributes are incorrect.

My problem is the details dictionary can come from dozens of different locations and each details dictionary can have different attributes - the one with the answer_string attribute is just one of many possible dictionary configurations. I can't possibly check for all possible configurations of a dictionary and verify that they all don't have empty strings.

Is there a way I can do a one time overall check on a dictionary and see if any one part of it is empty?

boto amazon-dynamodb amazon-s3 dictionary python
4 Answers

Or, if you want replace all empty strings with None values:

def removeEmptyString(dic):
    for e in dic:
        if isinstance(dic[e], dict):
            dic[e] = removeEmptyString(dic[e])
        if (isinstance(dic[e], str) and dic[e] == ""):
            dic[e] = None
        if isinstance(dic[e], list):
            for entry in dic[e]:
                removeEmptyString(entry)
    return dic

dictionaryWithEmptyStringsReplacedWithNone = removeEmptyString(dicrionaryWithEmptyStrings)

It is far from perfect but it works.

user_88865290

@PedoDorf's function worked for me though I had to add a check since sometimes it'd return "TypeError: string indices must be integers" when receiving a string

def removeEmptyString(dic):
  if isinstance(dic, str):
    if dic == "":
      return None
    else:
      return dic

  for e in dic:
    if isinstance(dic[e], dict):
      dic[e] = removeEmptyString(dic[e])
    if (isinstance(dic[e], str) and dic[e] == ""):
      dic[e] = None
    if isinstance(dic[e], list):
      for entry in dic[e]:
        removeEmptyString(entry)
  return dic

Thanks

user_124580000

If you need to account for nested objects, and clean them as well, give this a try. Requires some recursion:

def clean_ddb_data(obj):
    cleaned = {}
    for k, v in obj.items():
        if isinstance(v, dict):
            cleaned[k] = clean_ddb_data(v)
        elif isinstance(v, str):
            if len(v) > 0:
                cleaned[k]=v
        else:
            cleaned[k]=v
    return cleaned
user_30058460
Related