Converting a string that looks like a list of dicts into an actual list

Viewed 41

I have a columns in a dataframe that look like list of dictionaries, but they are of type string. However, they are not in JSON format (properties are not enclosed in double quotes)

df display:

col
[{name=john, userid=john-doe, age=23}, {name=mary, userid=mary-smith, age=21}]

What I want to do is iterate over the dictionaries in the list in a for loop. However I am having trouble converting the string into an actual Python list.

print(df['col'].dtype)
>>> object
for info in df['col']:
   print(info)
   print(type(info))
>>> [{name=john, userid=john-doe, age=23}, {name=mary, userid=mary-smith, age=21}]
>>> <class 'str'>

If I use json.loads, I get an error:

for info in df['col']:
   for i in json.loads(info):
   ...do work
>>> JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 3 (char 2)

I also tried ast.literal_eval:

import ast
for info in df['col']:
   for i in ast.literal_eval(info):
   ...do work
>>> SyntaxError: invalid syntax

Simply casting it to a list only makes it return the '[' part of the string:

for info in df['col']:
   for i in list(info):
     print(i)
>>> [
1 Answers

Use custom function:

f = lambda x: [dict([z.split('=') for z in y.split(', ')]) for y in x.strip('[{}] ').split('}, {')]
df['new'] = df['col'].apply(f)

List comprehenesion alternative:

df['new'] = [[dict([z.split('=') for z in y.split(', ')]) for y in x.strip('[{}] ').split('}, {')] for x in df['col']]
Related