Loading string representation of dictionaries with single quotes in their values as dictionary literals

user_8675314 0
Viewed 270

I'm trying to parse some logs that among other things display some python dictionaries and extract the values from them.

An example input would be something like {'name' : 'D'Artagnan'} ast lib's literal_eval() gives an invalid syntax error while json.loads() requires all quotes to be double quotes, but if I get to replace all quotes with double quotes it will become {"name" : "D"Artagnan"} which is invalid as the value of name is now D and it then expects a comma delimiter to take the next key-value pair.

The types of names are not always formatted in the same way (for example <single_quote> like in D'Artagnan but they sometimes have space or nothing between or after them (for example "'t Something").

I looked around here but while others have been having similar issues, I didn't find something that can help me figure this out. The only workaround that I found so far was to handle them on a case-by-case base, writing a try-except clause that would handle JSONDecodeErrors by writing the offensive line down in a log file and then reading the file(as a human) and creating if-elif cases of the type

if "D'Artagnan" in line:
    line = line.replace("'","\'")

which solves the issue but is tedious, creates unnecessary overhead and skips values until I fix it which I feel is very wrong. Any help would be welcome!

Thanks!

python-3.x dictionary python json
1 Answers

In the event that you need to escape those ' characters, I think the re module might help.

import ast
import re

data = "{'name' : 'D'Artagnan'}"
data = re.sub(r"(\w)'(\w)", r"\1\\'\2", data)
print(ast.literal_eval(data))

Should give you a dictionary:

{'name': "D'Artagnan"}
user_2186630
Related