Python removing nested unicode 'u' sign from string

user_4890123 0
Viewed 97

I have a unicode object which should represent a json but it contains the unicode u in it as part of the string value e.g. u'{u\'name\':u\'my_name\'}'

My goal is to be able to load this into a json object. Just using json.loads fails. I know this happens because of the u inside the string which are not part of an acceptable json format.

I, then, tired sanitizing the string using replace("u\'", "'"), encode('ascii', 'ignore') and other methods without success.

What finally worked was using ast.literal_eval but I'm worried about using it. I found a few sources online claiming its safe. But, I also found other sources claiming it's bad practice and one should avoid it.

Are there other methods I'm missing?

python-2.7 python
1 Answers

The unicode string is the result of unicode being called on a dictionary. 

>>> d = {u'name': u'myname'}
>>> u = unicode(d) 
>>> u  
u"{u'name': u'myname'}"

If you control the code that's doing this, the best fix is to change it to call json.dumps instead.

>>> json.dumps(d)
'{"name": "myname"}'

If you don't control the creation of this object, you'll need to use ast.literal_eval to create the dictionary, as the unicode string is not valid json. 

>>> json.loads(u)
Traceback (most recent call last):
...
ValueError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)


>>> ast.literal_eval(u)
{u'name': u'myname'}

The docs confirm that ast.literal_eval is safe:

can be used for safely evaluating strings containing Python values from untrusted sources

You could use eval instead, but as you don't control the creation of the object you cannot be certain that it has not been crafted by a malicious user, to cause damage to your system.

user_53209060
Related