insert quotation marks around all words in string in python

user_15924357 0
Viewed 564

I have a string in a column that looks likes this

x = pd.DataFrame(data={'a': 
                       [({"Name":"any","all":[{"First":"True","Second":False},{"Last":True,"Second":False}],"Entry":0})]}).applymap(str)

some random fields are not wrapped in quotation marks, but they should be. I tried stripping all quotes and then reinserting quotes around all words within the string with the following, but its not quite working correctly - as shown below

x['a'][0].strip(' \" ')

' '.join('"{}"'.format(w) for w in x['a'][0].split(' '))

which gives the following output

"{'Name':" "'any'," "'all':" "[{'First':" "'True'," "'Second':" "False}," "{'Last':" "True," "'Second':" "False}]," "'Entry':" "0}"

the expected output would be this

{"Name":"any","all":[{"First":"True","Second":"False"},{"Last":"True","Second":"False"}],"Entry":"0"}

any advice would be great. thanks so much!

python
2 Answers

Trying to insert quotation marks via regular expressions has a number of disadvantages:

  1. It disregards the fact that the given string is machine-readable
  2. It will fail (or at least produce strange results) as soon as any key or value contains something that's not matched by \w, e.g., "First-one" instead of "First"
  3. Using regular expressions is generally not very fast

If these concerns are irrelevant for your task, you can certainly do it that way – but it's quite hacky and not guaranteed to work all the time, so here's a cleaner approach.

The thing is that

s = """{"Name":"any","all":[{"First":"True","Second":False},{"Last":True,"Second":False}],"Entry":0}"""

is a structured string – from the looks of it, it's a stringified Python dict that can be turned back into a proper data structure with eval():

d = eval(s)

(I wonder where you got that string though? If it was a Python dict in the first place, turning it into a string first and trying to "fix" it by messing with it later on is generally not a good idea.)

The values that miss quotation marks are not "random" values, but a bool and an int, respectively, i.e., they are missing quotation marks because they aren't strings. However, they can be stringified by calling str() on each of them individually. So, a clean way to turn those values into strings is something like the following, which might look a bit different depending on how the complete data set is structured:

for i,elem in enumerate(d['all']):
    for k,v in elem.items():
        d['all'][i][k] = str(v)
        
d['Entry'] = str(d['Entry'])

Result:

{'Name': 'any', 'all': [{'First': 'True', 'Second': 'False'}, {'Last': 'True', 'Second': 'False'}], 'Entry': '0'}
user_158730430

This will do the trick:

import pandas as pd
import re
import json

x = pd.DataFrame(data={'a': 
                       [({"Name":"any","all":[{"First":"True","Second":False},{"Last":True,"Second":False}],"Entry":0})]})

replacer = re.compile("(\w+)")
x['a'] = replacer.sub(r'"\1"', json.dumps(x['a'][0]).replace('"', ''))

Explanation See the Python regex docs for more info...
(\w+) : () matches whatever is in the parentheses and denotes a capturing group. \w matches unicode word characters. + with one or more occurrences.

The sub method has the signature Pattern.sub(repl, string, count=0). This returns the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl.
"\1" is the first captured group in the regex expression with double quotation marks.

We need to use JSON since we want to serialise the now Pandas object into a JSON string. This leads to the expected behavior, instead of casting to a string in pandas which adds the ' you experienced.
The .replace('"', '') gets rid of the occurrences of double quotes such that the regex sub method doesn't add additional ones.

user_159817830
Related