Python replace ',' within quotes, but keep other commas, then remove the quotes in a string

user_8723105 0
Viewed 1982

I currently have an string output that looks like this:

Shares Mil,"1,457","1,388","1,341","1,287","1,214","1,155","1,103","1,010",983,959,949

and I need it to look like this:

Shares Mil,1457,1388,1341,1287,1214,1155,1103,1010,983,959,949

Basically I want to remove the commas in the numbers that are surrounded by quotes and then be able to separate the strings by the commas using .split(',') into a list.

I had an idea to find the commas within the quotes using regular expression to remove the commas and then removing the quotes with .replace('"','') but I'm not sure how to.

I could .split() by the commas and remove the quotes and then manually join the separated numbers but there has to be a more efficient way of doing this, so I thought I'd ask for help.

Thanks!

python-3.x
3 Answers

There is not an easy way to distinguish the commas you want to keep, vs the ones you want to discard, using the usual python string functions.

You'll need to use regular expressions, aka regex, to remove *only the ,'s that are within quoted numbers.

Here is an online regex compiler, and another to debug your regex expressions.

And here is python's re.sub() function, which is used to perform a Search and Replace operation on regex matches.

To do this, you'll also need to import re at the top of your file. You won't need to download or install anything in order to do this, as it's part of Python.

import re

input_str = 'Shares Mil,"1,457","1,388","1,341","1,287","1,214","1,155","1,103","1,010",983,959,949'
desired_output_str = 'Shares Mil,1457,1388,1341,1287,1214,1155,1103,1010,983,959,949'

# use regex to *only* remove commas that are part of a number within a string
# matches commas that are followed immediately by a one or more digits and then double-quote
regex_pattern = r',(?=\d+")'

# we want to replace *those* commas with empty string (aka delete them)
replacement = ''

comma_less_numbers_str = re.sub(regex_pattern, replacement, input_str)
print(comma_less_numbers_str, "\n")

# now it's easy: just remove the "
output_str = comma_less_numbers_str.replace('"', '')

print(output_str)
print(output_str == desired_output_str)
print(desired_output_str)

repl.it for this solution

Regex is Extremely powerful, and comes in handy more often than you might think. It simplifies many tasks you'll encounter.
I highly recommend spending a day to familiarize yourself with it's off putting nomenclature.
It's actually quite easy, once you start looking at it. I linked documentation above.
You can also download an app to your phone to quickly learn regex in bite size chunks in your spare time.

user_54118170

I don't think regex is necessary here. If you split the string on a double-quote character, you get a list where the odd numbered elements were inside the quotes.

So you can just replace commas in the odd numbered parts, and join it all back together.

s = 'Shares Mil,"1,457","1,388","1,341","1,287","1,214","1,155","1,103","1,010",983,959,949'
''.join([(ss if i%2==0 else ss.replace(',', '' )) 
          for i, ss in enumerate(s.split('"'))])
user_28046120

There's another, simpler approach than using regular expressions. Python has a CSV file parser that accounts for quoted fields. Here's an example of what could work for you, with file "foo.csv" containing the example string that you want to parse:

import csv

with open("foo.csv", "r", newline="") as f:
    reader = csv.reader(f, quotechar='"') # quotechar is the "magic" here

    for row in reader:
        print(",".join(x.replace(",", "") for x in row))

Note the quotechar argument in csv.reader. That's what allows Python to recognize a double-quote character as something that quotes CSV fields.

user_16439730
Related