Regular expression vectors

user_4323425 0
Viewed 64

I am trying to load a file using python (pandas) and regular expressions. I found a workaround to solve my issue but I still would like to find a one line solution, it may be useful for the future. The file format is the following:

1   (a b   c)  (a  b c)   (a b c)
2 (a b c) (a b c)    (a b c)
3       a b   c  a  b c   a b c

Notice that some are spaces and some are tabs and the number of tab and/or spaces is not constant. The expected result is the following dataFrame:

    0   1   2   3   4   5   6   7   8   9
0   1   a   b   c   a   b   c   a   b   c
1   2   a   b   c   a   b   c   a   b   c
2   3   a   b   c   a   b   c   a   b   c

My best result was obtained using:

pd.read_csv('filepath', delimiter='\)*\s+\(*', engine='python', header = None)

Which gives:

    0   1   2   3   4   5   6   7   8   9
0   1   a   b   c   a   b   c   a   b   c)
1   2   a   b   c   a   b   c   a   b   c)
2   3   a   b   c   a   b   c   a   b   c

And then i get rid of the ")" with additional processing. I still don't understand why \s+ is enough to remove the tabs i.e. \t (if somebody can answer this question as well, it is definitely a bonus).

I have tried different things but I couldn't find a single regex that does the job. Some of my attempts:

\)*\s*\(*
(\)*\s+\(*)|(\))
(\)*\s+\(*)|(\)\n)

Some are clear why they do not work, but I expected the last one to be the right solution. Another idea would be to use \)*\s*\(* but forcing at least one character to be in the regex, but I did not find if it is possible.

Any idea what would be a better regex?

P.S. If I read the file line by line and I use re.split with regex the splitting looks different, but I would prefer using pandas.

pandas python regex
2 Answers
>>> pd.DataFrame([re.sub(r'[\(\)]', '', line).split()
                      for line in open('data.txt').readlines()])

   0  1  2  3  4  5  6  7  8  9
0  1  a  b  c  a  b  c  a  b  c
1  2  a  b  c  a  b  c  a  b  c
2  3  a  b  c  a  b  c  a  b  c
user_152399510

You can use '\s*\(|\)\s*\(|\)|\s+' as delimiter.

  • \s*\( will match an opening parenthesis preceded by zero or any number of whitespace characters
  • \)\s*\( will match a closing parenthesis followed by zero or any number of whitespace characters followed by a closing parenthesis
  • \) will match a closing parenthesis
  • \s+ will match one or more white space character

Make sure to pass engine as python to avoid warning for using regex as separator, you'll get one NaN column in the end which you can just drop:

import pandas as pd
from io import StringIO


csv_data = '''1   (a b   c)  (a  b c)   (a b c)
2 (a b c) (a b c)    (a b c)
3       a b   c  a  b c   a b c'''

df = pd.read_csv(StringIO(csv_data),
                delimiter='\s*\(|\)\s*\(|\)|\s+', 
                engine='python', 
                header=None).dropna(axis=1)

OUTPUT:

   0  1  2  3  4  5  6  7  8  9
0  1  a  b  c  a  b  c  a  b  c
1  2  a  b  c  a  b  c  a  b  c
2  3  a  b  c  a  b  c  a  b  c
user_91363480
Related