Remove numbering, newline, break tags from pandas dataframe columns

user_10763617 0
Viewed 322

I have a dataframe with columns containing strings with newline, break takgs and list numbering:

df['Side_Effects'][0]
'1.Nausea\n<br/>2.Vomiting\n<br/>3.Diarrhoea\n<br/>4.Anorexia\n<br/>5.Malaise\n<br/>6.Fever\n<br/>7.Pruritis\n<br/>8.Rash\n<br/>9.Headache\n<br/>10.Pharyngitis\n<br/>11.Cough\n<br/>'

First I have to remove the numberings, newline and br tags from all the strings of column. I tried:

df['Side_Effects'].replace(r'\\n',' ', regex=True, inplace=True)

and this:

df['Side_Effects'] = df['Side_Effects'].str.replace('</br>','')

but nothing seems to work... Would appreciate any help !!!

pandas python regex
2 Answers

Using Regex and str methods

Ex:

df = pd.DataFrame({'Col': ['1.Nausea\n<br/>2.Vomiting\n<br/>3.Diarrhoea\n<br/>4.Anorexia\n<br/>5.Malaise\n<br/>6.Fever\n<br/>7.Pruritis\n<br/>8.Rash\n<br/>9.Headache\n<br/>10.Pharyngitis\n<br/>11.Cough\n<br/>']})
df['New'] = df['Col'].str.replace('(<br/>|\d+\.)','').str.split().agg(" ".join)    #IF you need as list skip .agg(" ".join)
print(df)

Output:

             Col                                 New                                            
0  1.Nausea\n<br/>2.Vomiting\n<br/>3.Diarrhoea\n<...  Nausea Vomiting Diarrhoea Anorexia Malaise Fev...
user_5323120

You may use

df['Side_Effects'] = df['Side_Effects'].str.replace(r'(?m)^(?:<br/>)?\d+\.|<br/>', '').str.strip()

See regex demo

Details

  • (?m)^ - start of a line ((?m) is an inline variant of the re.M / re.MULTILINE flag)
  • (?:<br/>)? - an optional <br/> string
  • \d+\. - 1 or more digits and then a .
  • | - or
  • <br/> - just <br/> string.

The .str.strip() removes any trailing whitespace.

user_38329700
Related