Split strings in a spark dataframe with multiple delimiters

Viewed 343

I'm trying to split strings in a pyspark dataframe column with names and titles separated by different delimiters, in different formats. Eg:

+---+------------------------------------------------------------------------------------+
| id|                                                                       Text         |                                                                         
|  0| first name last name (title), first name last name (title), and first name (title) |                                                                    |
|  1| title: first name last name title: first name last name                            |                       
|  2| first name last name: title. first name last name: title.                          |   

              

How can I get this into a column of just the names?

Desired output

| id|          Text         |                                                                         
|  0| first name last name  |                                                                    |
|  1| first name last name  |                       
|  2| first name last name  | 

I tried the following but it didn't work

test = table.withColumn('column_new', f.split(f.regexp_replace('column', '[(:;)]', r"$1,"), ","))

Thanks in advance.

0 Answers
Related