Delimit by Parentheses with Conditional Statement to Separate Columns

Regex patterns of the news chunks

It may be more pythonic using (foo) as a delimiter, but capturing a whole news chunk still works fine ;)

Regex patterns of the news chunks are \(nytimes\)\s?".*?" and \(wsj\)\s?".*?", respectively.

In the nutshell,

pattern = re.compile(r'\(nytimes\)\s".*?"|\(wsj\)\s".*?"')

Capture news chunks

Using the pattern, we can extract the target chunks as a list.

import re
import pandas as pd

my_text = '(nytimes) "news on war" (wsj) "news on business" (nytimes) "more news"'

pattern = re.compile(r'\(nytimes\)\s".*?"|\(wsj\)\s".*?"')
chunks = pattern.findall(my_text)
chunks
>>>['(nytimes) "news on war"', '(wsj) "news on business"', '(nytimes) "more news"']

Collate the chunks

The rest is an usual data cleaning work.

list_nyt = []
list_wsj = []
for chunk in chunks:
    if chunk.startswith('(nytimes)'):
        list_nyt.append(chunk.replace('(nytimes) ', '').strip('"'))
    elif chunk.startswith('(wsj)'):
        list_wsj.append(chunk.replace('(wsj) ', '').strip('"'))

Populate a DataFrame

df = pd.DataFrame(columns=['Text', 'NYTimes', 'WSJ'])
df.append(
    {
        'Text': str(my_text),
        'NYTimes': str(list_nyt),
        'WSJ': str(list_wsj)
    }, ignore_index=True
)
df
>>> 

i Believe this code can be better modified. But it does the job without loops

texts= ['(nytimes) "news on war" (wsj) "news on business" (nytimes) "more news"']

data = pd.DataFrame(texts,columns=['text'])
#use extract all to identify selected texts
new= data['text'].str.extractall(r'((?:\(\w+\)."\w+.\w+ \w+?"))')

#extractall will create a multi index columns, and so you need to adjust it

new_1= new.droplevel(level=1)
val = new_1[0].explode()[0].str.split().str[0]
val2 = new_1[0].explode()[0].str.split().str[1:].apply(lambda x: ' '.join(x))

#Created the new DataFrame to better do some cleaning and grouping earlier results
cleaner  = pd.DataFrame()
cleaner['one']= val
cleaner['two']= val2

cleaner =cleaner.groupby('one').aggregate(lambda tdf: tdf.unique().tolist()).reset_index().T
#cleaner.columns= ['NyTimes','WSJ']

cleaner.reset_index(inplace=True)
cleaner.drop(0,axis=0,inplace=True)
cleaner.drop('index',axis=1,inplace=True)

#once the data is is clean to expectation, include it back to main data.
data[['NyTimes','WSJ']]= cleaner.values
data.head()

Series.str.extractall

extract the ocurrences of the regular expression pattern from the strings in the column Text, then group by organization and aggregate the news articles using list, finally unstack to reshape

s = df['Text'].str.extractall(r'\((.*?)\)\s*(.*?)\s*(?=\(|$)')
s = s.set_index(0, append=True).groupby(level=[0, 2])[1].agg(list).unstack()

df.join(s)