Unable to produce a list to show any matches in a list

user_16272183 0
Viewed 49

I am trying to make a column of a dataframe to matches (if any) to a list. A custom function called return hits has been created to that effect.

def returnhits(a_list, long_string):
    matches =[]
    for match in a_list:
        if any(word in long_string.split() for match in a_list):
            matches.append(match)
    return ' , '.join(matches)

qualification_list = ('Professional Certificate', 'NiTEC ', "Bachelor's Degree", 'Diploma', 'Advanced/Higher/Graduate Diploma', 'Post Graduate Diploma' , 'Professional Degree', "Master's Degree" , 'Doctorate (PhD)')

However I am unable to produce the desired results.

df['Qualifications'] = df['Other information'].apply(lambda x : returnhits(qualification_list, x))

Ideally if there is a match in the text it will return say NiTEC ,Professional Degree

pandas apply dataframe lambda python
2 Answers

Don't use a loop for that, use pandas regex methods:

import re

df = pd.DataFrame({'Other information': ['something', ' blah blah NiTEC', 'other diploma']})
qualification_list = ('Professional Certificate', 'NiTEC', "Bachelor's Degree", 'Diploma', 'Advanced/Higher/Graduate Diploma', 'Post Graduate Diploma' , 'Professional Degree', "Master's Degree" , 'Doctorate (PhD)')

df['Qualifications'] = df['Other information'].str.extract('(%s)' % '|'.join(re.escape(s) for s in qualification_list), flags=re.IGNORECASE)
df

output:

  Other information Qualifications
0         something            NaN
1   blah blah NiTEC          NiTEC
2     other diploma        diploma
user_163434640

You can try this for checking and return multi match:


df = pd.DataFrame({'Other information': ['something', ' Diploma blah NiTEC', 'other Diploma']})
qualification_list = ('Professional Certificate', 'NiTEC', "Bachelor's Degree", 'Diploma', 'Advanced/Higher/Graduate Diploma', 'Post Graduate Diploma' , 'Professional Degree', "Master's Degree" , 'Doctorate (PhD)')

def returnhits(a_list, x):
    return(' , '.join(a for a in a_list if a in x))

df['matches'] = df['Other information'].apply(lambda x : returnhits(qualification_list,x))

print(df)

Output:

     Other information          matches
0            something                 
1   Diploma blah NiTEC  NiTEC , Diploma
2        other Diploma          Diploma
user_17405770
Related