Hello pandas users/experts,
I am trying to filter a dataframe for rows where a particular column does not contain matches to a list of strings. I was able to accomplish this partially by using the following code:
searchfor = ['VIP', 'Employee', 'Replacement', 'Complimentary', 'Complementary']
df = df[~df['Discount Code'].str.contains('|'.join(searchfor)]
However, I then noticed that in addition to removing all rows containing those strings, all rows that were null for df['Discount Code'] were removed as well. I looked through the pandas documentation for str.contains() and randomly stumbled across a solution by setting the parameter na=False, but I have no idea why this works based on the description (or why my original code removed all rows with null values even though they don't contain the search string.) Can someone please explain this behavior to me? Thanks in advance.
Working code:
df = df[~df['Discount Code'].str.contains('|'.join(searchfor), case=False, na=False)]
Sample Data:
id discount_code
83 C-72772 <NA>
89 C-72767 <NA>
90 C-72766 <NA>
93 C-72763 Employee Order
98 C-72759 VIP Order