Create a new column based on values in an existing column

Viewed 82

I would like to create a new column to classify if certain address is a residential or non-residential address.

Below is a column in the original dataframe:

Building_name
Fung Chak House, Choi Wan (II) Estate
Princess Margaret Hospital (non-residential)

And I write the below code to create a new column if the string "non-residential" is in Building_name it would be classified as Non-residential, otherwise it would be Residential.

def build_cat(row):
    if "(non-residential)" not in district_df['Building_name']:
        return ("Residential")
    if '(non-residential)' in district_df['Building_name']:
        return ('Non_residential')

district_df['Building_category'] = district_df.apply(lambda row: build_cat(row), axis =1) 

However, the about function returns everything as Residential.

Building_name Building_cateory
Fung Chak House, Choi Wan (II) Estate Residential
Princess Margaret Hospital (non-residential) Residential

Appreciate if you can let me know what's wrong in my code, or if there are other more efficient ways to get the same result.

Thanks.

5 Answers

or use np.where as python if-else ternary Operators

cond = district_df['Building_name'].str.contains(r'\(non-residential\)')
district_df['Building_cateory'] = np.where(cond, 'Non_residential', 'Residential')

apply function to the target column Building_name.

def build_cat(x):
    if "(non-residential)" not in x:
        return "Residential"
    else:
        return 'Non_residential'
district_df['Building_category'] = district_df['Building_name'].map(build_cat)

I chose to add this community wiki, for study:

on my Jupyter Notebook, I compared the answers from @MayankPorwal and @Ferris. Here is the result:

First, the str.contains with np.where method:

import pandas as pd
import numpy as np

df = pd.DataFrame({'Building_name': ['Fung Chak House, Choi Wan (II) Estate', 'Princess Margaret Hospital (non-residential)', 'Fung Chak 1', 'Fung Chak 2 (non-residential)', 'Fung Chak 3']})

df = pd.concat([df] * 10000, ignore_index=True)

def prop():
    df['Building_cateory'] = np.where(df.Building_name.str.contains('non-residential'), 'Non-residential', 'residential')

Now, timing it:

%timeit prop()

The result:

81.1 ms ± 15.5 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

On the other hand, the map() approach:

import pandas as pd
import numpy as np

df = pd.DataFrame({'Building_name': ['Fung Chak House, Choi Wan (II) Estate', 'Princess Margaret Hospital (non-residential)', 'Fung Chak 1', 'Fung Chak 2 (non-residential)', 'Fung Chak 3']})

df = pd.concat([df] * 10000, ignore_index=True)

def prop():
    def build_cat(x):
        if "(non-residential)" not in x:
            return "Residential"
        else:
            return 'Non_residential'
    df['Building_category'] = df['Building_name'].map(build_cat)

Now, timing it:

%timeit prop()

The result:

15.7 ms ± 1.13 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)

Thus, we have a clear winner!

Try the following pandas trick to do it faster:


Edit

Nope, applying lambda is faster.

Timing using the str.contains approach:

2.34 ms ± 367 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

Timing using the apply-lambda approach:

1.08 ms ± 138 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Using the map() approach is fastest:

742 µs ± 112 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

Please note, the new column Building_category is created after #Line 1

import pandas as pd

df = pd.DataFrame({'Building_name': ['Fung Chak House, Choi Wan (II) Estate', 'Princess Margaret Hospital (non-residential)']})

# Line 1
df.loc[df[df['Building_name'].str.contains('(non-residential)')].index, 'Building_category'] = "Non_Residential"

# Line 2
df.loc[df[~df['Building_name'].str.contains('(non-residential)')].index, 'Building_category'] = "Residential"

print(df)

Output generated:

                                  Building_name Building_category
0         Fung Chak House, Choi Wan (II) Estate       Residential
1  Princess Margaret Hospital (non-residential)   Non_Residential

Use numpy.where:

In [1197]: import numpy as np

In [1198]: df['Building_cateory'] = np.where(df.Building_name.str.contains('non-residential'), 'Non-residential', 'residential')

In [1199]: df
Out[1199]: 
                                  Building_name  Building_cateory
0         Fung Chak House, Choi_Wan (II) Estate       residential
1  Princess Margaret Hospital (non-residential)   Non-residential

Not sure this is the best way to go. Yet, relating to your precise question of unexpected result you got, please change district_df to rowin your function. Like this. For a better approach, take a looks at other answer belows. Kr.

    def build_cat(row):
        if "(non-residential)" not in row['Building_name']:
            return ("Residential")
        if '(non-residential)' in row['Building_name']:
            return ('Non_residential')
    
    district_df['Building_category'] = district_df.apply(lambda row: build_cat(row), axis =1) 
    ```
Related