How to filter a DataFrame by a function

user_16425417 0
Viewed 126

I want to filter a DataFrame using a condition which is passing a function I made. This function will return True or False.

But I get this exception:

TypeError: cannot convert the series to <class 'float'>

This is my code:

def filter():
    inicio = time.time()
    df = pd.read_csv('data_base.csv')
    nrows = df.shape[0]
    for i in range(nrows).__reversed__():
        coincidencias = df[
            (df['Fecha'] == df.loc[i]['Fecha']) &
            (df['Mercado'] == df.loc[i]['Mercado']) &
            (df['Local'].str.contains(main_word(df.loc[i]['Local']), case=False)) &
            (df['Visitante'].str.contains(main_word(df.loc[i]['Visitante']), case=False)) &
     ---->  (is_there_surebet(df['Contenido'], df.loc[i]['Contenido']))   <----
            ].dropna()

        if coincidencias.shape[0] > 1:
            print(coincidencias)


        df = df.drop([i])
    fin = time.time()
    print(fin-inicio)

The line between arrows is the one that cause the exeption. Let me know if the is_there_surebet function is necessary to fix the problem, but I think it won't be the case.

dataframe filter python
1 Answers

The idea is to create a function that returns True \ False and apply it on the relevant columns. Please check if you find the following example helpful -

def my_filter(x):
    return x > 0

df = pd.DataFrame({"x": [1, 2, 3, -4, 5, -3], "y": ["a", "b", "c", "d", "e", "f"]})
df[df["x"].apply(my_filter)]

The results will be only the rows where the value in column x is greater than 0.

For something more sophisticated that requires input from multiple columns, you can use something like this -

def my_filter(record):
    if record["x"] < 0:
        return False
    return record["y"] in ["a", "b", "c"]

df[df.apply(my_filter, axis=1)]

Edit - adding more examples

If you want to use a parameter you can do -

def my_filter(record, my_param):
    if record["x"] < my_param:
        return False
    return record["y"] in ["a", "b", "c"]

df[df.apply(lambda x: my_filter(x, 5), axis=1)]
user_14819860
Related