How to return none type in pandas GROUPEDMAP UDF

Viewed 39

I am using Pyspark UDF where I am calling a function that either returns a Pandas Dataframe or just None.

But the error says that

TypeError: Return type of the user-defined function should be pandas.DataFrame, but is <class 'NoneType'>

Here is my code looks like

@pandas_udf(schema,functionType=PandasUDFType.GROUPED_MAP)   
def my_func(df):
    if condition:
        return pd.DataFrame(....)

I am calling the function this way:

df.groupby('id').apply(my_func)

How can I handle the cases where the condition fails or also in case of any exceptions occur?

0 Answers
Related