I am trying to use the pandarallel module to speed-up my apply functions in pandas. When I run the example provided on the git page of pandarallel I get the following error: AttributeError: Can't pickle local object 'prepare_worker..closure..wrapper'
This is my code:
from pandarallel import pandarallel
import pandas as pd
import numpy as np
import math
pandarallel.initialize(nb_workers=4)
df_size = int(5e6)
df = pd.DataFrame(dict(a=np.random.randint(1, 8, df_size),
b=np.random.rand(df_size)))
def func(x):
return x
res_parallel = df.parallel_apply(func, axis=1)