multiprocessing not updating dataframe as the process defines

Viewed 18

I'm trying to run the following process using the multiprocessing library (there's only one process because I'm still testing it, the idea would be to run the process for several datasets. That's why I'm using multiprocessing).

from multiprocessing import Process
import multiprocessing as mp
import time
start = time.perf_counter()

processes = []

p = Process(target=test, args=[df_1])
p.start()
processes.append(p)

for process in processes:
    process.join()

finish = time.perf_counter()

print(f'Finished in {round(finish-start, 2)} s')

where the test function is:

def test(df):
    for i,row in df.iterrows():
        results = similarity(embeddings, df_acordaos,i,0)
        df.at[i,'SIMILARITY_ALL']=similarity_formated(results)
    print(df_1.columns)
    return df

The 'print()' performed inside the function displays the created column 'SIMILARITY_ALL' however, when printing the columns outside the function, the created column 'SIMILARITY_ALL' does not appear.

print of the dataset columns inside the function: enter image description here

print of the dataset columns outside the function: enter image description here

0 Answers
Related