pickle.PicklingError: Can't pickle <function past_match_sim at 0x7fa26e03b7b8>: attribute lookup past_match_sim on __main__ failed

user_17418911 0
Viewed 419

I am trying to run multiple files at a single time using multiprocessing module.....

import multiprocessing
import schedule
import time
if len(abc)==1:
    def live_run1():     
        def lv1():
            exec(open('/path to file/file1.py').read())                            
        def lv_s():
            exec(open('/path to file/file2.py').read())
        import multiprocessing
        if __name__ == '__main__':
            p11 = multiprocessing.Process(target=lv1)
            p12 = multiprocessing.Process(target=lv_s)
            p11.start()
            p12.start()
            p11.join()
            p12.join()                        
            time.sleep(500)
            exec(open('/path to file/file3.py').read())
            return schedule.CancelJob
        schedule.every().day.at("10:30").do(live_run1)
        while True:
            schedule.run_pending()
            time.sleep(1)

the error i got :

pickle.PicklingError: Can't pickle <function past_match_sim at 0x7fa26e03b7b8>: attribute lookup past_match_sim on __main__ failed

I am not able to come out from this problem....

Appreciate any help

multiprocessing linux python
1 Answers

Possible fixups to your code can be following.

I modified code not to use schedule module because it has some troubles pickling even global functions, hence I decided to implement same functionality without schedule:

Try it online!

import multiprocessing, time, datetime

def lv1():
    exec(open('0603.py').read())
    
def lv_s():
    exec(open('0604.py').read())

def live_run1():
    p11 = multiprocessing.Process(target=lv1)
    p12 = multiprocessing.Process(target=lv_s)
    p11.start()
    p12.start()
    p11.join()
    p12.join()                        
    time.sleep(2)
    exec(open('0605.py').read())

def today_at(hour, minute = 0, second = 0):
    now = datetime.datetime.now()
    return datetime.datetime(
        year = now.year, month = now.month,
        day = now.day, hour = hour,
        minute = minute, second = second)

def now():
    return datetime.datetime.now()

def run_at(hour, minute, second = 0, *, done = set()):
    at = today_at(hour, minute, second)
    if str(at) not in done and now() >= at and (
        now() - at <= datetime.timedelta(seconds = 10)
    ):
        print('started running at', now())
        live_run1()
        print('finished running at', now())
        done.add(str(at))
            
if __name__ == '__main__':
    print('now is', now())
    while True:
        run_at(7, 3, 40)
        run_at(10, 40)
        time.sleep(1)

Output:

now is 2021-11-22 07:03:32.389326
started running at 2021-11-22 07:03:40.404687
0603
0604
0605
finished running at 2021-11-22 07:03:42.447017
user_9415310
Related