I would appreciate any help or suggestion from the community regarding the following code example, see below.
In this example I invoke class member function from another member function in parallel. It turns out, when a member function self.print_id() is called by JobLib framework, then
- the entire class is cloned (without calling the constructor), serialized and passed to a process.
- a new unique ID is assigned to each copy of the class.
Questions:
- Is this a correct construction?
- Is there any apparent side effect that might happen?
- What if the "class A" is inherited, can I expect the correct behaviour of self.print_id() member of a derived class?
Thank you in advance.
from joblib import Parallel, delayed
class A:
def __init__(self):
self.state = -1
print(f"constructor is called, id: {id(self)}", flush=True)
def print_id(self, index):
self.state += 1
print(f"state: {self.state}, id: {id(self)}, index: {index}")
def parallel(self):
Parallel(n_jobs=8, prefer="processes")(
delayed(self.print_id)(i) for i in range(8))
a = A()
a.print_id(777)
a.parallel()
constructor is called, id: 140331251437424
state: 0, id: 140331251437424, index: 777
state: 1, id: 140226941702448, index: 0
state: 1, id: 140520215830832, index: 1
state: 1, id: 140520209383920, index: 2
state: 1, id: 140520215850624, index: 3
state: 1, id: 140520209384352, index: 4
state: 1, id: 140520215621200, index: 5
state: 1, id: 140520209384832, index: 6
state: 1, id: 140520215621296, index: 7