I'm trying to create an AI that use Genetic Algorithm to solve a problem (tetris game). Instead of creating a single pygame, wait for the end to get the score of the AI and create another one until the end of the popolation, i tried to create a lot of AI in different process, then using a manager.dict() i tried to store all the points and then going forward with the rest, but something doesn't work with the multiprocesing
While running this code:
def thread_function(procnum, tetris_ai, return_dict_scores, return_dict_shapes, current_processes,current_processes_lock, dict_process):
# TODO: Passare index anche da mettere nel titolo per capire quanti ne mancano
return_dict_scores[procnum] = Game(procnum, tetris_ai).play_game()
# return_dict_shapes[procnum] = game.end_game_shapes()
with current_processes_lock:
current_processes.value -= 1
del dict_process[procnum]
def calculate_fitness(population):
"""Calculate the fitness value for the entire population of the generation."""
# First we create all_fit, an empty array, at the start. Then we proceed to start the chromosome x and we will
# calculate his fit_value. Then we will insert, inside the all_fit array, all the fit_values for each chromosome
all_fit = []
pieces_backup = []
max_points = 0
processes_created = 0
processes = []
manager = multiprocessing.Manager()
return_dict_scores = manager.dict()
return_dict_shapes = manager.dict()
dict_process = manager.dict()
current_processes = multiprocessing.Value(c_int) # defaults to 0
current_processes_lock = multiprocessing.Lock()
max_process = 100
while processes_created < configuration.NUMBER_OF_POPULATION:
cpu_usage = psutil.cpu_percent(interval=True)
if current_processes.value <= max_process:
if cpu_usage < 80:
p = Process(target=thread_function,
args=(
processes_created, population[processes_created], return_dict_scores,
return_dict_shapes,current_processes,current_processes_lock, dict_process))
p.start()
dict_process[processes_created] = 1
processes_created += 1
with current_processes_lock:
current_processes.value += 1
if len(dict_process) > 0:
print("Aspetto finiscano tutti i processi")
sleep(1)
for value in return_dict_scores.values():
if value > max_points:
pieces_backup = None
max_points = value
all_fit.append(value)
return all_fit, pieces_backup
I know it's a bit dirty code and not optimimized, but i'm trying to fix the error before try to make cleaner code. I tried to create a dictionary where i put inside all the process number and remove them in the thread_function, so that i can check if it's empty meaning that all process are ended and then go on with the other code, but doesn't seems to be this the problem, or maybe the error is actually that not all the process are ended before going forward with the rest of the code but my try to fix it is wrong.
I get this exception
Process Process-283:
Traceback (most recent call last):
File "D:\Python\lib\multiprocessing\managers.py", line 811, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Python\lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "D:\Python\lib\multiprocessing\process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "D:\Davide\Uni\Sistemi intelligenti M - Milano\genetic_tetris\genetic_algorithm.py", line 15, in thread_function
return_dict_scores[procnum] = Game(procnum, tetris_ai).play_game()
File "<string>", line 2, in __setitem__
File "D:\Python\lib\multiprocessing\managers.py", line 815, in _callmethod
self._connect()
File "D:\Python\lib\multiprocessing\managers.py", line 802, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "D:\Python\lib\multiprocessing\connection.py", line 490, in Client
c = PipeClient(address)
File "D:\Python\lib\multiprocessing\connection.py", line 691, in PipeClient
_winapi.WaitNamedPipe(address, 1000)
FileNotFoundError: [WinError 2] The system cannot find the file specified
Crossover...
Process Process-289:
Traceback (most recent call last):
File "D:\Python\lib\multiprocessing\managers.py", line 811, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Python\lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "D:\Python\lib\multiprocessing\process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "D:\Davide\Uni\Sistemi intelligenti M - Milano\genetic_tetris\genetic_algorithm.py", line 15, in thread_function
return_dict_scores[procnum] = Game(procnum, tetris_ai).play_game()
File "<string>", line 2, in __setitem__
File "D:\Python\lib\multiprocessing\managers.py", line 815, in _callmethod
self._connect()
File "D:\Python\lib\multiprocessing\managers.py", line 802, in _connect
conn = self._Client(self._token.address, authkey=self._authkey)
File "D:\Python\lib\multiprocessing\connection.py", line 490, in Client
c = PipeClient(address)
File "D:\Python\lib\multiprocessing\connection.py", line 691, in PipeClient
_winapi.WaitNamedPipe(address, 1000)
FileNotFoundError: [WinError 2] The system cannot find the file specified
Process Process-287:
Traceback (most recent call last):
File "D:\Python\lib\multiprocessing\managers.py", line 811, in _callmethod
conn = self._tls.connection
AttributeError: 'ForkAwareLocal' object has no attribute 'connection'
As you can see, inside the exception log, there is "Crossover...", that print is inside another function that is called after the 2 above ends, i don't know if this can help find the problem (maybe not all process ends before the next function start?).
EDIT 1: Doing something like seems to solve the problem:
for j in range(3):
processes = []
for i in range(100):
index += 1
p = Process(target=thread_function, args=(index, population[index], return_dict_scores,
return_dict_shapes,current_processes,current_processes_lock, dict_process))
p.start()
processes.append(p)
for p in processes:
p.join()
But it's not really what i want, because i would like that as soon as a process terminate a new one will be created, so that there will be no waste time
EDIT 2: Happened even using the little code above, just occured after hours of going instead of each time the code executes