Goal: In Stable Baselines 3, I want to be able to run multiple workers on my environment in parallel (multiprocessing) to train my model.
Method: As shown in this Google Colab, I believe I just need to run the below line of code:
vec_env = make_vec_env(env_id, n_envs=num_cpu)
However, I have a custom environment, which doesn't have an env_id. So, when I run it just like "make_vec_env(MyEnvironment(), n_envs=3)", I get an error saying that my environment isn't callable. There seems to be a general lack of documentation around this, but from what I gather from this thread, I need to register my custom environment with Gym so that I can call on it with the make_vec_env() function.
My first question: Is there any other way to run multiple workers on a custom environment? If not...
My second question: How do I register my custom environment with Gym?
Again, documentation seems somewhat lacking. I have found these one, two, three posts which outline the steps. However, I don't understand - can I just place this folder anywhere I want? How does Gym know where to find it? Why do I need two init.py files?
Any guidance whatsoever would be hugely appreciated.