Missing seed functions to create reproducible RL experiments with Open AI Gym and PyTorch

user_11628437 0
Viewed 106

I am using a simple DDQN-PER (https://github.com/higgsfield/RL-Adventure/blob/master/4.prioritized%20dqn.ipynb) to run my RL experiments. This could be used as a minimal and verifiable code for my question.

Also, here are the seed functions that I have found to be useful -

random.seed(seed)
np.random.seed(seed)
torch.manual_seed(seed)
torch.cuda.manual_seed(seed)
torch.backends.cudnn.deterministic = True
env.seed(seed)
env.action_space.seed(seed)

Despite this, I cannot reproduce my experiments. I believe I am missing some seed function and will appreciate inputs on the same.

reinforcement-learning pytorch
1 Answers

Check you gym Version. There has been a major recent upgrade with v0.20 .

Here is also a blog post, describing that the new environments should be deterministic, quoting the essence:

Determinism

Up until now, the ALE hasn’t been completely deterministic. There exist games that implement their own PRNG seeded by various methods which depend on the console’s state. We now properly control for this source of stochasticity and can now say with confidence that the emulator is 100% deterministic.

user_80980680
Related