What does "deterministic=True" in stable baselines3 library means?

user_11512643 0
Viewed 1638

I am trying to apply the PPO algorithm from the stable baselines3 library https://stable-baselines3.readthedocs.io/en/master/ to a custom environment I made.

One thing I don't understand is the following line:

mean_reward, std_reward = evaluate_policy(model, env, n_eval_episodes=10, deterministic=True)

Should I always let deterministic equal True? When I keep deterministic="True", my custom environment "somehow" is always solved (i.e., always returning reward of 1 +/- 0 std).

And when I change it to "False", it starts behaving in a reasonable way (i.e., sometimes it succeeds (reward=1) and sometimes it fails (reward=0).

stable-baselines reinforcement-learning python-3.x
1 Answers

This parameter corresponds to "Whether to use deterministic or stochastic actions". So the thing is when you are selecting an action according to given state, the actor_network gives you a probability distribution. For example for two possible actions a1 and a2: [0.25, 0.75]. If you use deterministic=True, the result will be action a2 since it has more probability. In the case of deterministic=False, the result action will be selected with given probabilities [0.25, 0.75].

user_46808080
Related