Number of time steps in one iteration of RLlib training

user_3443033 0
Viewed 1298

I am new to reinforcement learning and I am working on the RL of a custom environment in OpenAI gym with RLlib. When I create a custom environment, do I need to specify the number of episodes in the __init__() method? ALso, when I train the agent with 

for _ in range(10):
     trainer.train()

how many time steps are taken in one iteration? is it equal to number of episodes defined in the custom environment? Thank you.

rllib reinforcement-learning ray python
2 Answers

I think what you need to set for the max number of steps in one episode is the hyperparameter `horizon'

user_97832530

I found with Ray that episodes only terminate when your environment sets 'done/_terminated'. When running on other frameworks, the algorithms often had a hyperparameter for num_steps, etc. I discovered this because if my agent got stuck, it would just sit there forever, so I needed to add a max time steps check in the environment itself.

The number of episodes is set up outside of the environment though.

user_1885870
Related