Is it normal to have the learn throughput decrease and the learn time increase as a Dueling DDQN agent gets trained?
A 7X increase in learn time after a few hours of training is pretty significant, is this something that you will expect to see? My system is only using 20% of the 8 core CPU and 25 GB out of 64 GB memory.
A ray.rllib.agents.dqn model is currently being trained on the CPU. The config are all defaults except
config['timesteps_per_iteration'] = 5000
config['noisy'] = True
config['compress_observations'] = True
config["num_workers"] = 4
config["num_envs_per_worker"] = 8
config["eager"] = True
After further training, the learn throughput has plummeted even more to 20. CPU usage remains at around 20%, memory usage at 50 GB.
Using Ray 0.8.5, TensorFlow 2.2.0, Python 3.8.3, Ubuntu 18.04 inside WSL2