Keras Tensorboard for DQN reinforcement learning

user_4049331 0
Viewed 843

I am using keras to build a DQN and train it in a classical DQN algorithm with a experience replay memory. Since in dqn you need to call model.fit many many times, meaning each time you sample batch data from the replay memory, new event log file is generation by each fit when using keras' model.fit( .... callbacks=TensorBoard(...)). It creates 2 problems, first it generates too mancy event log files and slow down a lot the training, and in the Tensorboard you cannot see any trend for example the gradual decrease of loss.

What is the method to visualize the training process like see the change of gradients and activations in reinforcement learning especial DQN implementation?

reinforcement-learning tensorboard keras tensorflow python
1 Answers

You can go through TensorFlow :

# Create FileWriter
file_writer = tf.summary.FileWriter(log_dir, tf.get_default_graph())
history = model.fit(state, Q_values, epochs=1, verbose=0)
loss = history.history['loss'][0]

# Add values to Tensorboard
training_summary = tf.Summary(value=[tf.Summary.Value(tag="loss", simple_value=loss),
                                     tf.Summary.Value(tag="score", simple_value=score)])
file_writer.add_summary(training_summary, global_step=total_frames)
user_115848570
Related