Deleting tensorflow dataset graph each iteration

Viewed 39

I need to manipulate a tensorflow dataset over many iterations. However, when manipulating the dataset using map, I'm getting memory issues. I'm pretty sure that tensorflow is drawing a function graph each time calling map, and I would like to delete that graph each iteration.

I'm trying to use tf.keras.backend.clear_session and deleting it manually, but I'm still running into memory issues.

for i in range(10000):
  keras.backend.clear_session()
  ds = tf_dataset.map(map_fn)
  del ds
0 Answers
Related