Tensorflow : Trainning and test into the same graph with input queues

user_6627095 0
Viewed 598

I am facing to an issue that can't solve with what I found on the internet.

I have build my neural network and connect it to inpute pipeline. Reading data from tfrecord, with tf.train.batch and queueRunners, Coords, etc..

I have build my NN into a python class named "Model" that I use like :

model = Model(...all hyperparameter here...)

...

model.predict()

or

model.step()

All the training phase works very well.

But now I would like to add a test phase every X epoch/step of training.

I really don't know how to do this. I have several idea but I don't find the best one:

  • Duplicate the code into my class to get : loss_train and loss_test, and so on for each node of my graph ? (using sharing variable between train and test)
  • create 2 instance of my model :

model_train = Model(reuse=false)

model_test = Model(reuse=true)

  • use tf.make_template ? I really don't found any good exemple of this fonction ...
  • any other solution ?

I would appreciate any suggestion,

tensorflow training-data dataset testing
1 Answers

I came across the same Problem when experimenting with TFRecords Datasets. There are several possibilities. Since I wanted to do this on a Computer with only one GPU anyways I implemented it as follows:

# Training Dataset
train_dataset = tf.contrib.data.TFRecordDataset(train_files)
train_dataset = train_dataset.map(parse_function)
train_dataset = train_dataset.shuffle(buffer_size=10000)
train_dataset = train_dataset.batch(200)
# Validation Dataset
validation_dataset = tf.contrib.data.TFRecordDataset(val_files)
validation_dataset = validation_dataset.map(parse_function)
validation_dataset = validation_dataset.batch(200)

# A feedable iterator is defined by a handle placeholder and its structure. We
# could use the `output_types` and `output_shapes` properties of either
# `training_dataset` or `validation_dataset` here, because they have
# identical structure.
handle = tf.placeholder(tf.string, shape=[])
iterator = tf.contrib.data.Iterator.from_string_handle(handle,
 train_dataset.output_types, train_dataset.output_shapes)
next_element = iterator.get_next()

# Generate the Iterators
training_iterator = train_dataset.make_initializable_iterator()
validation_iterator = validation_dataset.make_one_shot_iterator()

# The `Iterator.string_handle()` method returns a tensor that can be evaluated
# and used to feed the `handle` placeholder.
training_handle = sess.run(training_iterator.string_handle())
validation_handle = sess.run(validation_iterator.string_handle())

Then for accessing the elements, you can just go like:

img, lbl = sess.run(next_element, feed_dict={handle: training_handle})

And exchange the handle dependant on what you are willing to do ATM.

Keep in mind that this is not parallelizable, however. Following this link, you can get insight into the different methods of creating multiple input pipelines Tensorflow | Reading Data.

user_46206430
Related