Tensorflow Dataset issue at inference phase

Question

I created a char-level language generation with Tensorflow here. I used tf.placeholder API, which according to the google docs:

Feeding is least efficient way to feed data into a TensorFlow program.

I decided to change my code and replace it with new TensroFlow Dataset API.

I used from_generator to create Dataset:

dataset = tf.data.Dataset.from_generator(gen, (tf.int32, tf.int32),
                                             (tf.TensorShape([None, None]),
                                              tf.TensorShape([None, None])))
self.iterator = dataset.make_initializable_iterator()
self.inp, self.target = self.iterator.get_next()

As can be seen in above code, I used [None, None] for Tensorshape to give the model more generality. During the training everything is perfectly fine. But at inference some problem arise. In tf.placeholder API I used following code to generate characters:

def inference(self):
    converter = utils.TextReader(filename=FLAGS.CONVERTER_PATH)

    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())

        samples = []
        new_state = sess.run(self.init_state)
        c = 12 # random starting token
        samples.append(c)

        for i in range(1000):
            x = np.zeros((1, 1))
            x[0, 0] = c
            feed_dict = {
                self.inp: x,
                self.init_state: new_state
            }
            preds, new_state = sess.run([self.prediction, self.final_state], feed_dict=feed_dict)
            c = utils.pick_top_n(preds, converter.vocab_size)
            samples.append(c)

        samples = np.array(samples)
        print(converter.arr_to_text(samples))

In Dataset API, I dont have tf.placeholder to feed my previous character. And when I use the above code, as expected, following error happened:

InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,50] vs. shape[1] = [32,50]

At inference, the model use the same input shape ([32,50]) that I used for training. Which is not what I want (Actually, I define TensorShape([None,None]) to handle this but not works).

How can I fix the issue with new Dataset API?

Complete code.

Tensorflow Dataset issue at inference phase

0 Answers