Tensorflow batch: keep result as strings

user_192373 0
Viewed 465

This simple program

import tensorflow as tf

input = 'string'
batch = tf.train.batch([tf.constant(input)], batch_size=1)
with tf.Session() as sess:
    tf.train.start_queue_runners()
    output, = sess.run(batch)

    print(1, input, output)
    print(2, str(output, 'utf-8'))
    print(3, input.split('i'))
    print(4, str(output, 'utf-8').split('i'))
    print(5, output.split('i'))

prints

1 string b'string'
2 string
3 ['str', 'ng']
4 ['str', 'ng']
ERROR:tensorflow:Exception in QueueRunner: Session has been closed.
      print(5, output.split('i'))
TypeError: a bytes-like object is required, not 'str'

Why isn't the result a list of strings, if the input is?

OK, @jdehesa explained WHY, but not how to 'fix' it. I can apply bytes.decode() to the results of session:

output, = map(bytes.decode, sess.run(batch))

And there exists tf.map_fn() that should do the same on tensors. The only question is how I can use this in my scenario?

PS: actually, the error message is puzzling, too. The problem is that we provide a bytes object, not a string. But the TypeError suggests exactly the opposite.

PPS: the error message explained, thanks to @jdehesa: it was about the split()'s parameter, not the object. output.split(b'i') works well!

tensorflow python
1 Answers

The problem is that output is a bytes object, because TensorFlow tf.string tensors are indeed made of bytes. But then you are trying to use split with a str separator, and that is why it complains. Try:

output.split(b'i')

or:

output.decode().split('i')
user_17827920
Related