Extract one hot encoding from a file into a dataset

user_4296244 0
Viewed 193

I have a dataset images and corresponding labels, where to each image file there is a .txt file which contains the one hot encoding:

0
0
0
0
1
0

My code looks something like this:

imageString = tf.read_file('image.jpg')
imageDecoded = tf.image.decode_jpeg(imageString)

labelString = tf.read_file(labelPath)
# decode csv string

but labelString looks like this:

tf.Tensor(b'0\n0\n0\n0\n1\n', shape=(), dtype=string)

Is there a way to transform this into an array of numbers inside tensorflow?

tensorflow-datasets tensorflow python
2 Answers

Here is a function to do that.

import tensorflow as tf

def read_label_file(labelPath):
    # Read file
    labelStr = tf.io.read_file(labelPath)
    # Split string (returns sparse tensor)
    labelStrSplit = tf.strings.split([labelStr])
    # Convert sparse tensor to dense
    labelStrSplitDense = tf.sparse.to_dense(labelStrSplit, default_value='')[0]
    # Convert to numbers
    labelNum = tf.strings.to_number(labelStrSplitDense)
    return labelNum

A test case:

import tensorflow as tf

# Write file for test
labelPath = 'labelData.txt'
labelTxt = '0\n0\n0\n0\n1\n0'
with open(labelPath, 'w') as f:
    f.write(labelTxt)
# Test the function
with tf.Session() as sess:
    label_data = read_label_file(labelPath)
    print(sess.run(label_data))

Output:

[0. 0. 0. 0. 1. 0.]

Note the function, as I wrote it, uses some of the new-ish API endpoints, you can also write it as below for more backwards compatibility, with almost the same meaning (there are slight differences between tf.strings.split and tf.string_split):

import tensorflow as tf

def read_label_file(labelPath):
    labelStr = tf.read_file(labelPath)
    labelStrSplit = tf.string_split([labelStr], delimiter='\n')
    labelStrSplitDense = tf.sparse_to_dense(labelStrSplit.indices,
                                            labelStrSplit.dense_shape,
                                            labelStrSplit.values, default_value='')[0]
    labelNum = tf.string_to_number(labelStrSplitDense)
    return labelNum
user_17827920

You can use basic python commands and convert it to a tensor. Try...

with open(labelPath) as f:
    lines = f.readlines()
    lines = [int(l.strip()) for l in lines if l.strip()]
labelString = tf.convert_to_tensor(lines, dtype='int32')
user_46204200
Related