How to convert string labels to int labels when using `tf.data.Dataset.list_files`

user_12125250 0
Viewed 2609

I am new to machine learning and have tried many different ways to create an image data pipeline. I went through Udacity and Google Tutorials and they all work great, so long as you get your image data from TF Hub or some other online source. Unfortunately, I haven't found ANY end-to-end examples (that work) of how to do this correctly when you have your own locally stored images. This is as close as I have gotten.

I have my own image data and am trying to get it in the correct Tensor format to fit my model (I'm using a feature extractor from TensorFlow Hub). I froze all the layers, added a binary output layer.

I get the following error when I run a training session with train_batches model.fit

UnimplementedError: Cast string to float is not supported

I'm pretty sure it has to do with the class labels being strings (pulled from folder name). Is there anyway I can pass a dictionary with values 0 and 1 to relabel?

Am I missing something much more crucial than this?

image_path_list = glob('/content/sample_data/tng (1)/*/*.jpg')
data = tf.data.Dataset.list_files(image_path_list)
print(image_path_list)
print()
print(data)

def load_images(path):
    image = tf.io.read_file(path)
    image = tf.io.decode_jpeg(image, channels=3)
    label = tf.strings.split(path, os.path.sep)[-2]
    return image, label
data = data.map(load_images)

def preprocess(image, label):
    image = tf.image.resize(image, (IMG_RES, IMG_RES))
    image /= 255.
    return image, label

BATCH_SIZE = 1

train_batches = data.shuffle(len(image_path_list)).map(preprocess).batch(BATCH_SIZE).prefetch(1)
tensorflow image-processing python
2 Answers

Ok, looks like I found a fix that seems to work but it requires me to relabel the folders by number. Since I have only two classes, I wrote a simple script to relabel the folders by number 0 and 1 respectively.

Once that was done, I added the following line after the label is defined:

label = tf.strings.to_number(label)

This takes my numeric labels from str to int type and everything seems to work fine. Would love to hear if there is a completely different approach to take since I couldn't find any clue on Stackoverflow or in ANY of the TF documentation.

user_121252500

Say I know in advance the classes, I simply used if-else to return integer-code of classes. This method should work for TF>=2.0

def parse_label(label):
    if label == 'a':
        return 0
    elif label == 'b':
        return 1
    else:
        return 2

def load_images(path):
    image = tf.io.read_file(path)
    image = tf.io.decode_jpeg(image, channels=3)
    label = tf.strings.split(path, os.path.sep)[-2]
    label = parse_label(label)
    return image, label
data = data.map(load_images)

def preprocess(image, label):
    image = tf.image.resize(image, (IMG_RES, IMG_RES))
    image /= 255.
    return image, label

BATCH_SIZE = 1

train_batches = data.shuffle(len(image_path_list)).map(preprocess).batch(BATCH_SIZE).prefetch(1)
user_103720690
Related