TensorFlow "TypeError: Target data is missing" though dataset with 2 dimension tuple was supplied

user_16563202 0
Viewed 973

I'm trying to use a generator- based data set:

def gen():
    return zip(samples,feature)
ds = tf.data.Dataset.from_generator(gen,output_types=tf.dtypes.float32)


model.fit(ds,
          epochs=150,
          #callbacks=[tensorboard_callback]
          )
model.save("/sda/anyone/imagenet-in-np/transformer")

whereas feature is numpy.ndarray (2D array) whereas feature is numpy.ndarray (4D array) And I get the following error:

TypeError: Target data is missing. Your model has `loss`: BinaryCrossentropy, and therefore expects target data to be passed in `fit()`.

which is strange, as the target data is actually present. Whenever I separate the dataset to two

def gen():
    return samples
ds = tf.data.Dataset.from_generator(gen,output_types=tf.dtypes.float32)

def gen2():
    return feature
ds2= tf.data.Dataset.from_generator(gen2,output_types=tf.dtypes.float32)

model.fit(ds,ds2,
          epochs=150,
          #callbacks=[tensorboard_callback]
          )
model.save("/sda/anyone/imagenet-in-np/transformer")

I get:

    raise ValueError("`y` argument is not supported when using "
ValueError: `y` argument is not supported when using dataset as input.

Which means that TF doesn't accept this split.

So how should I use the generator API?

Edit #1: I tried as mentioned:

def gen():
    for element in zip(samples,feature):
        yield element
ds = tf.data.Dataset.from_generator(gen(),output_types=tf.dtypes.float32)

I get TypeError: generator must be a Python callable.

So I tried to swap it to :

def gen():
    for element in zip(samples,feature):
        yield element
ds = tf.data.Dataset.from_generator(gen,output_types=tf.dtypes.float32)

I get again:

    TypeError: Target data is missing. Your model has `loss`: BinaryCrossentropy, and therefore expects target data to be passed in `fit()`.
python-BaseException

Edit #2:

I've tried the following:

def gen():
    for element in zip(samples,feature):
        sample=tf.convert_to_tensor(element[0], dtype=tf.float32)
        f=tf.convert_to_tensor(element[1], dtype=tf.float32)
        print(sample.shape)
        yield (sample,f)
ds = tf.data.Dataset.from_generator(gen,output_types=(tf.float32, tf.float32))



model.fit(ds,
          epochs=150
          #callbacks=[tensorboard_callback]
          )

I get

2022-01-22 16:50:07.417109: W tensorflow/core/framework/op_kernel.cc:1745] OP_REQUIRES failed at conv_ops.cc:675 : INVALID_ARGUMENT: input must be 4-dimensional[16,17,10]
tensorflow-datasets keras tensorflow generator python
1 Answers

There are several things to note here,
how tf.fit works {link for further reading} : primarily it takes two parameters x & y separately, if you set so, like in your second case. Else, you can input both features & target under x as tf.Dataset object.
Use of tf .from_generator function {link for futher reading} : first of it is python generator as well, not as function. [FYI; do not use return, use yield instead] usually tf from_generator() uses only in the extreme cases [Ex. importing image with polygonal annotation, etc. ] and it slows down the input pipeline of the model training.

Since here you have to just input two numpy ndarrays, that can easily done by using tf. Dataset object directly : working example

user_107293030
Related