Multitask problem - define train/test generators

Question

I am working with a multitask problem and I want to define the appropriate train/test generators. So far I was working with a classification and a regression task separately so I would write eg for the classification task:

train_generator=img_gen.flow_from_dataframe(dataframe=train_dataset,x_col="file_loc",y_col="expr",target_size=(96, 96),batch_size=203,class_mode="raw")
test_generator=img_gen.flow_from_dataframe(dataframe=test_dataset_va,x_col="file_loc",y_col="expr",target_size=(96, 96),batch_size=93,shuffle=False,class_mode="raw")

and for the regression task:

train_generator=img_gen.flow_from_dataframe(dataframe=train_dataset,x_col="file_loc",y_col=["valence","arousal"],target_size=(96, 96),batch_size=203,class_mode="raw")
test_generator=img_gen.flow_from_dataframe(dataframe=test_dataset_va,x_col="file_loc",y_col=["valence","arousal"],target_size=(96, 96),batch_size=93,shuffle=False,class_mode="raw")

My data looks like below:

file_loc    expr    valence arousal
0   /content/train_set/images/0.jpg 1   0.785714    -0.055556
1   /content/train_set/images/100000.jpg    1   0.784476    -0.137627

I tried writing the train generator for the multitask like:

train_generator=img_gen.flow_from_dataframe(dataframe=train_dataset,x_col="file_loc",y_col=["expr","valence","arousal"],target_size=(96, 96),batch_size=203,class_mode="raw")

but it produces an error so I am sure it is not the right way. Any ideas?

resnet = tf.keras.applications.ResNet50( 
    include_top=False ,
    weights='imagenet' ,
    input_shape=(96, 96, 3) ,
    pooling="avg"
)

for layer in resnet.layers:
    layer.trainable = True

inputs = Input(shape=(96, 96, 3), name='main_input')
main_branch = resnet(inputs)
main_branch = Flatten()(main_branch)
#fully connected λιγα units
expr_branch = Dense(8, activation='softmax', name='expr_output')(main_branch)
va_branch = Dense(2, name='va_output')(main_branch)
model = Model(inputs = inputs,
     outputs = [expr_branch, va_branch])
plot_model(model)
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
              loss={'expr_output': 'sparse_categorical_crossentropy', 'va_output': 'mean_squared_error'},metrics={'expr_output': 'accuracy',
        'va_output': tf.keras.metrics.MeanSquaredError()})

history = model.fit_generator(
                   train_generator,
                   epochs=2,
                   steps_per_epoch=STEP_SIZE_TRAIN_resnet,
                   validation_data=test_generator,
                   validation_steps=STEP_SIZE_TEST_resnet,
                   max_queue_size=1,
                   shuffle=True,
                   verbose=1
                  )

When I put class_mode="raw" the error is: raw classmode

and when I put class_mode="multi_output" it says: multi_output classmode

Multitask problem - define train/test generators

0 Answers