Reshaping tensor in custom loss function in Keras

user_8916723 0
Viewed 1948

I'm getting None type when reshaping the tensor. This happens when compiling the model with loss function and the optimizer (before starting the training). What do I do?

Error:

TypeError: Failed to convert object of type <class 'tuple'> to Tensor. Contents: (None, -1). Consider casting elements to a supported type.

Custom loss function:

def custom_loss(y_true, y_pred):

        y_pred = K.reshape(y_pred, (K.get_variable_shape(y_pred)[0], -1))
        y_true = K.reshape(y_true, (K.get_variable_shape(y_true)[0], -1))
        y_pred = K.std(y_pred, axis=0)
        y_true = K.std(y_true, axis=0)
        loss = (1/2) * (y_pred - y_true) ** 2
        loss = K.mean(loss)

        return loss
keras tensorflow deep-learning machine-learning python
2 Answers

This happens because your y_true or y_pred tensors shape is not defined properly. None means here that the tensors shape is not strictly set, but it can vary based on the previous operations what we can't see. Or you just initialized your tensor like this.

How to fix it:

  • First you should investigate how the y_true or y_pred gets its shape and avoid getting a None shape, so the tensors will have a deterministic number of rows and cols

Example:

Your loss function works for proper inputs:

import tensorflow as tf
from keras import backend as K


def custom_loss(y_true, y_pred):
    y_pred = K.reshape(y_pred, (K.get_variable_shape(y_pred)[0], -1))
    y_true = K.reshape(y_true, (K.get_variable_shape(y_true)[0], -1))
    y_pred = K.std(y_pred, axis=0)
    y_true = K.std(y_true, axis=0)
    loss = (1 / 2) * (y_pred - y_true) ** 2

    return loss


a = tf.constant([[1.0, 2., 3.]])
b = tf.constant([[1., 2., 3.]])
loss = custom_loss(a, b)
loss = tf.Print(loss, [loss], "loss")

with tf.Session() as sess:
    _ = sess.run([loss])

But when using a placeholder where I have not defined the shape, will throw the same exception

a = tf.placeholder(tf.float32, (None, 32))
user_51080620

I fixed it:

def custom_loss(y_true, y_pred):

        y_pred = K.reshape(y_pred, (K.shape(y_pred)[0], -1))
        y_true = K.reshape(y_true, (K.shape(y_true)[0], -1))
        y_pred = K.std(y_pred, axis=0)
        y_true = K.std(y_true, axis=0)
        loss = (1/2) * (y_pred - y_true) ** 2
        loss = K.mean(loss)

        return loss

The problem was with finding the first dimension of the y_true and y_predict. During compilation, it will not get a true shape and hence it returns a None value. So, instead of getting the integer value of the shape, I get the tf.Tensor shape. I changed K.get_variable_shape(y_true) to just K.shape(y_true) and it solves.

user_89167230
Related