Why is my custom loss function not accepting converted numpy arrays?

user_6456798 0
Viewed 582

I am trying to write a custom loss function that will operate on the NN output after forward pass but before back prop by calling an external function (not written with the keras/tensorflow backend functions). The difference between this operated on output and an ideal output is what I want to use for the error calculation.

I have a custom loss function, loss_function(y_true, y_pred), which uses tf.py_function to call the external function, passing the y_pred argument to the function.

Ideally I would like to convert the y_pred argument to a numpy array, use this array with my function to return another numpy array, convert this numpy array to a tf tensor and then use this in the loss function as is required.

Here is the custom error function:

def loss_function(y_true,y_pred):

    y_pred2 = tf.py_function(func=SOA, inp=[y_pred], Tout=tf.float32)

    return tf.reduce_mean(tf.pow(y_true - y_pred2,2))

And here is the external function 'SOA' that it calls which just converts it to a numpy array then back to a tf tensor:


def SOA(y_pred):

    y_pred0 = np.square(np.copy(y_pred))
    print(type(y_pred0))
    y_pred1 = tf.convert_to_tensor((y_pred0))
    print(type(y_pred1))


    return y_pred1

I can operate directly on the y_pred in the external function using the various tf functions, but if I convert it to a numpy array (e.g. using np.copy(...)) then convert it back using tf.convert_to_tensor it gives the following error:

InvalidArgumentError: In[0] is not a matrix
     [[{{node training_188/Adam/gradients/dense_299/MatMul_grad/MatMul}}]]

Also, if I use y_true instead of y_pred it gives a different error:

ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.

This seems strange to me since it appears to be differentiating between objects of exactly the same type ().

loss-function keras tensorflow machine-learning python
1 Answers

Your post is a bit confusing but according to documentation of tf.py_func your function SOA should return ndarray (which is the numpy array type) not tensor. So omit the conversion and see if this helps.

Try this:

def SOA(y_pred):

    y_pred0 = np.square(np.copy(y_pred))
    print(type(y_pred0))

    return y_pred0
user_33998250
Related