How to loop through (none, 256) shape tensor array?

user_11857846 0
Viewed 210

I am trying to write a custom loss function to a Keras model. This loss function takes in the prediction and suppresses all the predictions except the highest one to zero. Something like this:

  def loss_function(y_true, y_pred):
    s=tf.shape(y_pred)
    loop=tf.unstack(y_pred)   
    th=[tf.math.argmax(y_pred[i]) for i in loop]
    output=tf.zeros([len(loop),len(anchor_list)])
    y_pred_in=tf.tensor_scatter_nd_update(output,[[i,th[i]] for i in loop],[y_pred[i,th[i]] for i in loop])
    cce = tf.keras.losses.CategoricalCrossentropy()
    loss=cce(y_true,y_pred)
    return loss

I realized that while doing so I have to take care of batch size as well. For this, I want to create lists and arrays according to the shape of the prediction tensor. I found out that we cannot iterate over a tensor. So, I unstacked the tensor. However, get the following error:

 shape=tf.unstack(y_pred)

    ValueError: Cannot infer argument `num` from the shape (None, None)

How do I solve this problem?

tensorflow2.0 keras tensorflow deep-learning python
2 Answers

This loss function takes in the prediction and suppresses all the predictions except the highest one to zero

I think this can easily be solved with tf.where:

import tensorflow as tf

y_true = [[0, 1, 0], [0, 0, 1]]
y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]]

def loss_function(y_true, y_pred):    
    max_pred_value = tf.reduce_max(y_pred)
    y_pred_new = tf.where(tf.not_equal(y_pred, max_pred_value), tf.zeros_like(y_pred, dtype=tf.float32), max_pred_value)
    cce = tf.keras.losses.CategoricalCrossentropy()
    tf.print('y_pred_new -->', y_pred_new)
    return cce(y_true, y_pred_new)

tf.print('loss -->', loss_function(y_true, y_pred))

y_pred_new --> [[0 0.95 0]
                [0 0 0]]
loss --> nan

But as you can see, converting almost all values to zero will lead to nan loss values.

Update 1: Try something like this if you want the max value across the second dimension:

import tensorflow as tf

y_true = [[0, 1, 0], [0, 0, 1]]
y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]]

def loss_function(y_true, y_pred):
    y_pred_shape = tf.shape(y_pred)   
    max_values = tf.reduce_max(y_pred, axis=1)
    max_values = tf.reshape(tf.repeat(max_values, repeats = y_pred_shape[1]), y_pred_shape)
    y_pred_new = tf.where(tf.not_equal(y_pred, max_values), tf.zeros_like(y_pred, dtype=tf.float32), max_values)
    cce = tf.keras.losses.CategoricalCrossentropy()
    tf.print('y_pred_new -->', y_pred_new)
    return cce(y_true, y_pred_new)

tf.print('loss -->', loss_function(y_true, y_pred))

y_pred_new --> [[0 0.95 0]
 [0 0.8 0]]
loss --> 8.0590477
user_96578610

you can get a list representation of a tensor by using

list_from_tensor = list(tenor.numpy())

you can convert the list back to a tensor by using

tensor = tf.convert_to_tensor(list_from_tensor, dtype=tf.float32) #  tensor of floats
tensor = tf.convert_to_tensor(list_from_tensor, dtype=tf.int32) # tensor of integers

Here a link to a website where all possible types are listed

user_148562410
Related