tensorflow.keras.losses.cosine_similarity returns positive values in TensorFlow 1.15

user_9560387 0
Viewed 557

Not sure if I am doing something wrong here, but for whatever reason when I follow the example for tf 2.4.1 here

I don't get the same results, in fact, I get the negative of the result in the example.

Here's what I am doing

import tensorflow as tf
from tensorflow.keras.losses import cosine_similarity

y_true = [[0., 1.], [1., 1.], [1., 1.]]
y_pred = [[1., 0.], [1., 1.], [-1., -1.]]

loss = cosine_similarity(y_true, y_pred, axis=1) 
loss_numpy = loss.eval(session=tf.Session())

print(loss_numpy) # array([ 0.        ,  0.99999994, -0.99999994], dtype=float32)
# expected output array([-0.        , -0.99999994,  0.99999994], dtype=float32)

I am assuming this is not the case in tf 2+, but is this a known issue that I just have to deal with, or is something else going on?

P.S.

I have gone around that by defining

cosine_proximity_loss = lambda y_true, y_pred: -1. * cosine_similarity(y_true, y_pred)

and using that in my model. If anyone knows that I should not do this any advice would be appreciated!

keras tensorflow machine-learning python-3.x python
1 Answers

It's not your fault, for some historical reasons, tf.keras.losses.cosine_similarity in tf v1.15 will only return cosine value, but in tf v2.4.1 will return negative cosine value.

tf v1.15 source code of tf.keras.losses.cosine_similarity:

y_true = nn.l2_normalize(y_true, axis=axis)
y_pred = nn.l2_normalize(y_pred, axis=axis)
return math_ops.reduce_sum(y_true * y_pred, axis=axis)

tf v2.4.1 source code of tf.keras.losses.cosine_similarity:

y_true = nn.l2_normalize(y_true, axis=axis)
y_pred = nn.l2_normalize(y_pred, axis=axis)
return -math_ops.reduce_sum(y_true * y_pred, axis=axis)

So, if you want using tf.keras.losses.cosine_similarity in tf v1.15 as same way as in tf v2.4.1, just add minus sign before it outputs

user_135095400
Related