I use tf.keras.backend.gradient to calculate the gradient, but why the result is all 0

Viewed 29

I met a problem when i try to caculate the gradient with package keras.backent.
Here is my code

from tensorflow.keras import backend as K, losses, Model, models
import tensorflow as tf
import numpy as np
tf.compat.v1.disable_eager_execution()

# pre-operation
model = models.load_model("")
image = ""
y_true = ""


def fun(model, image, y_true):
    y_pred = model.output

    loss = losses.categorical_crossentropy(y_true, y_pred)

    gradient = K.gradients(loss, model.input)
    gradient = gradient[0]

    sess = tf.compat.v1.keras.backend.get_session()
    res = sess.run(gradient, feed_dict={model.input: image})
    if res.any() == 0.:
        print('all is 0')

    return res

img_attack = fun(model, image, y_true)

I trained a nice model with mnist, after that i wrote the code above and try to calculate the gradient, but the gradient is always '0', like below, every item is '0' during the matrix.

  ...

  [[0.]
   [0.]
   [0.]
   ...
   [0.]
   [0.]
   [0.]]

  [[0.]
   [0.]
   [0.]
   ...
   [0.]
   [0.]
   [0.]]

  [[0.]
   [0.]
   [0.]
   ...
   [0.]
   [0.]
   [0.]]]]

I've watched the param like 'loss', 'y_pred',... , all is normal, like loss is 1e-07. I wonder if the value of y_pred and y_true is too closed after softmax, but the gradient should not be all 0, right? I really have no idea.

Pre-Thanks for all advice!

0 Answers
Related