Why are gradients disconnected

Viewed 23

Consider the following code

@tf.function
def get_derivatives(function_to_diff,X):
  f                        = function_to_diff(X)
  ## Derivatives
  W                        = X[:,0]
  Z                        = X[:,1]
  V                        = X[:,2]
  df_dW                    = tf.gradients(f, X[:,0])
  return df_dW

I wanted get_derivatives to return the partial derivative of function_to_diff with respect to the first element of X.

However, when I run

def test_function(X):
  return tf.pow(X[:,0],2) * X[:,1] * X[:,2]

get_derivatives(test_function,X)

I get None.

If I use unconnected_gradients='zero' for tf.graidents, I'd get zeros. In other words, the gradients are disconnected.

Questions

  1. Why are the gradients disconnected?
  2. How can I get the derivative with respect to the first element of X, i.e. how can I restore the connection? I know that if I wrote
def test_function(x,y,z)
  return tf.pow(x,2) * y * z

@tf.function
def get_derivatives(function_to_diff,x,y,z):
  f                        = function_to_diff(x,y,z)
  df_dW                    = tf.gradients(f, x)
  return df_dW

This could fix the problem. What if my function can only take in one argument, i.e. what if my function looks like test_function(X)? For example, test_function could be a trained neural network that takes in only one argument.

0 Answers
Related