While migrating code from Tensorflow 1 to Tensorflow 2, how do I deal with the attribute error: 'Adam' object has no attribute 'compute_gradient'?

user_15482559 0
Viewed 585

I'm working with Tensorflow and am using code which has been implemented for Tensorflow 1 (https://github.com/openai/maddpg/blob/master/maddpg/common/tf_util.py). While migrating code to tensorflow 2, I'm able to make most of the changes using the literature provided online. However, I'm not able to find a suitable alternative to line 145:

gradients = optimizer.compute_gradients(objective, var_list=var_list)

Which throws an error

Attribute Error: 'Adam' object has no attribute 'compute_gradient'

Since this function no longer exists, what are the possible alternatives I can use? I have read that it is possible to use the following function instead:

gradients = optimizer.get_gradients(objective, var_list)

This throws a value error

ValueError: Variable <tf.Variable 'agent_0/q_func/fully_connected/weights:0' shape=(9, 64) 
dtype=float32> has `None` for gradient. Please make sure that all of your ops have a 
gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, 
K.round, K.eval.

Versions:
tensorflow              2.4.1
tensorflow-estimator    2.4.0
tensorflow python
2 Answers

You should do it using tf.GradientTape. Something like this:

with tf.GradientTape() as tape:
    y_pred = my_obj_function(w, b, x)
    loss = my_loss(y_pred, y)

dw, db = tape.gradient(loss, [w, b])
optimizer.apply_gradients(zip([dw, db], [w, b]))

So the gradient is inside the tape, as this object was watching the variables (w and b) and the gradient of the loss function with regard to these variables. Then you can pass the variables and their gradients to the optimizer to perform an iteration of optimization. The objective function and loss function for the above example:

def my_loss(y_pred, y_true):
    return tf.abs(y_pred - y_true)


def my_obj_function(w, b, x):
    return w * x + b

Update

As I mentioned in my comment, make sure all elements inside the var lists are TF variables. if some of them are not, covert them simply by using:

var_list = [tf.variable(var) for var in var_list]

Then you would be able to calculate the gradient as fallow:

with tf.GradientTape() as tape:
    cost = objective(target, the_inputs)
gradients = tape.gradient(cost, var_list)

the_inputs are the necessary values to calculate the cost using objective function. the tape only watches the values that are defined as TF variables and only can calculate the gradients for them. And having the gradients, you can simply use the optimizer to reduce the cost in this iteration:

optimizer.apply_gradients(zip(gradients, var_list))
user_18218770

tf.GradientTape will work. Look here:

lst_var = [tf.variable(var) for var in var_list]

Now you can calculate it:

with tf.GradientTape() as tp:
    cost = objective(target, inputs)
    gradients = tp.gradient(cost, lst_var)
user_131770270
Related