How to compute grads_and_vars with GradientDescentOptimizer

user_5407129 0
Viewed 1229

I have the following CNN code implementation.

optimizer = tf.train.GradientDescentOptimizer(cnn.learning_rate).minimize(loss = cnn.loss, global_step=global_step)
grads_and_vars = optimizer.compute_gradients(cnn.loss)
train_op = optimizer.apply_gradients(grads_and_vars, global_step=global_step)

Using it, I get the following error:

grads_and_vars = optimizer.compute_gradients(cnn.loss)
AttributeError: 'Operation' object has no attribute 'compute_gradients'

I noticed that using the above GradientDescentOptimizer implementation with tf.train.exponential_decay does not provide the attribute 'compute_gradients' for the optimizer.

Can someone help me with this?

gradient-descent conv-neural-network tensorflow deep-learning
2 Answers

Both compute_gradients and apply_gradients are methods of the tf.train.GradientDescentOptimizer class.

The problem is that you are defining your optimizer as the step. You should remove the minimize call, like this:

optimizer = tf.train.GradientDescentOptimizer(cnn.learning_rate)
grads_and_vars = optimizer.compute_gradients(cnn.loss)
train_op = optimizer.apply_gradients(grads_and_vars, global_step=global_step)

and it should work.

However, if you are not doing any specific use of the gradients, you can indeed use .minimize(...) to directly define a training step, similarly to your train_op:

optimizer = tf.train.GradientDescentOptimizer(cnn.learning_rate)
train_op = optimizer.minimize(loss=cnn.loss, global_step=global_step)

Further explanation on how apply_gradients works here.

user_39914700
Related