Tape is required when a tensor loss is passed

user_12654090 0
Viewed 2529

I have the following loss:

loss = loss(y_train_left_noc[:,:,0], soft_argmin).tolist()

where type(loss) is <class 'float'>

However, upon using this loss in the optimizer like so:

train = tf.keras.optimizers.Adam().minimize(loss, [k1, k2, k3])

where k1, k2 and k3 are convolution kernels, I get the following error:

Traceback (most recent call last):
  File "train.py", line 277, in <module>
    k3
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 385, in minimize
    loss, var_list=var_list, grad_loss=grad_loss, tape=tape)
  File "/usr/local/lib/python3.7/site-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py", line 440, in _compute_gradients
    raise ValueError("`tape` is required when a `Tensor` loss is passed.")
ValueError: `tape` is required when a `Tensor` loss is passed.

If the loss is of type float, then why does Tensorflow say that a Tensor loss is passed?

tensorflow numpy python
2 Answers

Answering the solution from here by @nikitamaia, for the benefit of the community.

The error message you're seeing is here in the source code:

if not callable(loss) and tape is None:
    raise ValueError("`tape` is required when a `Tensor` loss is passed.")

This error message seems only to be in tf-nightly, and if you run the code in TF 2.3 you'll notice a slightly different error message TypeError: 'float' object is not callable which is perhaps a more clear message. Either way, since you're passing a scalar loss value, and not a callable, an error is expected. If you want to use a scalar loss value and not pass the callable, then you should use GradientTape, as noted in the error message.

user_142906970

Example code.

If loss is a tensor the following need to be added:

with tf.control_dependencies(gen_updates):
  d_train_opt = tf.optimizers.Adam(
    learning_rate=LR_D, 
    beta_1=BETA1
  ).minimize(
      d_loss, 
      var_list=d_vars
    )
  

with tf.control_dependencies(gen_updates):
  d_train_opt = tf.optimizers.Adam(
    learning_rate=LR_D, 
    beta_1=BETA1
  ).minimize(
      d_loss, 
      var_list=d_vars,
      tape=tf.GradientTape(persistent=False)
    )
user_1275080
Related