How to solve a type error while using RAdam optimizer?

user_12780914 0
Viewed 198

I am building a neural network using keras and tensorflow and I get a error at this place

def create_model():
  model = Sequential()
  model.add(Dense(4, input_dim=2, kernel_initializer='normal', activation='tanh'))
  model.add(Dense(6, activation='relu'))
  model.add(Dense(1, activation='sigmoid'))

  model.compile(loss='binary_crossentropy', optimizer=RAdam(learning_rate), metrics=['accuracy'])
  return model

model = create_model()

And I get the following error when I run my code in jupyter notebook,

TypeError                                 Traceback (most recent call last)
<ipython-input-14-2358feb9246f> in <module>
      1 # make a shallow neural network
----> 2 model = create_model()
      3 model.summary()

<ipython-input-13-7c6ab8b2130e> in create_model()
     10 
     11   # Compile model
---> 12   model.compile(loss='binary_crossentropy', optimizer=RAdam(learning_rate), metrics=['accuracy'])
     13   return model

~\anaconda3\envs\tf\lib\site-packages\keras_radam\optimizers.py in __init__(self, learning_rate, beta_1, beta_2, epsilon, decay, weight_decay, amsgrad, total_steps, warmup_proportion, min_lr, **kwargs)
     32                  total_steps=0, warmup_proportion=0.1, min_lr=0., **kwargs):
     33         learning_rate = kwargs.pop('learning_rate', learning_rate)
---> 34         super(RAdam, self).__init__(**kwargs)
     35         with K.name_scope(self.__class__.__name__):
     36             self.iterations = K.variable(0, dtype='int64', name='iterations')

TypeError: __init__() missing 1 required positional argument: 'name'

And these are the imports I have used for my code to run. I think I have most of the codes imported to build a shallow neural network

import numpy as np
import keras
import tensorflow as tf
from keras.models import Sequential
from keras.layers import Dense
from keras import backend as K
from keras.wrappers.scikit_learn import KerasClassifier
from keras_radam import RAdam
keras tensorflow
2 Answers

For others who may be looking for another solution.

RAdam is not in tensorflow.keras.optimizers and neither in keras by default, but in tensorflow-addons package, which is a better alternative (IMHO) than the external keras_radam library, considerably less prone to errors.

What you are looking for is here: https://www.tensorflow.org/addons/api_docs/python/tfa/optimizers/RectifiedAdam

#pip install tensorflow-addons
import tensorflow_addons as tfa
optimizer = tfa.optimizers.RectifiedAdam(lr=1e-3)
user_61170170

I was able to reproduce your problem. It happened when you have tf. keras but you load keras-radam with old keras. But this implementation supports both versions of keras or tf. keras. To use it with the new version, as also mentioned here, all you need to do as follows:

import os 
os.environ['TF_KERAS']='1'
from keras_radam import RAdam

The package will choose the tf. keras compatible version of RAdam()

from .backend import TF_KERAS

__all__ = ['RAdam']

if TF_KERAS:
    from .optimizer_v2 import RAdam
else:
    from .optimizers import

So, RAdam() will be imported from this script. But there is another issue. In the very latest version of tf, the following import has been updated

# from 
from tensorflow.python import os, math_ops, state_ops, control_flow_ops

# to 
from tensorflow.python.ops import math_ops, state_ops, control_flow_ops

From this point, you need to modify this import from the source script and it will solve this issue. Just modify the source script by replacing the above imports.

from keras import Sequential
from keras.layers import Dense

def create_model():
  model = Sequential()
  model.add(Dense(4, input_dim=2, kernel_initializer='normal', activation='tanh'))
  model.add(Dense(6, activation='relu'))
  model.add(Dense(1, activation='sigmoid'))

  model.compile(loss='binary_crossentropy', 
                optimizer=RAdam(learning_rate), 
                metrics=['accuracy'])
  return model

model = create_model()
user_92157800
Related