Keras Regularizer class: access to the original model

Viewed 84

I'm implementing a custom regularizer myReg as a subclass of keras.regularizers.Regularizer. However, the computation of the regularization term needs the access to the original model that is being regularized. I checked the documentation of Keras and I didn't see there are things like self.model for me to access to the original model in the regularizer class. Since the regularizer needs to be created before the model is created, I can't even let the model to be an attribute of myReg. Is there any way to do it so that myReg can have an attribute like self.model?

class myReg(Regularizer):
    def __init__(self, lambda):
        self.Lambda = lambda
        self.model = ?

    def __call__(self, x):
        regularization = 0
        y_pred = self.model.predict(x_train)
        regularization += self.Lambda * K.sum(x * y_pred) // just an example of using self.model in the regularizer class
        return regularization


model = Sequential()
model.add(Dense(128, activation='relu', input_dim=784, kernel_regularizer=myReg(0.01)))
model.add(Dense(10, activation='softmax'))

model.compile('adam', 'sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, 100, 10)

Thanks in advance! Any suggestion is appreciated!

0 Answers
Related