How can I remove / delete specific weights below a threshold value from a neural network layer (dense layer or convolution layer) in Keras?

user_10072623 0
Viewed 2088

I have a neural network model in Keras which is trained on a dataset. Now, I would like to remove / delete some specific weights from Dense layers and Convolution layers. Is there a way to do that in Keras ? Please help. Thank you.

keras
1 Answers

You can use the get_weights and set_weights Layer methods. I will illustrate the principle for a toy model with one dense layer:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense

x = np.random.rand(10,10)
y = np.random.rand(10,1)

model = Sequential()
model.add(Dense(units=1, input_shape=(10, )))

model.compile(loss='mse', optimizer='sgd')
model.fit(x, y)

The weights for our layer consist of two numpy arrays, for the weights and the bias respectively:

weights = model.get_layer(index=-1).get_weights()
weights

[array([[ 0.51373583],
        [-0.64512885],
        [ 0.41436252],
        [-0.5230512 ],
        [-0.7272273 ],
        [ 0.17996567],
        [-0.63398594],
        [ 0.3416807 ],
        [ 0.66054064],
        [-0.11221145]], dtype=float32), array([0.01157765], dtype=float32)]

To 'remove' weights that are under a certain threshold, you can modify the weight arrays. For instance, to set all negative weights to zero:

sparsified_weights = []
threshold = 0.

for w in weights:
    bool_mask = (w > threshold).astype(int)
    sparsified_weights.append(w*bool_mask)

You can then set these weights on the layer with set_weights():

model.get_layer(index=-1).set_weights(sparsified_weights)

Verify this by calling get_weights() again.

user_54953810
Related