Set a pre-trained weight at keras dense layer

user_15776410 0
Viewed 292

I have a terrible problem on doing my work with Keras, and here's a problem;

encoder_path = "my own encoder path"
if (os.path.exists(encoder_path)):
    encoder = tf.keras.models.load_model(encoder_path, compile=False)
    # encoder.summary()
    print("Encoder model exist & loaded ...")

else:
    print("There is no file! Check " + encoder_path + ' ...')

## Making latent vector layer code ##
loc_z_mean = len(encoder.layers) - 11
loc_z_log_var = len(encoder.layers) - 10
z_mean = encoder.layers[loc_z_mean]
z_log_var = encoder.layers[loc_z_log_var]
print(z_mean.get_weights())
z_mean_weights = z_mean.get_weights()[0]
z_mean_bias = z_mean.get_weights()[1]
print(np.shape(z_mean_weights))
print(np.shape(z_mean_bias))
z_log_var_weights = z_log_var.get_weights()[0]
z_log_var_bias = z_log_var.get_weights()[1]

z_weights = z_mean_weights + np.exp(0.5 * z_log_var_weights)
z_bias = z_mean_bias + np.exp(0.5 * z_log_var_bias)
# z_weights_init = z_weights.numpy()
# z_bias_init = z_bias.numpy()
z = tf.keras.layers.Dense(16, name="latent_z").set_weights([z_weights, z_bias])
# z = tf.keras.layers.Dense(16, kernel_initializer=z_weights_init, bias_initializer=z_bias_init, name="latent_z")
# z.trainable = False # Freeze layer
print(z)

I'm trying to make a new weight from former model. But it doesn't work when trying

z = tf.keras.layers.Dense(16, name="latent_z").set_weights([z_weights, z_bias])

with this error;

ValueError: You called `set_weights(weights)` on layer "latent_z" with a weight list of length 2, but the layer was expecting 0 weights. Provided weights: [array([[0.85919297, 0.39330506, 1.4273021 , 0.780...

I set the shape of z_weights and z_bias size, (16, 16) and (16,) respectively since those size are exactly same as first loaded weights, but it didn't work.

Is there any solution?

Thanks in advance.

keras tensorflow
1 Answers

You need to invoke build() for the layer with the required input_shape before being able to set the weights

z = tf.keras.layers.Dense(16, name="latent_z")
# set shape
z.build(input_shape=(100,)

# using random weights and bias for now
bias = np.random.randn(16)
weight = np.random.randn(100,16)
z.set_weights([weight, bias])

Without invoking build(), the layers weights are not defined since the shape of the weights depend on the shape of the input. In the above sample code, the input shape was 100 and hence the weight shape is [100,16].

user_128073620
Related