I'm trying to initialize the weights and biases of tensorflow.keras.layers.Dense with pretrained weights and biases. I've tried 2 methods, first
dims = [960,480,200,75,25]
dense_params['weights'] = (weights['w1'], biases['b1']) # [960, 480], 960
dense2_params['weights'] = (weights['w2'], biases['b2']) # [480, 200], 480
dense3_params['weights'] = (weights['w3'], biases['b3']) # [200, 75], 200
dense4_params['weights'] = (weights['w4'], biases['b4']) # [75, 25], 75
# define and initialize MLP model
mlp = Sequential([
Dense(dims[1], input_shape=(dims[0],), # 480
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=weights['w1'],
bias_initializer=biases['b1']),
Activation('sigmoid'),
Dense(dims[2], # 200
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=weights['w2'],
bias_initializer=biases['b2']),
Activation('sigmoid'),
Dense(dims[3], # 75
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=weights['w3'],
bias_initializer=biases['b3']),
Activation('sigmoid'),
Dense(dims[4], # 25
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=weights['w4'],
bias_initializer=biases['b4']),
Activation('sigmoid'),
Dense(4, kernel_initializer=glorot_uniform(seed=5555)),
Activation('softmax'),
])
mlp.compile(optimizer=Adam(lr=lr),
loss='categorical_crossentropy',
metrics=['accuracy'])
which gives the exception
ValueError: Could not interpret initializer identifier: tf.Tensor(
[[-0.2785529 0.4547253 0.893165 ... -0.73468804 -0.9307138
0.52965516]
[-0.31190312 0.5024654 0.35986277 ... -0.61041874 -0.39063498
-0.8553045 ]
[ 0.8878816 -0.75287884 -0.63842094 ... -0.38421196 -0.16647834
-0.26172128]
...
The second method I've tried is
dims = [960,480,200,75,25]
dense_params['weights'] = (weights['w1'], biases['b1']) # [960, 480], 960
dense2_params['weights'] = (weights['w2'], biases['b2']) # [480, 200], 480
dense3_params['weights'] = (weights['w3'], biases['b3']) # [200, 75], 200
dense4_params['weights'] = (weights['w4'], biases['b4']) # [75, 25], 75
mlp = Sequential([
Dense(dims[1], input_shape=(dims[0],), # 480
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=glorot_uniform(seed=3333),
**dense_params),
Activation('sigmoid'),
Dense(dims[2], # 200
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=glorot_uniform(seed=4444),
**dense2_params),
Activation('sigmoid'),
Dense(dims[3], # 75
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=glorot_uniform(seed=4444),
**dense3_params),
Activation('sigmoid'),
Dense(dims[4], # 25
kernel_regularizer=regularizers.l2(mlp_l2),
kernel_initializer=glorot_uniform(seed=4444),
**dense4_params),
Activation('sigmoid'),
Dense(4, kernel_initializer=glorot_uniform(seed=5555)),
Activation('softmax'),
])
mlp.compile(optimizer=Adam(lr=lr),
loss='categorical_crossentropy',
metrics=['accuracy'])
which gives ValueError: Layer weight shape (480,) not compatible with provided weight shape (960, 1)
I've also tried the last method without input_shape which gives the error
ValueError: Layer weight shape (200,) not compatible with provided weight shape (480, 1)
The weights are 2D Tensors with dimensions shown in the comments, the biases are 1D tensors also shown in the comments. How do I initialize weights and biases?