How do I correctly extract Keras ConvNet weights matrices for use in Excel?

user_13840139 0
Viewed 265

I'm trying to build a very simple convolutional neural network in Excel. The model is an image classifier that tries to identify hand-written I's, O's, and X's; and was trained on a subset of the EMNIST letters dataset using Keras. The Excel spreadsheet is only the forward propagation phase (i.e. all the actual training took place in Keras), and I'm using it primarily for illustration and understanding purposes.

The model architecture is very simple: input > convolution layer (5 3x3 filters) > maxpool > dense output layer (3 classes).

The model scores 99.33% accuracy on the testing set, so I know it works fine. Individual samples are correctly classified using model.predict, but when I try the same sample in the Excel version the prediction is often different (and incorrect).

As I see it there could be two problems: first, the weights I've extracted from the trained model (and copy-pasted into Excel) could be in the wrong order/shape. Second, the Excel model is incorrectly set up. I'm fairly sure the Excel model is fine, so I think the weights are the issue.

So my question is the following: how do I extract the weights from Keras using get_weights in such a way that I can arrange them in Excel? Specifically in a 15x3 grid i.e. a vertical stack of the 5 3x3 filter weights for the convolutional layer, and a 980x3 grid for the Dense layer.

Here's what I've been doing - but it seems to be incorrect:

raw_conv1_weights = iox_model.get_weights()[0]
raw_conv1_biases = iox_model.get_weights()[1]
raw_dense_weights = iox_model.get_weights()[2]
raw_dense_biases = iox_model.get_weights()[3]

conv1_weights = pd.DataFrame(raw_conv1_weights.swapaxes(0,2).swapaxes(1,3).reshape(15,3))
conv1_biases = pd.DataFrame(raw_conv1_biases)
dense_weights = pd.DataFrame(raw_dense_weights)
dense_biases = pd.DataFrame(raw_dense_biases)

The dataframes are then saved to .xlsx format, and the grids are copy-pasted into the Excel model.

The model itself as constructed in Keras is shown below, in case it is relevant.

# defining the model architecture
n_classes = 3

iox_model = models.Sequential()

# first convolutional layer (with padding to preserve output dimensions)
iox_model.add(layers.Conv2D(5,(3,3), activation='relu', input_shape=img_shape, padding='same'))

# max pooling layer
iox_model.add(layers.MaxPooling2D((2,2)))

# fully connected output layer
iox_model.add(layers.Flatten())
iox_model.add(layers.Dense(n_classes, activation='softmax'))

Many thanks in advance!

EDIT:

I'm adding some details on how the Excel spreadsheet works as well, just in case.

  • The input image is a 28x28 grid with values in the range 0-255 (single-channel images).
  • The normalized image is created by dividing the value of each cell by 255 and is then 0 padded.
  • The convolution is performed using SUMPRODUCT on the normalized image with each of the filters in turn, which creates a 28x28 grid (feature map) for each of the 5 filters, and the bias term for the relevant filter is added to each cell in the new grid.
  • The ReLU activation is performed using MAX(0,cell) for each cell in the convolved feature maps, creating another 28x28 grid for each of the 5 filters. -The max-pooling is performed using MAX(2x2 cell batch) for each 2x2 cell batch in the activation maps.
  • The output is then flattened into a single column in order (max-pool map 1, row 1; map 1, row 2; ... ; map 2, row 1; and so on).
  • The logit output score is obtained by using SUMPRODUCT on on flattened max-pooling output and each of the 3 dense-layer weight columns in turn, with the relevant bias term added to each. The logit output is then softmax'ed using EXP and so on.
conv-neural-network keras numpy excel python
1 Answers

Happy to report that I've solved the problem. I figured I'd just post an answer here in case anyone ever needs it.

It turns out that it was neither the weights nor the forward propagation spreadsheet that was the issue. I tried re-shaping and swapping axes for the weights many times to no avail, and I also extracted the activations from Keras for the actual sample under investigation and compared them to the ones in Excel as follows:

layer_outputs = [layer.output for layer in iox_model.layers] 
activation_model = models.Model(inputs=iox_model.input, outputs=layer_outputs)
activations = activation_model.predict(np.expand_dims(sample, axis=0))

All the activations extracted in this way matched the ones in the Excel version. The problem was with the Flatten() layer. The array produced by flattening in Keras is ordered in the following way:

  • Filter 1, Col 1, Row 1; Filter 2, Col 1, Row 1; Filter 3, Col 1, Row 1; ... Filter 5, Col 14, Row 14;

rather than:

  • Filter 1, Col 1, Row 1; Filter 1, Col 2, Row 1; Filter 1, Col 3, Row 1; ... Filter 5, Col 14, Row 14;

as I had assumed.

The way I did it meant that the SUMPRODUCT representing the fully-connected output layer was incorrectly set up. I fixed the Excel version of the flattening layer so that the column it produces follows the same order as in Keras, and now it works fine.

user_138401390
Related