How do stacked convolutional layers work in a CNN?

user_14037329 0
Viewed 2643

I am having trouble understanding the way 2 or more convolutional layers (each followed by a pooling layer) work in a CNN.

Consider the input to be a 3 channel 300x300 image. If the first convolution layer has 32 convolutions and the second layers have 64 convolutional layers, then the first layer creates 32 feature maps. But how many feature maps does the second layer create? Does every convolution out of 64 act on the previously generated 32 feature maps, thus creating 32*64 = 2048 feature maps in total? Or does something else take place?

A simple code relating the question is:

model = keras.models.Sequential([
keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(300, 300, 3)),
keras.layers.MaxPooling2D(2, 2),

keras.layers.Conv2D(64, (3, 3), activation='relu'),
keras.layers.MaxPooling2D(2, 2)])
conv-neural-network keras tensorflow deep-learning python
2 Answers

The number of channels of the input matrix and the number of channels in each filter must match in order to be able to perform element-wise multiplication.

So the main difference between first and second convolutions is that the # of channels in the input matrix in the first convolution is 3 so we will use 32 filters where each filter has 3 channels (depth of kernel matrix).

For the second convolution, the input matrix has 32 channels (feature maps), so each filter for this convolution must have 32 channels as well. For example: each of the 64 filters will have the 32@3x3 shape.

The result of a convolution step for a single filter of 32@3x3 shape will be a single channel of WxH (Width, Height) shape. After applying all 64 filters (where each of them has shape: 32@3x3) we will get 64 channels, where each channel is a result of the convolution of a single filter.

user_88906040

The first convolution layer has 32 filters, but it applies to all THREE channels of the image, so the feature maps after the first Conv2D is 32x3 = 96, then the 64 filters of the second Conv2D each apply to each of that 96 feature maps, so after the 2nd Conv2D it's 64x96 = ... (fill in the blank).
But for simplicity, Keras only shows (..., 32) or (..., 64). You can use model.summary() to check.

user_112352050
Related