Keras: ValueError: Negative dimension size caused by subtracting 5 from 1

user_1528443 0
Viewed 9700

When I try to define my model, I get the following error message: ValueError: Negative dimension size caused by subtracting 5 from 1 for 'conv2d_19/convolution' (op: 'Conv2D') with input shapes: [?,1,1,32], [5,5,32,64].

Here is a code snippet to reproduce the issue:

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D, AveragePooling2D
import numpy as np

nb_filters = 32
batch_size = 128
epochs = 5
nb_classes = 10

pool_size = (3, 3)
kernel_size = (5, 5)

img_rows = 32
img_cols = 32

input_shape = (img_rows, img_cols, 1)

model = Sequential()
model.add(Convolution2D(nb_filters, kernel_size,
                        input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Convolution2D(nb_filters, kernel_size))
model.add(Activation('relu'))
model.add(AveragePooling2D(pool_size=pool_size))
model.add(Convolution2D(nb_filters*2, kernel_size))
model.add(Activation('relu'))
model.add(AveragePooling2D(pool_size=pool_size))
model.add(Flatten())
model.add(Dense(nb_classes))
model.add(Activation('softmax'))

The issue has been observed on MacOS, with Python 2.7.11 and Keras 2.1.1.

Is there something wrong in the way I build my model?

Here is the architecture I want to build:

  1. Convolutional layer with 32 filters of size 5 × 5, max-pooling over 3 × 3 regions, and ReLU activations
  2. Convolutional layer with 32 filters of size 5 × 5, ReLU activations, and average-pooling over 3 × 3 regions
  3. Convolutional layer with 64 filters of size 5 × 5, ReLU activations, and average-pooling over 3 × 3 regions
  4. Fully connected layer with softmax.
keras python
1 Answers

You're input shape is simply to small for all the pooling that is applied in your model.

Let's have a look at your model:

model = Sequential()
model.add(Convolution2D(nb_filters, kernel_size, input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=pool_size))
model.add(Convolution2D(nb_filters, kernel_size))
model.add(Activation('relu'))
model.add(AveragePooling2D(pool_size=pool_size))

Until here everything is fine. The output shape of the AveragePooling2D layer is (1, 1, 32).

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 28, 28, 32)        832       
_________________________________________________________________
activation_1 (Activation)    (None, 28, 28, 32)        0         
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 9, 9, 32)          0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 5, 5, 32)          25632     
_________________________________________________________________
activation_2 (Activation)    (None, 5, 5, 32)          0         
_________________________________________________________________
average_pooling2d_1 (Average (None, 1, 1, 32)          0         
=================================================================

Now you want to apply another Convolution2D layer with a kernel size of 3, altough the output of the last layer is just (1, 1, 32). So you want to learn a kernel of size (3, 3) that is actually smaller than the data you want to learn it on.

Now you can either increase the input shape of your data ((79, 79, ?) is the smallest that works for your model) or you change your model.

  • Decrease the pool size
  • Apply a padding by adding padding='same' to your convolutional layer
  • Cut down yout model

Cheers

user_44954390
Related