This is the Convolutional neural network I´ve build:
Convolutional Neural Network architecture
And here´s a regular ANN with only dense layers in comparison:
The number of parameters is quite comparable. Both were used to classify handwritten digits from the MNIST dataset (60.000 arrays of 28x28). Indepently of my computing power the training of the CNN is waayyyyy slower. ANN for 250 Epochs on 60.000 images: 0:05 h CNN for 20 Epochs on 30.000 images: 12:00 h
So the question is why? My Guess is because of the parameter sharing of CNNs, the amount of parameters might be similiar but a parameter of the convolution layer is used multiple times:
3x3 kernel on a 28x28 image -> each parameter is used approx 784 times (same padding) using 32 kernels, that yields a 28x28x32 output (after pooling 14x14x32). So the kernel for the next convolutional layer is 3x3x32, so here each parameter will be used approx 6.272 times.
Can someone with more understanding of that whole training process agree or disagree with me? Or do you have any other ideas on that topic?