Why is this image (image of the number 4) not classified correctly with this simple neural network in MATLAB?

Viewed 119

First year CS student here. I am preparing for a bigger project by trying to understand a simple MATLAB example with a convolutional neural network.

The MATLAB example (about Image Recognition) I mentioned can be found here: https://nl.mathworks.com/help/deeplearning/ug/create-simple-deep-learning-network-for-classification.html

I have chosen two images to classify:

  • image3007.png
  • image3007saveas.png

image3007.png is an image I copied from the folder where the pictures of numbers are stored that were used to train/validate/test the network. So I would expect that it would be classified correctly (and it is).

image3007saveas.png was created by me, by simply opening image3007.png in software (I tried MS paint first, and then an online photoshop clone called photopea.com) and saving it with a different name. Again, I would expact it to be classified correctly (but it is not, you will see below).

image3007.png looks like this: enter image description here

image3007saveas.png looks like this: enter image description here

Of course both images look almost exactly the same (image3007saveas.png is slightly darker).

Both images have the size 28 x 28 x 1.

My script:

function cnn_letter_recognition

digitDatasetPath = fullfile(matlabroot,'toolbox','nnet','nndemos', ...
    'nndatasets','DigitDataset');
imds = imageDatastore(digitDatasetPath, ...
    'IncludeSubfolders',true,'LabelSource','foldernames');
%------------
fracTrainFiles = 0.6;
fracValFiles = 0.2;
fracTestFiles = 0.2;
[imdsTrain, imdsValidation, imdsTest ] = splitEachLabel(imds, ...
    fracTrainFiles, fracValFiles, fracTestFiles, 'randomize');
%------------
layers = [

    imageInputLayer([28 28 1])

    convolution2dLayer(3,10,'Padding','same')
    batchNormalizationLayer 
    reluLayer 

    maxPooling2dLayer(2,'Stride',2)

    convolution2dLayer(3,10,'Padding','same')
    batchNormalizationLayer
    reluLayer
    
    fullyConnectedLayer(10)
    softmaxLayer 
    classificationLayer];
%------------
options = trainingOptions('sgdm', ...
    'InitialLearnRate',0.01, ...
    'MaxEpochs',10, ...
    'Shuffle','every-epoch', ...
    'ValidationData',imdsValidation, ...
    'ValidationFrequency',30, ...
    'ValidationPatience', 5, ...
    'Verbose',false, ...
    'Plots','training-progress');
%------------
net = trainNetwork(imdsTrain,layers,options);
%------------
YPred = classify(net,imdsTest);
YTest = imdsTest.Labels;

accuracy = sum(YPred == YTest)/numel(YTest)
%------------
im1 = imread('image3007.png');
YPred_Should_Be_4 = predict(net,im1)

im2 = imread('image3007saveas.png');
YPred_Should_Be_4_too = predict(net,im2)

The result I get is this:

accuracy =

    0.9900


YPred_Should_Be_4 =

  1×10 single row vector

    0.0000    0.0001    0.0000    0.0000    0.9987    0.0000    0.0000    0.0000    0.0000    0.0012


YPred_Should_Be_4_too =

  1×10 single row vector

    0.0006    0.9474    0.0182    0.0044    0.0022    0.0041    0.0168    0.0011    0.0008    0.0043

Which means:

  • image3007.png is classified correctly as the number 4.
  • image3007saveas.png is classified as the number 1, although the image itself is almost identical to the other image.

im1 (image3007.png) in the workspace:

enter image description here

im2 (image3007saveas.png) in the workspace:

enter image description here

I tried increasing the brightness of image3007saveas.png to make it more similar to the original image. I tried to set epoch to 100 at the options in the code to make the training last longer. The outcome did not change.

0 Answers
Related