My model keeps predicting the same face, how can i fix it?

Admin 1
Viewed 118

I trained a model with a CNN and used it with opencv for real-time face recognition on a webcam. I had some problems with the model, like I got 100% accuracy when it was training and I knew something must have went wrong, and another problem is that when it predicts on webcam images it only gives me one label no matter whose face I show: it gives a label of "me" no matter if I show my face or another person's face.

Note, I only have 250 images in the dataset: 200 images for training and 50 images for testing. For each set, I separated them into 2 classes, "me" and "not_me", where the "me" folder contains images of my face and the "not_me" folder contains images of different peoples' faces.

Is the data not enough for the model or something else?

Please help so that I can learn from my mistake. Thanks before hand.

myface_model.py

# Importing the Keras libraries and packages
from keras.models import Sequential
from keras.layers import Conv2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.preprocessing.image import ImageDataGenerator

# Initialising the CNN
classifier = Sequential()

# Step 1 - Convolution
classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))

# Step 2 - Pooling
classifier.add(MaxPooling2D(pool_size = (2, 2)))

# Adding a second convolutional layer
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))

# Step 3 - Flattening
classifier.add(Flatten())

# Step 4 - Full connection
classifier.add(Dense(units = 128, activation = 'relu'))

# output layer
classifier.add(Dense(units = 1, activation = 'sigmoid'))

# Compiling the CNN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

# Part 2 - Fitting the CNN to the images
train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('dataset/training_set',
                                                 target_size = (64, 64),
                                                 batch_size = 32,
                                                 class_mode = 'binary')

test_set = test_datagen.flow_from_directory('dataset/test_set',
                                            target_size = (64, 64),
                                            batch_size = 32,
                                            class_mode = 'binary')

classifier.fit_generator(training_set,
                         steps_per_epoch = 200,
                         epochs = 25,
                         validation_data = test_set,
                         validation_steps = 50)

#save model for later use
fer_json = classifier.to_json()
with open("fer.json", "w") as json_file:
    json_file.write(fer_json)
classifier.save_weights("fer.h5")

myface_detector.py

import os
import cv2
import numpy as np
from keras.models import model_from_json
from keras.preprocessing import image

#load model
classifier = model_from_json(open("fer.json", "r").read())
#load weights
classifier.load_weights('fer.h5')

face_haar_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

cap=cv2.VideoCapture(0)

while True:
    # captures frame and returns boolean value and captured image
    ret,test_img=cap.read()

    if not ret:
        continue

    convt_img= cv2.cvtColor(test_img, cv2.COLOR_BGR2RGB)

    faces_detected = face_haar_cascade.detectMultiScale(convt_img, 1.3, 5)

    for (x,y,w,h) in faces_detected:
        cv2.rectangle(test_img, (x,y), (x+w,y+h), (255,0,0), thickness=4)
        roi = convt_img[y:y+h,x:x+w]
        roi = cv2.resize(roi,(64,64))
        roi = roi.astype("float") / 255.0
        img_pixels = image.img_to_array(roi)
        img_pixels = np.expand_dims(img_pixels, axis = 0)

        predictions = classifier.predict(img_pixels)

        #find max indexed array
        max_index = np.argmax(predictions[0])

        myFace = ('me', 'not_me')

        predicted_myFace = myFace[max_index]

        cv2.putText(test_img, predicted_myFace, (int(x), int(y)), cv2.FONT_HERSHEY_SIMPLEX, 1, (0,0,255), 2)

    cv2.imshow('my face predictions',test_img)

    if cv2.waitKey(10) == ord('q'):#wait until 'q' key is pressed
        break

cap.release()
cv2.destroyAllWindows()
keras tensorflow deep-learning machine-learning python
1 Answers

Binary classes

So the crux of the issue comes from how we choose to represent our model. Specifically in these lines:

classifier.add(Dense(units = 1, activation = 'sigmoid'))

and

training_set = train_datagen.flow_from_directory('dataset/training_set',
                                                 target_size = (64, 64),
                                                 batch_size = 32,
                                                 class_mode = 'binary') #<--------

test_set = test_datagen.flow_from_directory('dataset/test_set',
                                            target_size = (64, 64),
                                            batch_size = 32,
                                            class_mode = 'binary') #<-------------

Here we choose to give our CNN one output node and to have binary class labels. This means that the return value of the predict method will look something like this [[0.12956393]]. This value represents the predicted probability that the test image has a class label 1 (in this case, a label of 'not_me'). To extract the class label, we can simply do:

predictions = classifier.predict(img_pixels)
predicted_label = predictions[0][0] > 0.5
myFace = ('me', 'not_me')
predicted_myFace = myFace[predicted_label]

However, to follow the intent of your original code to choose the class label based on highest probabilities, and to allow us to extend this application to easily predict more classes if needed, we only need to make minor modifications to the model to use categorical class labels.

Categorical classes

Instead of using binary labels, we can use categorical labels. This will allow us to have class labels from 0...n. In this case, we simply have to make the following edits

classifier.add(Dense(units = 2, activation = 'sigmoid')) # note that units = 2 now

and

training_set = train_datagen.flow_from_directory(
    'dataset/training_set',
    target_size = (64, 64),
    batch_size = 32,
    class_mode = 'categorical') # note the new class mode

test_set = test_datagen.flow_from_directory(
    'dataset/test_set',
    target_size = (64, 64),
    batch_size = 32,
    class_mode = 'categorical') # note the new class mode

By giving our CNN two output nodes and changing the class mode from binary to categorical, our CNN is now able to output a predicted probability for each class label. i.e. the return value of the predict method will look something like:

[[0.012211  0.9917401]]

which means that your code in myface_detector.py will work as is.

user_136027500
Related