count Yaw and Roll in Python Head pose estimation MTCNN

user_15394203 0
Viewed 1147

I have encountered implementation of MTCNN network which is able to detect our head movement in 3 axis called Yaw,Roll and Pitch.

Here are crucial elements:

bbox, points = detector.detect_faces(frame)

points gives me keypoints list from network preduction such as:

enter image description here

Drawing function:

def draw_landmarks(frame, bb, points):
    # draw rectangle and landmarks on face
    cv2.rectangle(frame,(int(bb[0]),int(bb[1])),(int(bb[2]),int(bb[3])),green,2)
    cv2.circle(frame, (points[0], points[5]), 2, (255,0,0), 2)# eye
    cv2.circle(frame, (points[1], points[6]), 2, (255,0,0), 2)
    cv2.circle(frame, (points[2], points[7]), 2, (255,0,0), 2)# nose
    cv2.circle(frame, (points[3], points[8]), 2, (255,0,0), 2)# mouth
    cv2.circle(frame, (points[4], points[9]), 2, (255,0,0), 2)

Now to detect some head movement in different axis there was defined Yaw and Roll functions.

def Roll(points):
    return points[6] - points[5]

def Yaw(points):
    le2n = points[2] - points[0]
    re2n = points[1] - points[2]
    return le2n - re2n

Now having said that:

"Roll is the rotation about the x axis (between -180 and 180 deg)
Yaw is the rotation about the z axis (between -180 and 180)."

How does exactly Yaw function work here?
Github: https://github.com/fisakhan/Face_Pose/blob/master/pose_detection_mtcnn.py

neural-network opencv python
1 Answers

Yaw, Roll and Pitch are Euler angles - the image below shows a more easy to understand example, it is important to note that the rotations are not in relation to the global axis but are in fact in relation to objects axis that's why a plane is a good thing to think about. Also there are several different formats of Euler angles to understand better look at the wiki

looking at the github link you provided I have found the following:

  1. the points contain the coordinates of different facial features within the frame, where:

    X=points[0:5] Y=points[5:10]

  2. they do not measure these angles in degrees:

Roll: -x to x (0 is frontal, positive is clock-wise, negative is anti-clock-wise)

Yaw: -x to x (0 is frontal, positive is looking right, negative is looking left)

Pitch: 0 to 4 (0 is looking upward, 1 is looking straight, >1 is looking downward)

  1. the function for Yaw, Roll and Pitch do not return the angle:

what is returned from Roll is the Y coordinate of the left eye minus the y coordinate of the right eye

Yaw is essentially calculating which eye the noise is closer to along the x axis - as you turn your head the nose appears closer to one eye from an observer

  1. find_pose might have what you are looking for but I need to do further research into what is meant by xfrontal and yfrontal - you may need to pose the question directly to the person on github

http://www.chrobotics.com/library/understanding-euler-angles

Update: after posing the question directly to the developer, they have responded with the following:

Roll, Yaw and Pitch are in pixels and provide an estimate of where the face looking at. In case of mtcnn Roll is (-50 to 50), Yaw is (-100 to 100). Pitch is 0 to 4 because you can divide the distance between eyes and lips into 4 units where one unit is between lips to nose-tip and 3 units between nose-tip to eyes.

Xfrontal and Yfrontal provide pose (Yaw and Pitch only) in terms of angles (in degrees) along X and Y axis, respectively. These values are obtained after compensating the roll (aligning both the eyes horizontally).

https://github.com/fisakhan/Face_Pose/issues/2

user_134672240
Related