Geometrical transformation

It is important that the faces are all aligned together, otherwise the face recognition algorithm might be comparing part of a nose with part of an eye, and so on. The output of the face detection we've just seen will give aligned faces to some extent, but it is not very accurate (that is, the face rectangle will not always be starting from the same point on the forehead).

To have better alignment, we will use eye detection to align the face, so the positions of the two detected eyes line up perfectly in the desired positions. We will do the geometrical transformation using the warpAffine() function, which is a single operation that will do the following four things:

• Rotate the face so that the two eyes are horizontal
• Scale the face so that the distance between the two eyes is always the same
• Translate the face so that the eyes are always centered horizontally, and at the desired height
• Crop the outer parts of the face, since we want to crop away the image background, hair, forehead, ears, and chin

Affine warping takes an affine matrix that transforms the two detected eye locations into the two desired eye locations, and then crops to a desired size and position. To generate this affine matrix, we will get the center between the eyes, calculate the angle at which the two detected eyes appear, and look at their distance apart, as follows:

    // Get the center between the 2 eyes. 
    Point2f eyesCenter; 
    eyesCenter.x = (leftEye.x + rightEye.x) * 0.5f; 
    eyesCenter.y = (leftEye.y + rightEye.y) * 0.5f; 

    // Get the angle between the 2 eyes. 
    double dy = (rightEye.y - leftEye.y); 
    double dx = (rightEye.x - leftEye.x); 
    double len = sqrt(dx*dx + dy*dy); 

    // Convert Radians to Degrees. 
    double angle = atan2(dy, dx) * 180.0/CV_PI; 

    // Hand measurements shown that the left eye center should  
    // ideally be roughly at (0.16, 0.14) of a scaled face image. 
    const double DESIRED_LEFT_EYE_X = 0.16; 
    const double DESIRED_RIGHT_EYE_X = (1.0f - 0.16); 

    // Get the amount we need to scale the image to be the desired 
    // fixed size we want. 
    const int DESIRED_FACE_WIDTH = 70; 
    const int DESIRED_FACE_HEIGHT = 70; 
    double desiredLen = (DESIRED_RIGHT_EYE_X - 0.16); 
    double scale = desiredLen * DESIRED_FACE_WIDTH / len;

Now, we can transform the face (rotate, scale, and translate) to get the two detected eyes to be in the desired eye positions in an ideal face, as follows:

    // Get the transformation matrix for the desired angle & size. 
    Mat rot_mat = getRotationMatrix2D(eyesCenter, angle, scale); 
    // Shift the center of the eyes to be the desired center. 
    double ex = DESIRED_FACE_WIDTH * 0.5f - eyesCenter.x; 
    double ey = DESIRED_FACE_HEIGHT * DESIRED_LEFT_EYE_Y -  
      eyesCenter.y; 
    rot_mat.at<double>(0, 2) += ex; 
    rot_mat.at<double>(1, 2) += ey; 
    // Transform the face image to the desired angle & size & 
    // position! Also clear the transformed image background to a  
    // default grey. 
    Mat warped = Mat(DESIRED_FACE_HEIGHT, DESIRED_FACE_WIDTH, 
      CV_8U, Scalar(128)); 
    warpAffine(gray, warped, rot_mat, warped.size());