Face landmark detection is the process of finding points of interest in an image of a human face. It recently saw a spur of interest in the computer vision community, as it has many compelling applications; for example, detecting emotion through facial gestures, estimating gaze direction, changing facial appearance (face swap), augmenting faces with graphics, and puppeteering of virtual characters. We can see many of these applications in today's smartphones and PC web-camera programs. To achieve these applications, the landmark detector must find dozens of points on the face, such as corners of the mouth, corners of eyes, the silhouette of the jaws, and many more. To that end, many algorithms were developed, and a few were implemented in OpenCV. In this chapter, we will discuss the process of face landmark (also known as facemark) detection using the cv::face module, which provides an API for inference, as well as training of a facemark detector. We will see how to apply the facemark detector to locating the direction of the face in 3D.
The following topics will be covered in this chapter:
- Introducing face landmark detection history and theory, and an explanation of the algorithms implemented in OpenCV
- Utilizing OpenCV's face module for face landmark detection
- Estimating the approximate direction of the face by leveraging 2D–3D information