Face tracking with the dlib DCF-based tracker

In the face_tracking_correlation_filters.py script, we perform face tracking using the dlib frontal face detector for initialization and the dlib DCF-based tracker DSST for face tracking. In order to initialize the correlation tracker, we execute the following command:

tracker = dlib.correlation_tracker()

This initializes the tracker with default values (filter_size = 6,num_scale_levels = 5, scale_window_size = 23, regularizer_space = 0.001, nu_space = 0.025, regularizer_scale = 0.001, nu_scale = 0.025, and scale_pyramid_alpha = 1.020). A higher value of filter_size and num_scale_levels increases tracking accuracy, but it requires more computational power, increasing CPU processing. The recommended values for filter_size are 5, 6, and 7, and for num_scale_levels, 4, 5, and 6. 

To begin tracking the method, tracker.start_track() is used. In this case, we perform face detection. If successful, we will pass the position of the face to this method, as follows:

if tracking_face is False:
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    # Try to detect a face to initialize the tracker:
    rects = detector(gray, 0)
    # Check if we can start tracking (if we detected a face):
    if len(rects) > 0:
        # Start tracking:
        tracker.start_track(frame, rects[0])
        tracking_face = True

This way, the object tracker will start tracking what is inside the bounding box, which, in this case, is the detected face.

Additionally, to update the position of the tracked object, the tracker.update() method is called:

tracker.update(frame)

This method updates the tracker and returns the peak-to-side-lobe ratio, which is a metric that measures how confident the tracker is. Larger values of this metric indicate high confidence. This metric can be used to reinitialize the tracker with frontal face detection.

To get the position of the tracked object, the tracker.get_position() method is called:

pos = tracker.get_position()

This method returns the position of the object being tracked. Finally, we can draw the predicted position of the face:

cv2.rectangle(frame, (int(pos.left()), int(pos.top())), (int(pos.right()), int(pos.bottom())), (0, 255, 0), 3)

In this script, we coded the option to reinitialize the tracker if the number 1 is pressed. If this number is pressed, we reinitialize the tracker trying to detect a frontal face. To clarify how this script works, the following two screenshots are included.

In the first screenshot, the tracking algorithm is waiting until a frontal face detection is performed to initialize the tracking:

In the second screenshot, the tracking algorithm is currently tracking a previously detected face:

In the previous screenshot you can see that the algorithm is currently tracking the detected face. You can also see that you can also press the number 1 in order to re-initialize the tracking.