Implementing the Angora Blue app

The Angora Blue app uses three new files: GeomUtils.py, MailUtils.py, and AngoraBlue.py, which should all be in our project's top folder. Given the app's dependencies on our previous work, the following files are relevant to Angora Blue:

First, let's create GeomUtils.py. This does not need any import statements. Let's add the following intersects function, which accepts two rectangles as arguments and returns either True (if they intersect) or False (otherwise):

def intersects(rect0, rect1):
  x0, y0, w0, h0 = rect0
  x1, y1, w1, h1 = rect1
  if x0 > x1 + w1: # rect0 is wholly to right of rect1
    return False
  if x1 > x0 + w0: # rect1 is wholly to right of rect0
    return False
  if y0 > y1 + h1: # rect0 is wholly below rect1
    return False
  if y1 > y0 + h0: # rect1 is wholly below rect0
    return False
  return True

Using the intersects function let's write the following difference function, which accepts two lists of rectangles, rects0 and rects1, and returns a new list that contains the rectangles in rects0 that do not intersect with any rectangle in rects1:

def difference(rects0, rects1):
  result = []
  for rect0 in rects0:
    anyIntersects = False
    for rect1 in rects1:
      if intersects(rect0, rect1):
        anyIntersects = True
        break
      if not anyIntersects:
        result += [rect0]
  return result

Later, we will use the difference function to filter out cat faces that intersect with human faces.

Now, let's create MailUtils.py. This needs the following import statement:

import smtplib

For the task of sending an e-mail, let's copy the following function from Rosetta Code, a free wiki that offers utility functions in many programming languages:

def sendEmail(fromAddr, toAddrList, ccAddrList, subject, message,
  login, password, smtpServer='smtp.gmail.com:587'):

  # Taken from http://rosettacode.org/wiki/Send_an_email#Python

  header = 'From: %s\n' % fromAddr
  header += 'To: %s\n' % ','.join(toAddrList)
  header += 'Cc: %s\n' % ','.join(ccAddrList)
  header += 'Subject: %s\n\n' % subject
  message = header + message

  server = smtplib.SMTP(smtpServer)
  server.starttls()
  server.login(login,password)
  problems = server.sendmail(fromAddr, toAddrList, message)
  server.quit()
  return problems

By default, the sendEmail function uses Gmail. By specifying the optional smtpServer argument, we can use a different service.

Tip

Since July 2014, the default security settings on Google accounts require apps to use not only SMTP authentication, but also OAuth authentication in order to send an e-mail via Gmail. Our sendEmail function uses a secure TLS connection, but handles SMTP authentication only (as this is sufficient for most e-mail services other than Gmail). To reconfigure your Google account for compatibility with our function, log in to your account, go to https://www.google.com/settings/security/lesssecureapps, select the Enable option, and click Done. For the best security, you might wish to create a dummy Google account for this project and apply the custom security setting to this dummy account only. Alternatively, most e-mail services besides Gmail should not require special configuration.

Now, we are ready to implement AngoraBlue.py. This starts with the following imports:

import cv2
import numpy # Hint to PyInstaller
import os
import socket
import sys

import BinasciiUtils
import GeomUtils
import MailUtils
import PyInstallerUtils
import ResizeUtils

Angora Blue simply uses a main function and one helper function, recognizeAndReport. This helper function begins as follows, by iterating over a given list of face rectangles and using a given recognizer (be it a human recognizer or a cat recognizer) to get a label and distance (non-confidence) for each face:

def recognizeAndReport(recognizer, grayImage, rects, maxDistance,
  noun='human'):
  for x, y, w, h in rects:
    crop = cv2.equalizeHist(grayImage[y:y+h, x:x+w])
    labelAsInt, distance = recognizer.predict(crop)
    labelAsStr = BinasciiUtils.intToFourChars(labelAsInt)

For testing, it is useful to log the recognition results here. However, we will comment out the logging in the final version, as follows:

    #print noun, labelAsStr, distance

If any of the faces is recognized with a certain level of confidence (based on a maxDistance argument), we will attempt to send an e-mail alert. If the alert is sent successfully, the function returns True, meaning it did recognize and report a face. Otherwise, it returns False. Here is the remainder of the implementation:

    if distance <= maxDistance:
      fromAddr = 'username@gmail.com' # TODO: Replace
      toAddrList = ['username@gmail.com'] # TODO: Replace
      ccAddrList = []
      subject = 'Angora Blue'
      message = 'We have sighted the %s known as %s.' % \
        (noun, labelAsStr)
      login = 'username' # TODO: Replace
      password = 'password' # TODO: Replace
      # TODO: Replace if not using Gmail.
      smtpServer='smtp.gmail.com:587'
    try:
      problems = MailUtils.sendEmail(
        fromAddr, toAddrList, ccAddrList, subject,
        message, login, password, smtpServer)
      if problems:
        print >> sys.stderr, 'Email problems:', problems
      else:
        return True
      except socket.gaierror:
    print >> sys.stderr, 'Unable to reach email server'
  return False

The main function starts by defining paths to the detection and recognition models. If either recognition model does not exist (because it has not been trained), we will print an error and exit, as follows:

def main():

  humanCascadePath = PyInstallerUtils.resourcePath(
    # Uncomment the next argument for LBP.
    #'cascades/lbpcascade_frontalface.xml')
    # Uncomment the next argument for Haar.
    'cascades/haarcascade_frontalface_alt.xml')
  humanRecognizerPath = PyInstallerUtils.resourcePath(
    'recognizers/lbph_human_faces.xml')
  if not os.path.isfile(humanRecognizerPath):
    print >> sys.stderr, \
    'Human face recognizer not trained. Exiting.'
    return
  catCascadePath = PyInstallerUtils.resourcePath(
    # Uncomment the next argument for LBP.
    #'cascades/lbpcascade_frontalcatface.xml')
    # Uncomment the next argument for Haar with basic
    # features.
    'cascades/haarcascade_frontalcatface.xml')
    # Uncomment the next argument for Haar with extended
    # features.
    #'cascades/haarcascade_frontalcatface_extended.xml')
  catRecognizerPath = PyInstallerUtils.resourcePath(
    'recognizers/lbph_cat_faces.xml')
  if not os.path.isfile(catRecognizerPath):
    print >> sys.stderr, \
      'Cat face recognizer not trained. Exiting.'
    return

As in Interactive Recognizer, we will start capturing video from a camera and we will store the video's resolution in order to calculate the relative, minimum size of a face. Here is the relevant code:

  capture = cv2.VideoCapture(0)
  imageWidth, imageHeight = \
    ResizeUtils.cvResizeCapture(capture, (1280, 720))
  minImageSize = min(imageWidth, imageHeight)

We will load detectors and recognizers from the file and set a minimum face size for detection and maximum distance (non-confidence) for recognition. We will specify the values separately for human and feline subjects. You might need to tweak the values based on your particular camera setup and models. The code proceeds as follows:

  humanDetector = cv2.CascadeClassifier(humanCascadePath)
  humanRecognizer = cv2.createLBPHFaceRecognizer()
  humanRecognizer.load(humanRecognizerPath)
  humanMinSize = (int(minImageSize * 0.25),
    int(minImageSize * 0.25))
  humanMaxDistance = 10

  catDetector = cv2.CascadeClassifier(catCascadePath)
  catRecognizer = cv2.createLBPHFaceRecognizer()
  catRecognizer.load(catRecognizerPath)
  catMinSize = humanMinSize
  catMaxDistance = 10

We will read frames from the camera continuously, until an e-mail alert is sent as a result of face recognition. Each frame is converted to grayscale and is equalized. Next, we will detect and recognize human faces and possibly send an alert, as follows:

  while True:
    success, image = capture.read()
    if image is not None:
      grayImage = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
      equalizedGrayImage = cv2.equalizeHist(grayImage)

      humanRects = humanDetector.detectMultiScale(
        equalizedGrayImage, scaleFactor=1.3,
        minNeighbors=4, minSize=humanMinSize,
        flags=cv2.cv.CV_HAAR_SCALE_IMAGE)
      if recognizeAndReport(
        humanRecognizer, grayImage, humanRects,
        humanMaxDistance, 'human'):
        break

If no alert has been sent, we will continue to perform cat detection and recognition. For cat detection, we will make extra efforts to eliminate false positives by specifying a higher minNeighbors value and by filtering out any cat faces that intersect human faces. Here is this final part of Angora Blue's implementation:

      catRects = catDetector.detectMultiScale(
        equalizedGrayImage, scaleFactor=1.3,
        minNeighbors=8, minSize=catMinSize,
        flags=cv2.cv.CV_HAAR_SCALE_IMAGE)
      # Reject any cat faces that overlap with human faces.
        catRects = GeomUtils.difference(catRects, humanRects)
      if recognizeAndReport(
        catRecognizer, grayImage, catRects,
        catMaxDistance, 'cat'):
      break

if __name__ == '__main__':
  main()

Before testing Angora Blue, ensure that the two recognition models are trained using Interactive Human Face Recognizer and Interactive Cat Face Recognizer. Preferably, each model should contain two or more individuals. Then, set up a computer and webcam in a place where frontal human faces and frontal cat faces will be encountered. Try to get your friends and pets to participate in the following test cases:

Once the recognition model and Angora Blue are tweaked, we are ready to deploy our alarm system to a vast network of webcam-enabled computers! Let the search for the blue-eyed Angora begin!