In the previous chapter we have used a PIR sensor to sense any human activity, however the problem with the sensor is, that no matter who comes or leaves it would deliver the same message. That basically means that even when you come home after a long day, it would end up asking the same question. Pretty dumb huh?

So in this chapter we would use the previous repository and integrate the vision and the voice together to make an amazing duo. In this, the camera would identify who is on the gate and would recognize if it is a human and a stranger, if so then, it would deliver the message you intend to give. On the other hand if its you then it would simply let you pass with a simple greeting. However if the face is detected but not recognized then it would give a set of instructions to the person standing in-front of the camera.

To implement it all you need to do is to set up a camera on the gate of your door along with the PIR. The PIR is basically to activate the camera. In other words the camera would not get activated till the time no movement is detected. This set up is very straight forward and does not need any GPIO to be used. Simply fix the camera and PIR and upload the following code:

import RPi.GPIO as GPIO
import time
Import os
import cv2
import numpy as np
import cv2

faceDetect = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
cam = cv2.VideoCapture(0)
rec = cv2.face.LBPHFaceRecognizer_create()
rec.read("recognizer/trainningData.yml")
id = 0

while True:

  GPIO.setmode(GPIO.BCM)
PIR = 13
GPIO.setup(PIR, GPIO.IN)

if GPIO.input(PIR) == 1:

  ret, img = cam.read()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
faces = faceDetect.detectMultiScale(gray, 1.3, 5)
for (x, y, w, h) in faces:
  cv2.rectangle(img, (x, y), (x + w, y + h), (0, 0, 255), 2)
id, conf = rec.predict(gray[y: y + h, x: x + w])

if id == 1:
  id = "BEN"
os.system('echo "Hello, welcome to the house BEN"|festival --tts ')
time, sleep(0.2)

else :

  os.system('echo "If you are a delivery agent then please leave the package here"|festival --tts ')
time, sleep(0.2)

os.system('echo "If you are a guest then I'
    m sorry I have to leave I will be back after 7 pm "|festival --tts ')
    time, sleep(0.2)

    os.system('echo "also Kindly don'
      t step over the grass, its freshly grown and needs some time "|festival --tts ')
      time.sleep(1)

      os.system('echo "Thank you !"|festival --tts ') cv2.imshow("face", img) if cv2.waitKey(1) == ord('q'):
      break cam.release()

      cv2.destroyAllWindows()

faceDetect = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

In the preceding code, we are creating a cascade classifier using the method CascadeClassifier so that faces can be detected by the camera.

cam = cv2.VideoCapture(0)
rec = cv2.face.LBPHFaceRecognizer_create()

In the preceding code, we are reading the frames from the camera using VideoCapture(0) method of cv2. Also, the face recognizer is being created to recognize a particular face.

 ret, img = cam.read()

Now read the data from the camera using cam.read() as done in the previous code.

gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
faces = faceDetect.detectMultiScale(gray,1.3,5)

The images are converted into gray color. Then, faceDetect.detectMultiScale() will be using the gray color-converted images.

 for (x,y,w,h) in faces:
     cv2.rectangle(img, (x,y), (x+w, y+h), (0,0,255), 2)
     id, conf = rec.predict(gray[y:y+h, x:x+w])
     if id==1:
         id = "BEN" 
         os.system('echo "Hello, welcome to my house BEN"|festival --tts ')
         time, sleep(0.2)

As the face is detected, the part of the image containing the face will be converted into gray and passed to a predict function. This method will tell if the face is known or not, it also returns the ID if the face is identified. Suppose the person is BEN, then Jarvis would say Hello, welcome to my house BEN. Now BEN can tell the Jarvis to turn on the lights, and the Jarvis would respond as the wake word Jarvis gets activated. And if the person is not recognized, then maybe it was a delivery boy. Then, the following commands get executed:

os.system('echo "If you are a delivery agent then please leave the package here"|festival --tts ')
time, sleep(0.2)

os.system('echo "If you are a guest then I'm sorry I have to leave I will be back after 7pm"|festival --tts ')
 time, sleep(0.2)

os.system('echo "also Kindly don't step over the grass, its freshly grown and needs some time"|festival --tts ')
time.sleep(1)

os.system('echo "Thank you !"|festival --tts ')