Tracking back and forth gestures

Several common gestures consist of a repetitive, back and forth movement. Consider the following examples of this type of gesture:

• Nodding (yes or I'm listening)
• Shaking one's head (no or dismay)
• Waving (a greeting)
• Shaking hands (a greeting)
• Shaking one's fist (a threat or a protest)
• Wagging a finger (scolding)
• Wiggling a finger or fingers (beckoning)
• Tapping one's foot against the ground (impatience)
• Tapping four fingers against a table (impatience)
• Tapping two fingers against a table ("Thanks for the green tea")
• Pacing (anxiety)
• Jumping up and down (excitement, joy)

To help us recognize such gestures, let's write a class, BackAndForthGesture, which keeps track of the number of times that a value (such as an x coordinate or a y coordinate) has oscillated between a low threshold and a high threshold. A certain number of oscillations can be considered as a complete gesture.

Create a file, src/BackAndForthGesture.java. As a member variable, BackAndForthGesture will store the minimum distance or threshold that defines a back or forth motion, an initial position, the latest delta from this position, and counts of the number of back movements and forth movements. Here is the first part of the class's code:

package com.nummist.goldgesture;

public final class BackAndForthGesture {

  private double mMinDistance;

  private double mStartPosition;
  private double mDelta;

  private int mBackCount;
  private int mForthCount;

The back and forth count (or number of oscillations) is the lesser of the back count and the forth count. Let's implement this rule in the following getter method:

  public int getBackAndForthCount() {
    return Math.min(mBackCount, mForthCount);
  }

The constructor takes one argument, the minimum distance or the threshold of movement:

  public BackAndForthGesture(final double minDistance) {
    mMinDistance = minDistance;
  }

To begin tracking movement, we will call a start method with an initial position as an argument. This method records the initial position and resets the delta and counts:

  public void start(final double position) {
    mStartPosition = position;
    mDelta = 0.0;
    mBackCount = 0;
    mForthCount = 0;
  }

Note

Note that we are considering position as a one-dimensional value because a head nodding (up and down) or shaking (left and right) is a linear gesture. For an upright head, only one of the image's two dimensions is relevant to a nod or shake gesture.

To continue tracking movement, we will call an update method with the new position as an argument. This method recalculates the delta and, if a threshold has just been passed, the back count or the forth count is incremented:

  public void update(final double position) {
    double lastDelta = mDelta;
    mDelta = position - mStartPosition;
    if (lastDelta < mMinDistance &&
        mDelta >= mMinDistance) {
      mForthCount++;
    } else if (lastDelta > -mMinDistance &&
        mDelta < -mMinDistance) {
      mBackCount++;
    }
  }

If we consider the gesture complete, or for some other reason we believe the counts to be invalid, we will call a resetCounts method:

  public void resetCounts() {
    mBackCount = 0;
    mForthCount = 0;
  }
}

Note that BackAndForthGesture contains no computer vision functionality of its own, but the position values we pass to it will be derived from computer vision.