Adding the tracker filters to CameraActivity

To use instances of ImageDetectionFilter, we make the same kind of modifications to CameraActivity as we did for other filters in the previous chapter. Recall that all our filter classes implement the Filter interface so that CameraActivity can use them all in similar ways.

First, we need to define some text (for the menu button) in res/values/strings.xml:

  <string name="menu_next_image_detection_filter">Next
    Tracker</string>

Next, we need to define the menu button itself in res/menu/activity_camera.xml:

<menu xmlns:android="http://schemas.android.com/apk/res/android">
  <item
    android:id="@+id/menu_next_image_detection_filter"
    android:orderInCategory="100"
    android:showAsAction="ifRoom|withText"
    android:title="@string/menu_next_image_detection_filter" />
  <!-- ... -->
</menu>

The rest of our modifications pertain to CameraActivity.java. We need to add new member variables to keep track of the selected image detection filter:

  // Keys for storing the indices of the active filters.
  private static final String STATE_IMAGE_DETECTION_FILTER_INDEX =
    "imageDetectionFilterIndex";
  private static final String STATE_CURVE_FILTER_INDEX =
    "curveFilterIndex";
  private static final String STATE_MIXER_FILTER_INDEX =
    "mixerFilterIndex";
  private static final String STATE_CONVOLUTION_FILTER_INDEX =
    "convolutionFilterIndex";

  // The filters.
  private Filter[] mImageDetectionFilters;
  private Filter[] mCurveFilters;
  private Filter[] mMixerFilters;
  private Filter[] mConvolutionFilters;

  // The indices of the active filters.
  private int mImageDetectionFilterIndex;
  private int mCurveFilterIndex;
  private int mMixerFilterIndex;
  private int mConvolutionFilterIndex;

Once OpenCV is initialized, we need to instantiate all of the image detection filters and put them in an array. For brevity, I have added just two image detection filters as examples but you can easily modify the following code to support tracking of more images, or different images:

    public void onManagerConnected(final int status) {
      switch (status) {
        case LoaderCallbackInterface.SUCCESS:
          Log.d(TAG, "OpenCV loaded successfully");
          mCameraView.enableView();
          mBgr = new Mat();

          final Filter starryNight;
           try {
             starryNight = new ImageDetectionFilter(
               CameraActivity.this,
                 R.drawable.starry_night);
          } catch (IOException e) {
            Log.e(TAG, "Failed to load drawable: " +
              "starry_night");
            e.printStackTrace();
            break;
          }

          mImageDetectionFilters = new Filter[] {
            new NoneFilter(),
              starryNight,
                akbarHunting
          };

          // ...
      }
    }
  };

When the activity is created, we need to load any saved data about the selected image detection filter:

  protected void onCreate(final Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    final Window window = getWindow();
    window.addFlags(
      WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);

    if (savedInstanceState != null) {
      mCameraIndex = savedInstanceState.getInt(
        STATE_CAMERA_INDEX, 0);
      mImageDetectionFilterIndex = savedInstanceState.getInt(
        STATE_IMAGE_DETECTION_FILTER_INDEX, 0);
      mCurveFilterIndex = savedInstanceState.getInt(
        STATE_CURVE_FILTER_INDEX, 0);
      mMixerFilterIndex = savedInstanceState.getInt(
        STATE_MIXER_FILTER_INDEX, 0);
      mConvolutionFilterIndex = savedInstanceState.getInt(
        STATE_CONVOLUTION_FILTER_INDEX, 0);
    } else {
      mCameraIndex = 0;
      mImageDetectionFilterIndex = 0;
      mCurveFilterIndex = 0;
      mMixerFilterIndex = 0;
      mConvolutionFilterIndex = 0;
    }

    // ...
  }

Conversely, before the activity is destroyed, we need to save data about the selected image detection filter:

  public void onSaveInstanceState(Bundle savedInstanceState) {
    // Save the current camera index.
    savedInstanceState.putInt(STATE_CAMERA_INDEX, mCameraIndex);

    // Save the current filter indices.
    savedInstanceState.putInt(STATE_IMAGE_DETECTION_FILTER_INDEX,
      mImageDetectionFilterIndex);
    savedInstanceState.putInt(STATE_CURVE_FILTER_INDEX,
      mCurveFilterIndex);
    savedInstanceState.putInt(STATE_MIXER_FILTER_INDEX,
      mMixerFilterIndex);
    savedInstanceState.putInt(STATE_CONVOLUTION_FILTER_INDEX,
      mConvolutionFilterIndex);

    super.onSaveInstanceState(savedInstanceState);
  }

When the Next Tracker menu button is pressed, the selected image detection filter needs to be updated:

  public boolean onOptionsItemSelected(final MenuItem item) {
    if (mIsMenuLocked) {
      return true;
    }
    switch (item.getItemId()) {
    case R.id.menu_next_image_detection_filter:
      mImageDetectionFilterIndex++;
      if (mImageDetectionFilterIndex ==
        mImageDetectionFilters.length) {
        mImageDetectionFilterIndex = 0;
      }
      return true;
    // ...
    default:
      return super.onOptionsItemSelected(item);
    }
  }

Finally, when the camera captures a frame, the selected image detection filter needs to be applied to the frame. To ensure that other filters do not interfere with the image detection, it is important to apply the image detection filter first:

  public Mat onCameraFrame(final CvCameraViewFrame inputFrame) {
    final Mat rgba = inputFrame.rgba();

    // Apply the active filters.
    if (mImageDetectionFilters != null) {
      mImageDetectionFilters[mImageDetectionFilterIndex].apply(
        rgba, rgba);
    }
    if (mCurveFilters != null) {
      mCurveFilters[mCurveFilterIndex].apply(rgba, rgba);
    }
    if (mMixerFilters != null) {
      mMixerFilters[mMixerFilterIndex].apply(rgba, rgba);
    }
    if (mConvolutionFilters != null) {
      mConvolutionFilters[mConvolutionFilterIndex].apply(
        rgba, rgba);
    }

    if (mIsPhotoPending) {
      mIsPhotoPending = false;
      takePhoto(rgba);
    }

    if (mIsCameraFrontFacing) {
      // Mirror (horizontally flip) the preview.
      Core.flip(rgba, rgba, 1);
    }

    return rgba;
  }

That's all! Print the target images or display them on-screen. Then, run the app, select an appropriate image detection filter, and point the camera at the target. Depending on your Android device, you might need to hold it still for a second or two in order for the camera to focus on the target. Then, you should see the target outlined in green. For example, see the outline around The Starry Night in the following screenshot:

Adding the tracker filters to CameraActivity