Our goal in this chapter is to combine image tracking with 3D rendering. We will modify our existing image tracker so that it fully determines the target's position and rotation in 3D. Then, using Android SDK's implementation of OpenGL ES, we will draw a 3D cube sitting atop the tracked image. This is a case of augmented reality (AR), meaning that we are superimposing a virtual object (the cube) on a specific part of a real scene.
The complete Eclipse project for this chapter can be downloaded from my website at http://nummist.com/opencv/5206_05.zip.
For this chapter, we will modify our existing
ImageDetectionFilter class. We will also add files for the following new classes and interfaces:
com.nummist.secondsight.ARCubeRenderer: A class representing the rendering logic for a cube that sits atop a tracked, real-world object. The class implements the GLSurfaceView.Renderer interface from the Android standard library. The projection matrix is determined by a CameraProjectionAdapter instance, and the cube's pose matrix is determined by an ARFilter instance, as described later.com.nummist.secondsight.adapters.CameraProjectionAdapter: A class representing the relationship between a physical camera and a projection matrix. The projection matrix may be fetched in either OpenCV or OpenGL format.com.nummist.secondsight.filters.ar.ARFilter: An interface representing a filter that captures the position and rotation of a real-world object as an OpenGL matrix. We will modify ImageDetectionFilter to implement this interface.com.nummist.secondsight.filters.ar.NoneARFilter: A class representing a filter that does nothing. It extends the NoneFilter class and implements the ARFilter interface. We use NoneARFilter when we want to turn off filtering but still have an object that conforms to the ARFilter interface.Together, these types support the rendering of a virtual 3D environment that is consistent with certain properties of the real video camera and scene.