In Chapter 1 we saw something of OpenCV's past. This was followed by Chapters Chapter 2–Chapter 13, in which OpenCV's present state was explored in detail. We now turn to OpenCV's future. Computer vision applications are growing rapidly, from product inspection to image and video indexing on the Web to medical applications and even to local navigation on Mars. OpenCV is also growing to accommodate these developments.
OpenCV has long received support from Intel Corporation and has more recently received support from Willow Garage (www.willowgarage.com), a privately funded new robotics research institute and technology incubator. Willow Garage's intent is to jump-start civilian robotics by developing open and supported hardware and software infra-structure that now includes but goes beyond OpenCV. This has given OpenCV new resources for more rapid update and support, with several of the original developers of OpenCV now recontracted to help maintain and advance the library. These renewed resources are also intended to support and enable greater community contribution to OpenCV by allowing for faster code assessment and integration cycles.
One of the key new development areas for OpenCV is robotic perception. This effort focuses on 3D perception as well as 2D plus 3D object recognition since the combination of data types makes for better features for use in object detection, segmentation and recognition. Robotic perception relies heavily on 3D sensing, so efforts are under way to extend camera calibration, rectification and correspondence to multiple cameras and to camera + laser rangefinder combinations (see Figure 14-1).[268]
Should commercially available hardware warrant it, the "laser + camera calibration" effort will be generalized to include devices such as flash LIDAR and infrared wavefront devices. Additional efforts are aimed at developing triangulation with structured or laser light for extremely accurate depth sensing. The raw output of most depth-sensing methods is in the form of a 3D point cloud. Complementary efforts are thus planned to support turning the raw point clouds resulting from 3D depth perception into 3D meshes. 3D meshes will allow for 3D model capture of objects in the environment, segmenting objects in 3D and hence the ability for robots to grasp and manipulate such objects. Three-dimensional mesh generation can also be used to allow robots to move seamlessly from external 3D perception to internal 3D graphics representation for planning and then back out again for object registration, manipulation, and movement.
Figure 14-1. New 3D imager combinations: calibrating a camera (left) with the brightness return from a laser depth scanner (right). (Images courtesy of Hai Nguyen and Willow Garage)
Along with sensing 3D objects, robots will need to recognize 3D objects and their 3D poses. To support this, several scalable methods of 2D plus 3D object recognition are being pursued. Creating capable robots subsumes most fields of computer vision and artificial intelligence, from accurate 3D reconstruction to tracking, identifying humans, object recognition, and image stitching and on to learning, control, planning, and decision making. Any higher-level task, such as planning, is made much easier by rapid and accurate depth perception and recognition. It is in these areas especially that OpenCV hopes to enable rapid advance by encouraging many groups to contribute and use ever better methods to solve the difficult problems of real-world perception, recognition, and learning.
OpenCV will, of course, support many other areas as well, from image and movie indexing on the web to security systems and medical analysis. The wishes of the general community will heavily influence OpenCV's direction and growth.
[268] At the time of this writing, these methods remain under development and are not yet in OpenCV.