Mobile devices running Google's Android outnumber all other mobile OSes and, in recent years, they have featured incredible computing power alongside high-quality cameras, which allows them to perform computer vision at the highest levels. One of the most sought after applications for mobile computer vision is augmented reality (AR). Blending real and virtual worlds has applications in entertainment and gaming, medicinel and healthcare, industry and defense, and many more. The world of mobile AR is advancing quickly, with new compelling demos popping up daily, and it is undeniably an engine for mobile hardware and software development. In this chapter, we will learn how to implement an AR application from scratch in the Android ecosystem, by using OpenCV's ArUco contrib module, Android's Camera2 APIs, as well as the jMonkeyEngine 3D game engine. However, first we will begin with simply calibrating our Android device's camera using ArUco's ChArUco calibration board, which provides a more robust alternative to OpenCV's calib3d chessboards.
The following topics will be covered in this chapter:
- Introduction to light theory of camera intrinsic parameters and calibration process
- Implementing camera calibration in Android using Camera2 APIs and ArUco
- Implementing a see-through AR world with jMonkeyEngine and ArUco markers