Comes the morning and the headlights fade away. | ||
| --The Living Daylights (1987) | ||
James Bond is a car thief. The movies show that he has stolen many automobiles, often from innocent bystanders. We do not know whether these unfortunate people ever recovered their property, but even if they did, the damages from collisions, submersions, bullets, and rockets would have had a lasting impact on their insurance premiums. Bond has also stolen a propeller plane, a tank, and a moon buggy.
The man has been driving since the 1950s, and perhaps it is time that he stopped.
Be that as it may, we can break away from the bad, old Cold War days of indifference to collateral damage. With modern technology, we can provide a driver with timely information about others who are sharing the road. This information might make it easier to avoid collisions and to properly aim the vehicle's rocket launchers such that a chase scene can be conducted in an orderly manner, without flattening whole city blocks. Secret agents will not lose so many cars and, thus, will not feel compelled to steal so many.
Since driver assistance is a broad topic, let's focus on one scenario. Twilight and nighttime are difficult times for drivers, including secret agents. We might be blinded by the lack of natural light or the glare of headlights. However, we can make a computer vision system that sees headlights (or rear lights) clearly and can estimate the distance to them. It can also distinguish between lights of different colors, a feature that is relevant to identifying signals and types of vehicles. We will choose computationally inexpensive techniques, suitable for a low-powered computer—namely, Raspberry Pi—that we can plug into a car's cigarette lighter receptacle via an adapter. An LCD panel can display the relevant information along with a live rearview video feed that is less glaring than the real headlights.
This project presents us with several new topics and challenges, which are as follows:
- Detecting blobs of light and classifying their color
- Estimating the distance from the camera to a detected object whose real-world size is known
- Setting up a low-budget lab where we can experiment with lights of many colors
- Setting up a Raspberry Pi and peripherals in a car
Realistically, our quick, homemade project is not sufficiently robust to be relied upon as an automotive safety tool, so take it with a grain of salt. However, it is a fun introduction to analyzing signal lights and wiring up a custom in-car computer. The choice of Raspberry Pi as a platform challenges us to think about the car as an environment for rapid prototyping. We can plug in any standard peripherals including a webcam, keyboard, mouse, and even a monitor, which will give us a complete desktop Linux system with Python—on wheels! (Snakes in a car!) For more exotic projects, Pi is compatible with many electronics kits too! A smartphone or tablet is also a good alternative for use in a car—and is easier to power than a Pi with a monitor—but the Pi excels as a well-rounded prototyping tool.
Let the app be known as The Living Headlights.
Note
The completed project for this chapter can be downloaded from my website at http://nummist.com/opencv/7376_05.zip.
For this app, we will return to the cross-platform wxPython framework. Optionally, we can develop and test our wxPython application on a Windows, Mac, or Linux desktop or laptop before deploying it to our Raspberry Pi computer in the car. With the Raspbian operating system, Raspberry Pi can run wxPython just as any Linux desktop could.
The GUI for The Living Headlights includes a live video feed, a set of controls where the user can enter the true distance to the currently imaged headlights, and a label that initially displays a set of instructions, as seen in the following screenshot:
When a pair of headlights is detected, the user must perform a one-time calibration step. This consists of entering the true distance between the camera and headlights (specifically, the midpoint between the headlights) and then clicking on the Calibrate button. Thereafter, the app continuously updates and displays an estimate of the headlights' distance and color, as seen in the label at the bottom of the following screenshot:
The calibration and the selected unit (meters or feet) are stored in a configuration file when the app closes. They are reloaded from this file when the app reopens.
Atop the video feed, colored circles are drawn to mark detected lights, and lines are drawn between pairs of detected lights whose colors match. Such a pair is considered to be a set of headlights.
Next, let's consider the techniques we will use to detect lights and classify their colors.