Python applications can be written in a purely procedural style. This is often done with small applications like our basic I/O scripts, discussed previously. However, from now on, we will use an object-oriented style because it promotes modularity and extensibility.
From our overview of OpenCV's I/O functionality, we know that all images are similar, regardless of their source or destination. No matter how we obtain a stream of images or where we send it as output, we can apply the same application-specific logic to each frame in this stream. Separation of I/O code and application code becomes especially convenient in an application like Cameo, which uses multiple I/O streams.
We will create classes called CaptureManager and WindowManager as high-level interfaces to I/O streams. Our application code may use a CaptureManager to read new frames and, optionally, to dispatch each frame to one or more outputs, including a still image file, a video file, and a window (via a WindowManager class). A WindowManager class lets our application code handle a window and events in an object-oriented style.
Both CaptureManager and WindowManager are extensible. We could make implementations that did not rely on OpenCV for I/O. Indeed, Appendix A, Integrating with Pygame uses a WindowManager subclass.
As we have seen, OpenCV can capture, show, and record a stream of images from either a video file or a camera, but there are some special considerations in each case. Our CaptureManager class abstracts some of the differences and provides a higher-level interface for dispatching images from the capture stream to one or more outputs—a still image file, a video file, or a window.
A CaptureManager class is initialized with a VideoCapture class and has the enterFrame() and exitFrame() methods that should typically be called on every iteration of an application's main loop. Between a call to enterFrame() and a call to exitFrame(), the application may (any number of times) set a channel property and get a frame property. The channel property is initially 0 and only multi-head cameras use other values. The frame property is an image corresponding to the current channel's state when enterFrame() was called.
A CaptureManager class also has writeImage(), startWritingVideo(), and stopWritingVideo() methods that may be called at any time. Actual file writing is postponed until exitFrame(). Also during the exitFrame() method, the frame property may be shown in a window, depending on whether the application code provides a WindowManager class either as an argument to the constructor of CaptureManager or by setting a property, previewWindowManager.
If the application code manipulates frame, the manipulations are reflected in any recorded files and in the window. A CaptureManager class has a constructor argument and a property called shouldMirrorPreview, which should be True if we want frame to be mirrored (horizontally flipped) in the window but not in recorded files. Typically, when facing a camera, users prefer the live camera feed to be mirrored.
Recall that a VideoWriter class needs a frame rate, but OpenCV does not provide any way to get an accurate frame rate for a camera. The CaptureManager class works around this limitation by using a frame counter and Python's standard time.time() function to estimate the frame rate if necessary. This approach is not foolproof. Depending on frame rate fluctuations and the system-dependent implementation of time.time(), the accuracy of the estimate might still be poor in some cases. However, if we are deploying to unknown hardware, it is better than just assuming that the user's camera has a particular frame rate.
Let's create a file called managers.py, which will contain our implementation of CaptureManager. The implementation turns out to be quite long. So, we will look at it in several pieces. First, let's add imports, a constructor, and properties, as follows:
import cv2
import numpy
import time
class CaptureManager(object):
def __init__(self, capture, previewWindowManager = None,
shouldMirrorPreview = False):
self.previewWindowManager = previewWindowManager
self.shouldMirrorPreview = shouldMirrorPreview
self._capture = capture
self._channel = 0
self._enteredFrame = False
self._frame = None
self._imageFilename = None
self._videoFilename = None
self._videoEncoding = None
self._videoWriter = None
self._startTime = None
self._framesElapsed = long(0)
self._fpsEstimate = None
@property
def channel(self):
return self._channel
@channel.setter
def channel(self, value):
if self._channel != value:
self._channel = value
self._frame = None
@property
def frame(self):
if self._enteredFrame and self._frame is None:
_, self._frame = self._capture.retrieve(channel = self.channel)
return self._frame
@property
def isWritingImage (self):
return self._imageFilename is not None
@property
def isWritingVideo(self):
return self._videoFilename is not NoneNote that most of the member variables are non-public, as denoted by the underscore prefix in variable names, such as self._enteredFrame. These non-public variables relate to the state of the current frame and any file writing operations. As previously discussed, application code only needs to configure a few things, which are implemented as constructor arguments and settable public properties: the camera channel, the window manager, and the option to mirror the camera preview.
Continuing with our implementation, let's add the enterFrame() and exitFrame() methods to managers.py:
def enterFrame(self):
"""Capture the next frame, if any."""
# But first, check that any previous frame was exited.
assert not self._enteredFrame, \
'previous enterFrame() had no matching exitFrame()'
if self._capture is not None:
self._enteredFrame = self._capture.grab()
def exitFrame (self):
"""Draw to the window. Write to files. Release the frame."""
# Check whether any grabbed frame is retrievable.
# The getter may retrieve and cache the frame.
if self.frame is None:
self._enteredFrame = False
return
# Update the FPS estimate and related variables.
if self._framesElapsed == 0:
self._startTime = time.time()
else:
timeElapsed = time.time() - self._startTime
self._fpsEstimate = self._framesElapsed / timeElapsed
self._framesElapsed += 1
# Draw to the window, if any.
if self.previewWindowManager is not None:
if self.shouldMirrorPreview:
mirroredFrame = numpy.fliplr(self._frame).copy()
self.previewWindowManager.show(mirroredFrame)
else:
self.previewWindowManager.show(self._frame)
# Write to the image file, if any.
if self.isWritingImage:
cv2.imwrite(self._imageFilename, self._frame)
self._imageFilename = None
# Write to the video file, if any.
self._writeVideoFrame()
# Release the frame.
self._frame = None
self._enteredFrame = FalseNote that the implementation of enterFrame() only grabs (synchronizes) a frame, whereas actual retrieval from a channel is postponed to a subsequent reading of the frame variable. The implementation of exitFrame() takes the image from the current channel, estimates a frame rate, shows the image via the window manager (if any), and fulfills any pending requests to write the image to files.
Several other methods also pertain to file writing. To finish our class implementation, let's add the remaining file-writing methods to managers.py:
def writeImage(self, filename):
"""Write the next exited frame to an image file."""
self._imageFilename = filename
def startWritingVideo(
self, filename,
encoding = cv2.cv.CV_FOURCC('I','4','2','0')):
"""Start writing exited frames to a video file."""
self._videoFilename = filename
self._videoEncoding = encoding
def stopWritingVideo (self):
"""Stop writing exited frames to a video file."""
self._videoFilename = None
self._videoEncoding = None
self._videoWriter = None
def _writeVideoFrame(self):
if not self.isWritingVideo:
return
if self._videoWriter is None:
fps = self._capture.get(cv2.cv.CV_CAP_PROP_FPS)
if fps == 0.0:
# The capture's FPS is unknown so use an estimate.
if self._framesElapsed < 20:
# Wait until more frames elapse so that the
# estimate is more stable.
return
else:
fps = self._fpsEstimate
size = (int(self._capture.get(
cv2.cv.CV_CAP_PROP_FRAME_WIDTH)),
int(self._capture.get(
cv2.cv.CV_CAP_PROP_FRAME_HEIGHT)))
self._videoWriter = cv2.VideoWriter(
self._videoFilename, self._videoEncoding,
fps, size)
self._videoWriter.write(self._frame)The public methods, writeImage(), startWritingVideo(), and stopWritingVideo(), simply record the parameters for file writing operations, whereas the actual writing operations are postponed to the next call of exitFrame(). The non-public method, _writeVideoFrame(), creates or appends to a video file in a manner that should be familiar from our earlier scripts. (See the Reading/Writing a video file section.) However, in situations where the frame rate is unknown, we skip some frames at the start of the capture session so that we have time to build up an estimate of the frame rate.
Although our current implementation of CaptureManager relies on VideoCapture, we could make other implementations that do not use OpenCV for input. For example, we could make a subclass that was instantiated with a socket connection, whose byte stream could be parsed as a stream of images. Also, we could make a subclass that used a third-party camera library with different hardware support than what OpenCV provides. However, for Cameo, our current implementation is sufficient.
As we have seen, OpenCV provides functions that cause a window to be created, be destroyed, show an image, and process events. Rather than being methods of a window class, these functions require a window's name to pass as an argument. Since this interface is not object-oriented, it is inconsistent with OpenCV's general style. Also, it is unlikely to be compatible with other window or event handling interfaces that we might eventually want to use instead of OpenCV's.
For the sake of object-orientation and adaptability, we abstract this functionality into a WindowManager class with the createWindow(), destroyWindow(), show(), and processEvents() methods. As a property, a WindowManager class has a function object called keypressCallback, which (if not None) is called from processEvents() in response to any key press. The keypressCallback object must take a single argument, an ASCII keycode.
Let's add the following implementation of WindowManager to managers.py:
class WindowManager(object):
def __init__(self, windowName, keypressCallback = None):
self.keypressCallback = keypressCallback
self._windowName = windowName
self._isWindowCreated = False
@property
def isWindowCreated(self):
return self._isWindowCreated
def createWindow (self):
cv2.namedWindow(self._windowName)
self._isWindowCreated = True
def show(self, frame):
cv2.imshow(self._windowName, frame)
def destroyWindow (self):
cv2.destroyWindow(self._windowName)
self._isWindowCreated = False
def processEvents (self):
keycode = cv2.waitKey(1)
if self.keypressCallback is not None and keycode != -1:
# Discard any non-ASCII info encoded by GTK.
keycode &= 0xFF
self.keypressCallback(keycode)Our current implementation only supports keyboard events, which will be sufficient for Cameo. However, we could modify WindowManager to support mouse events too. For example, the class's interface could be expanded to include a mouseCallback property (and optional constructor argument) but could otherwise remain the same. With some event framework other than OpenCV's, we could support additional event types in the same way, by adding callback properties.
Appendix A, Integrating with Pygame, shows a WindowManager subclass that is implemented with Pygame's window handling and event framework instead of OpenCV's. This implementation improves on the base WindowManager class by properly handling quit events—for example, when the user clicks on the window's close button. Potentially, many other event types can be handled via Pygame too.
Our application is represented by a class, Cameo, with two methods: run()
and
onKeypress(). On initialization, a Cameo class creates a WindowManager class with onKeypress() as a callback, as well as a CaptureManager class using a camera and the WindowManager class. When run() is called, the application executes a main loop in which frames and events are processed. As a result of event processing, onKeypress() may be called. The Space bar causes a screenshot to be taken, Tab causes a screencast (a video recording) to start/stop, and Esc causes the application to quit.
In the same directory as managers.py, let's create a file called cameo.py containing the following implementation of Cameo:
import cv2
from managers import WindowManager, CaptureManager
class Cameo(object):
def __init__(self):
self._windowManager = WindowManager('Cameo',
self.onKeypress)
self._captureManager = CaptureManager(
cv2.VideoCapture(0), self._windowManager, True)
def run(self):
"""Run the main loop."""
self._windowManager.createWindow()
while self._windowManager.isWindowCreated:
self._captureManager.enterFrame()
frame = self._captureManager.frame
# TODO: Filter the frame (Chapter 3).
self._captureManager.exitFrame()
self._windowManager.processEvents()
def onKeypress (self, keycode):
"""Handle a keypress.
space -> Take a screenshot.
tab -> Start/stop recording a screencast.
escape -> Quit.
"""
if keycode == 32: # space
self._captureManager.writeImage('screenshot.png')
elif keycode == 9: # tab
if not self._captureManager.isWritingVideo:
self._captureManager.startWritingVideo(
'screencast.avi')
else:
self._captureManager.stopWritingVideo()
elif keycode == 27: # escape
self._windowManager.destroyWindow()
if __name__=="__main__":
Cameo().run()When running the application, note that the live camera feed is mirrored, while screenshots and screencasts are not. This is the intended behavior, as we pass True for shouldMirrorPreview when initializing the CaptureManager class.
So far, we do not manipulate the frames in any way except to mirror them for preview. We will start to add more interesting effects in Chapter 3, Filtering Images.