RESOLUTION
The concept of resolution is used in all digital media, and is therefore applicable to video games. Resolution refers to the number of discrete and indivisible units (such as pixels, frames, available colors, polygons, or samples) used to represent (or resolve) a portion of an analog spectrum, in particular, those of space, time, color, geometry, or sound, respectively. Due to memory limitations, processing speed, and screen and speaker capabilities, these types of resolution are always limited in some way, requiring graphic designers, sound designers, and game designers to take them into consideration, especially in projects with more restrictive limitations regarding resolution. When there is insufficient resolution in any of these areas, some type of artifacting occurs, which disrupts the smoothness of the transitions between the discrete units involved, revealing the borders or gaps between them, which often disrupts continuity and calls attention to the lack of resolution. As such, attention to the boundaries of individual units is generally considered undesirable, since this usually results in what is considered a reduction in quality of the final output, requiring techniques to smooth over these gaps or boundaries and restore smoothness to the final output.
The first four types of resolution, spatial resolution, temporal resolution, color resolution, and geometric resolution, have to do with computer graphics. They can all be limited both by the way software is programmed as well as the capabilities of the hardware that the programs run on, although hardware limitations also place an upper bound on what can be done with software on any given system.
Spatial Resolution
Spatial resolution is measured in pixels per inch, and refers to the amount of detail possible in a digital image, which is made up of a grid of pixels (which is short for picture elements). The more spatial resolution an image has, the more it is capable of resolving small details. Standard resolutions of imaging devices include 640 × 480 pixels for standard NTSC television, and 1,920 × 1,080 pixels for full high-definition television. Some cameras, such as the Red One by Red Digital Cinema Camera Company, can produce digital images that are over 4,000 pixels across, but such images are still far less than what the human eye can perceive. Since the human eye is able to resolve around 0.3 minutes of arc, images produced by the eye, depending on conditions, are somewhere between 52 megapixels (McHugh, 2005) and 576 megapixels (Clark, 2005). The screens used by visual media devices, though, usually only occupy a portion of the eye’s field of vision during viewing, and gestalt processes performed by the eye and brain also process and interpolate imagery, so images with resolution far lower than what the eye is capable of can still be used without attention being drawn to issues of resolution that would disrupt the viewer’s contemplation of them.
The imaging device used by a video game, however, only presents an upper bound for resolution; processing power and software-related restrictions can also limit resolution, as in early home video games, such as those of the Atari VCS 2600, which used an NTSC television but had a resolution of only 320 × 192 pixels. Likewise, early home computer software had various graphics display standards that often did not use all the screen resolution offered by monitors. Prior to 1984, the CGA (Color Graphics Adaptor) standard, which allowed image resolutions of 320 × 200 pixels with a four-color palette (or 620 × 200 with a two-color palette), was used by DOS computers for graphic displays. Such harsh restrictions made representational imagery difficult, leading to the 1984 release of the EGA (Enhanced Graphics Adaptor) standard, which allowed image resolutions of 640 × 350 with 16 supported colors from a 64-color palette. In 1987, graphics improved again when IBM released the VGA (Video Graphics Array) standard with images of 640 × 480 pixels and a 256-color palette, which was later improved to the SVGA (Super Video Graphics Array) standard, with an image resolution of 800 × 600 pixels. Today, console-based games are also available for high-definition television monitors, and games with three-dimensional graphics can be scaled to a variety of resolutions, unlike two-dimensional games that were resolution-specific.
The lower an image’s spatial resolution, the more apparent the edges of individual pixels will be, resulting in a jagged appearance referred to as aliasing. The effects of aliasing can be lessened by using rows of pixels of interpolated colors or tones at boundaries between different colors or tones, to make the transition between them more gradual; this process is called anti-aliasing. Various anti-aliasing algorithms use such things as subpixel rendering, the colors of neighboring pixels, and knowledge of the workings of the human visual perception system in order to determine the correct coloring of pixels for the reduction of aliasing.
Temporal Resolution
Temporal resolution refers to the number of frames per second (fps) used in time-based media. The more frames per second used in moving imagery, the smoother apparent motion can appear within the imagery. Filmmakers in the silent era discovered that 16 fps was the rate at which “flicker fusion” occurred; that is, the rate at which a projected image appears to be continuous rather than flickering, thus setting a lower bound for temporal resolution in moving image media. Sound film raised the rate to 24 fps (due the demands of sound technology) and some film formats use higher rates; for example, Showscan footage is shot and projected at 60 fps, giving its imagery a more realistic appearance, due to the lack of visible grain in the imagery.
For video games, however, the frame rate is determined by both hardware and software, similar to spatial resolution. While computer monitors usually have a frame rate of 30Hz or higher in order to produce a continuous image, the processor of the computer using the monitor may produce imagery at a lower frame rate, causing frames to be held on-screen longer than the screen’s refresh rate; for example, the computer game 3D Monster Maze (J. K. Greye Software, 1982) had a frame rate of only 6 fps, due to the demands it made on early systems. Other games that involve fast action require higher frame rates, often 30 fps or 60 fps. Quake III Arena (id Software, 1999) was designed to have a maximum frame rate of 125 fps, though processing demands and hardware limitations could slow the game down.
Temporal aliasing, known as strobing, occurs because a frame rate is too low to convey a sense of smooth motion, and moving objects appear to jump from one position to another rather than moving smoothly between them. The effects of strobing can be lessened through the use of motion-blurring, which simulates the blur that an object would have passing through a given span of space in a given span of time, all within a single image. The addition of motion blur to a moving object fills in the gaps between the object’s positions from one frame to the next, smoothing the overall appearance of the motion. Micro stuttering is another type of temporal aliasing, found specifically in game systems that use more than one graphics processing unit (GPU) to produce their imagery. When multiple GPUs are producing imagery at slightly different rates, the result is disruption of smoothness, in which some images remain on-screen longer than others.
Color Resolution
Color resolution or depth (or in the case of grayscale imagery, tonal resolution or depth), is measured in bits per pixel (bpp), and refers to the number of colors available for use in an image or series of images (for n bits there are 2n possibilities). Color resolution first depends on hardware capabilities that determine what range of colors can be displayed, and which set an upper bound for resolution. Most display systems are RGB-based, meaning that their colors are produced by combining red, green, and blue, each of which can occur at different levels depending on the resolution available. Within hardware limitations, color resolution is also determined by software programming, which determines the number of bpp that will be used. As mentioned above, different graphics standards had a range of color palettes, from black and white imagery (1 bpp) to a four-color palette (2 bpp), eight-color palette (3 bpp), 16-color palette (4 bpp), 64-color palette (6 bpp), 128-color palette (7 bpp), 256-color palette (8 bpp) and so on, to palettes with millions or billions of colors. By comparison, the human eye is said to be able to distinguish as many as ten million colors, though estimates vary widely (Judd and Wyszecki, 1975, p. 388).
When the color resolution of an image is low, the jump from one color to another along a gradient is more abrupt and noticeable, resulting in color aliasing or mach banding, also known as posterization. This can be alleviated through the use of dithering, in which pixels of different colors are mixed in changing ratios across the boundary between colored areas, allowing one color to increase while another decreases, simulating a gradient between different colors or tones when the image is viewed from a distance or if the spatial resolution is high enough. To get around color limitations, some games also use an adaptive palette that has a limited number of colors but changes what those colors are from one screen to another, depending on the needs of the scene being displayed. For example, the pre-rendered images used in Myst (Cyan, 1993) used a 256-color adaptive palette and dithering to smooth color gradients within a scene. By contrast, Myst Masterpiece (Cyan Worlds, 2000) had 24-bit color and did not need to rely on dithering. Some monitors are now capable of 48-bit color, which can produce 281.5 trillion colors, far beyond what the human eye can distinguish.
Geometric resolution, when applied to three-dimensional graphics, refers to the number of polygons used to resolve a three-dimensional shape within a three-dimensional space. The more polygons used, the more curved surfaces can be approximated and accurately represented in an image. Geometric resolution, then, is measured by the number of polygons per second that a computer is able to render on-screen in real time. For example, the Nintendo 64 was able to render between 100,000 and 150,000 polygons per second, while the PlayStation 3 is said to be able to render 275 million polygons per second. While geometric resolution sets limitations on the modeling of three-dimensional objects, how realistic those objects appears also depends on such factors including color resolution, textures, lighting, and movement.
Low geometric resolution, in which the edges and vertices of individual polygons are discernible, results in a blocky or faceted appearance, whereas higher resolution allows for smoother curves and flowing forms. Naturally, simpler objects require fewer polygons, while more complex ones require more. One of the challenges of computer modeling is to represent the object being modeled with as few polygons as possible while still maintaining as realistic an appearance as possible. The geometric aliasing that occurs in low-resolution models can be aided by certain shading techniques, such as Gouraud shading or Phong shading, which apply color or tonal gradients across polygons so that their boundary colors match, making the boundaries between them are less noticeable and smoothing their appearance (Foley et al., 1990).
Since each visible polygon must be accounted for during rendering, objects with higher geometric resolution take longer to render than objects with lower resolution. This means that distant copies of an object that are barely visible and take up very few pixels on-screen will take just as long to render as the same objects seen in close-up, thus wasting rendering time on details that will not be visible. To remedy the situation and reduce the time needed for rendering, computer graphics processes, such as NURBS (Non-Uniform Rational Basis (or Bézier) Splines), allow geometric resolution to change dynamically based on the apparent distance from the viewer, so as to save calculation and rendering time when objects take up less on-screen space (Polevoi, 2000).
In addition to techniques involving dynamic resolution, video game designers have found other ways to limit the number of objects that needed to be rendered in real time, including the obscuring of distant objects in darkness or fog, and the designing of spaces to avoid views that involve great z-axis depth, thus limiting the distance at which objects are visible.
Sonic Resolution
The quality of a game’s sound depends on sonic resolution, which is measured in the number of samples per second and bits per sample. Samples are used to digitally reconstruct an analog sound wave as accurately as possible, and each sample is used to indicate the amplitude of a sound wave at a particular point in time. The number of samples per second, then, places an upper boundary on the highest frequency that can be represented, while the number of bits per sample determines the accuracy of representing the waveform’s amplitude at any given sample. Since human hearing typically ranges from 20 Hz to 20,000 Hz, compact discs have a sampling frequency of 44,100 samples per second, which places an upper bound of 22,050 Hz for signals that can be reconstructed at that sample rate, thus covering the range of human hearing. Many newer formats, such as DVD-Audio, use even higher sampling rates, some as high as 192,000 Hz.
Although analog signals can suffer from several different types of distortion (namely those of attenuation, phase, amplitude, harmonic, and intermodulation), digital signals can also suffer distortion such as clipping (when not enough amplitude is available during playback due to too few bits per sample), or aliasing due to too few samples per second. Oversampling and anti-alias filtering also help to smooth out signals, but require additional memory and processing. Certain sound formats have been designed specifically for video games, including the VGM (Video Game Music) format, used for SEGA systems in the 1980s, and the PSF (Portable Sound Format), originally used for the first Sony PlayStation, and since then adapted to a number of other systems. Both formats now also include a number of subformats, with different specifications and sampling rates.
Interactive Resolution
Finally, the concept of resolution can also be applied to a video game’s interactivity. Like graphical resolution, the resolution of a game’s interactivity has two dimensions to it, which can be measured according to the number of choices per second, and the number of options per choice. Fast-action games will usually have a high rate of choices per second, with reaction an important factor in gameplay. Players often have to react quickly and have little time to decide between options, with choices continually being made. Fast-action games can be made easier by limiting the number of options per choice; for example, in Space Invaders (Taito, 1978), players usually have four choices available: move left, move right, shoot, or do nothing. Other kinds of games, such as those of the adventure genre that have more developed storylines and worlds, have slower paces where more time is allotted to players to consider what they should do next, but the number of possible actions they can take is higher and the series of choices they will have are often more integrated, interdependent, and complicated. Usually games will need to have either a high number of choices per second or a high number of options per choice to be considered interesting or challenging; yet if both types of resolution are high the game may be considered too difficult.
Just as other types of low resolution may distract players and call attention to a game’s limitations, reducing the frequency of choices and number of options per choice can also frustrate players and make a game’s interactive potential seem inadequate. Games with an overreliance on cut-scenes or video clips may be seen as relatively uninteractive, an accusation sometimes leveled at the genre of games known as interactive movies. Too few options per choice may make choices too easy or uninteresting, leading to decreased involvement and engagement in a game, which may also decrease a game’s replayability. The greater the frequency of choices that a player must make, the more that player feels a sense of agency during gameplay, while a greater number of options per choice increases the need for decision-making, demanding more consideration from players and giving them more alternatives to explore in later replayings of the same game. These two dimensions of interactive resolution can also compensate for each other; since a greater number of options per choice requires more thought, a player will not need as many choices per second to remain engaged. Likewise, having limited options will not seem as constricting if a player must deal with a large number of choices per second.
Like other types of resolution, interactive resolution depends on the limitations of both the hardware and software used. In the area of hardware, the ability to interact is limited by the sensitivity of input devices, as well as the number of actions and functions they allow (for example, directions of movement and the number of buttons or triggers they contain), as well as things such as processor speeds and loading times. Likewise, the software running a program will determine input speeds, the frequency of screen updates and other types of feedback, and what is possible at any given point within gameplay.
Relationships between Types of Resolution
The various types of resolution found in video games are not isolated in their effects, but compete for resources (such as memory and processing power) resulting in balances and tradeoffs that must be taken into consideration during game design and programming. At the same time, increasing one type of resolution can sometimes be used to compensate for decreases in other types of resolution, as in the example given in the previous section. Thus, one must consider not only the various types of resolution but also the relationships between them.
For example, because they all deal directly with graphics, three types of resolution—spatial resolution, color resolution, and geometric resolution—are closely related. The aliasing in an image with low spatial resolution can be eased with higher color resolution that allows smoother anti-aliasing to be done. Higher color resolution can help reduce temporal aliasing, because it makes motion-blurred imagery possible, since blurs require gradients. Smoother gradients, used by shading techniques, can also reduce the effects of limited geometric resolution. On the other hand, higher spatial resolution can make up for low color resolution by making dithering less noticeable, allowing dithered color gradients to appear smoother. The quality of grayscale imagery is also perceived differently from color imagery, with a wider dynamic range of color making up for lower spatial resolution: so a designer wishing to save memory should reduce the tonal resolution in grayscale imagery while leaving the spatial resolution unchanged; whereas for color imagery, the spatial resolution of color images should be reduced while the color resolution is left unchanged (Ester, 1990). Finally, geometric resolution also depends on spatial resolution, since the number of pixels available for imaging will limit the degree to which complex geometry can be adequately represented on-screen, thus effectively limiting the amount of geometric resolution necessary.
Other relationships exist as well. Temporal and spatial resolution both are factors in determining the limits of interactivity, since they determine the speed of gameplay and what is seen of the game’s world. Greater spatial resolution and greater geometric resolution both require more render time when graphics are produced in real time, slowing down the rendering of frames and decreasing the number of frames per second that a game is able to display. Likewise, more textures and colors mean more use of processing power and a potentially slower frame rate as well. Also, because sound can influence the human perception of color, one could even suggest a relationship between sound and color (Letourneau and Zeidel, 1971). While graphics and sound do not directly limit the resolution of interactivity, they may place limitations on a game’s content that in turn limits interactivity. This is true especially of earlier game technology, such as the Atari VCS 2600, where the program running the game had to alternate between accepting input and other tasks such as putting graphics on-screen, producing sounds, and changing color look-up tables.
Although issues involving resolution are less likely to arise as systems grow faster and more powerful and are thus able to provide all the memory and processing power needed for high resolution, new venues such as mobile phones have reintroduced smaller screens to gaming, and state-of-the-art games tend to push their boundaries whatever they may be, allowing issues of resolution to remain important. Also, the concept of resolution often provides a way of comparing and benchmarking technologies, and the measurements of a system’s capabilities in regard to the different types of resolution are typically included in lists of specifications.
References
Clark, Roger N. (2005). “Notes on the Resolution and Other Details of the Human Eye.” Clarkvision.com, January 2005. Available at www.clarkvision.com/imagedetail/eye-resolution.html.
Ester, Michael (1990). “Image Quality and Viewer Perception.” Leonardo, Supplemental Issue, pp. 51–63.
Foley, James, Andries Van Dam, Steven K. Feiner, and John F. Hughes (1990). Computer Graphics: Principles and Practice (2nd edition). Upper Saddle River, NJ: Addison-Wesley Professional.
Judd, Deane B. and Günter Wyszecki (1975). Color in Business, Science and Industry. Wiley Series in Pure and Applied Optics (3rd edition), New York: Wiley-Interscience.
Letourneau, Jacques and Neil S. Zeidel (1971). “The Effect of Sound on the Perception of Color,” American Journal of Optometry & Archives of American Academy of Optometry, 48. 2, February, pp. 133–137.
McHugh, Sean (2005). “Cameras vs. The Human Eye,” Cambridge in Color.com. Available at www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm.
Polevoi, Robert (2000) “Lesson 83—3D E-Commerce With MetaStream—Part 3,” from his January 5, 2000 column 3-D Animation Workshop. Available at www.webreference.com/3d/lesson83/part3.html.