14    Packets

In the internet, communication between two parties takes place in terms of “chunks” of data, each of which is handled separately by the network. Computer scientists refer to this approach as packet-based. Although aspects of the internet design will surely change, this characteristic is all but certain to be preserved. It’s worth taking a little time to understand why this arrangement works well and to make some connections back to our earlier discussion of the digital model (chapter 2).

Suppose that we are given some large collection of bits to communicate on a continuing basis between distant locations. There is a design choice to make: we can operate like a telephone connection or we can operate like letters in a post office. If we operate like a telephone, we establish a connection between the two communicating parties for the stream of bits to flow across. In contrast, if we operate like letters in a post office, we break the (logically continuous) stream into packets, each handled separately.

As described so far, the broken-into-packets approach just seems like extra complexity, which adds to the potential surprise in saying that this packet-based approach is the dominant means of data transmission in the modern world. In fact, even phone calls are typically handled as packets of sound (in ways that are imperceptible to the typical user of the phone system).

We should pause here to observe that there is no issue of sampling and encoding quality to be concerned about: we can always convey the bits in the stream exactly. That guarantee contrasts with some of the concerns we had in chapter 2 about analog vs. digital representation. Here, the issue isn’t analog vs. digital; we’re just trying to understand how the exact same (digital) bits should be handled when the task is to send them across the network.

Why would there be any advantage to breaking up the stream into packets? We can draw an analogy to roads. While it might be nice to have a dedicated lane of highway leading from each separate house to the mall, we recognize the impracticality of such an arrangement. While we may have a small piece of “private road” at our house—usually called a driveway—all of the other roads we use are shared. It’s too expensive (in multiple ways) to do otherwise.

Similarly, the packet-based approach is a way of sharing network resources efficiently. Cars can share the road in much the same way that packets can share the network. In fact, computer scientists refer to collections of packets on a network as “traffic.”

Even in situations that might seem to involve a continuous stream of new information, most sources of digital data don’t actually deliver a continuous stream of traffic (as we will see in the next section). If the demand is only intermittent, then it’s wasteful to reserve network capacity between the communicating parties for only occasional transfers. Establishing a connection is like establishing a private dedicated lane—overkill unless the traffic volume and need for special service can justify it.

Even for situations where some identifiable fraction of traffic needs better handling than the other traffic, we can typically accommodate that requirement within a packet-based approach: the solution is somewhat like car-pool lanes on highways, which are not dedicated to a single user but are instead limited to a particular class of traffic.

Compression

We mentioned in chapter 2 that audio and video can be represented digitally. We know that the analog representation of a movie is a piece of film that moves at a constant rate through the projector. Similarly, one analog representation of a piece of music is a segment of magnetic tape that moves at a constant rate past the playback head; another such representation is a vinyl disc that moves at a constant rate under a phonograph stylus. With those examples in mind, it would be easy to assume that digitally encoded audio or video is a constant-rate stream of bits, and perhaps the sort of thing that could justify a “dedicated highway” style of communication.

However, most of this audio and video data isn’t really a steady stream of bits if we represent it efficiently. At an early stage of the transformation from analog to digital there may well be a constant rate of “raw” bits. Those raw bits are samples of continuous analog input—electrical signals derived from sound waves or light waves. The sampling rate may well be constant. In chapter 2, we said that we can perfectly reconstruct the original wave if we sample at least twice as fast as the highest frequency of interest. So we might reasonably expect to be getting a steady stream of measurements at that “double speed,” or perhaps even a little faster. Instead, most well-designed systems send relatively “lumpy” data: a lot of bits at one point in time, many fewer bits at another point in time. The “lumpiness” arises when the raw bits are compressed by the best available techniques.

Why does compression make the steady stream into a lumpy stream? Compression finds ways to represent the same information but using fewer bits. We have previously examined digital-to-analog and analog-to-digital transformations, and in a roughly similar fashion there can be raw-to-compressed and compressed-to-raw transformations. Whereas the analog-to-digital transformation shifts from the analog representation to the digital representation, the raw-to-compressed shifts from an “obvious” digital representation (with a simple mapping back to analog) to a “sophisticated” digital representation that is smaller.

For example, video traffic often consists of scenes in which only a small portion of the screen is moving (for example, an actor in a room) while the rest of the scene is unchanging (for example, the room itself). Various sophisticated schemes allow the compression of this traffic into just the information required to recreate the movement, while essentially reusing the image of the unchanging background. While that transformation lowers the overall size of the video stream, it also has the effect of making it variable in size: relatively few bits are required for a scene that is unchanging, while many more bits are required when a jump cut occurs to a completely unrelated setting (effectively, the data rate shifts briefly from almost zero to the full data rate of the video technology).

In the presence of compression, both audio and video traffic are unpredictable and lumpy in their demands for network capacity. The “raw” versions of audio and video consist of steady streams that suggest using a kind of digital pipeline for transport; but once that data is “cooked” with compression, it all becomes bursty and more effectively transmitted as packets.

Incompressible Data

Do all data streams turn lumpy when we apply compression? No, some kinds of traffic don’t compress well and really do require the full data stream. One important example of traffic that doesn’t compress well is something that’s already been compressed. If we have a steady stream of compressed traffic, we typically can’t get dramatic changes in its size by applying compression again. After all, if we could always compress even previously compressed traffic, we’d be able to apply compression repeatedly to keep making traffic smaller and smaller. Eventually, anything at all would be reduced so it was only one or two bits long—which we know can’t be right.

If we look at the underlying reason we can’t just keep compressing already compressed data, we find that it helps us understand the other main sources of incompressible traffic. A compression system finds and exploits patterns in the data being transmitted and effectively squeezes out repeating patterns. That means that the compressed data doesn’t have any patterns left to squeeze out. This phenomenon is sort of like drying fruit. After we’ve dried some fruit (say, apple rings) we don’t expect that another trip through the drier will reduce its volume the same way the first trip did—most of the water’s already gone.

There are two other common ways we can encounter traffic that doesn’t have any repeating patterns, and they’re closely related to each other. One is that the traffic can be genuinely random, perhaps representing a sequence of coin tosses. The other is that the traffic can be encrypted, so that the underlying patterns have been deliberately hidden. The result of effective encryption looks random. The defining quality of random or random-looking data is that it doesn’t have any recognizable patterns. So, naturally enough, a compression system looking for patterns doesn’t find much when it’s handed random(ish) data. Both random data and encrypted data will be important in later chapters when we look at security. For now, it’s enough to notice that they are among the kinds of data that are typically not compressible.