Footnotes

Chapter 1

1. Since Wikipedia contains Linux information and must therefore refer to Linux frequently, the debate has particular relevance to Wikipedia itself. The discussion page for the Wikipedia article is also well worth reading.

2. After all, “GNU’s not UNIX!”

3. Several of our technical reviewers protested that we seem to be crediting GNU with the creation of most of the world’s free software. We are not! However, GNU has certainly done more than any other group to promote the idea of free software as a social enterprise and to structure ongoing debate about licensing terms and interactions between free and nonfree software.

4. We consider a “production” environment to be one that an organization relies on to accomplish real work (as opposed to testing, research, or development).

5. See page 1264 for some background on BSD, System V, and the general history of UNIX.

6. It’s not really a fixed-width font, but it looks like one. We liked it better than the real fixed-width fonts that we tried. That’s why the columns in some examples may not all line up perfectly.

7. Solaris 10’s default shell for root is the original Bourne shell, which (rather surprisingly) does not understand ~ or ~user notation.

8. Or worse yet, a link to Google through lmgtfy.com

9. OpenSolaris does offer a Linux-quality package management system and Internet repository. This feature does not exist in Solaris 10, but it’s likely to be featured in Solaris 11.

10. A tendency lovingly and sadistically documented in Simon Travaglia’s Bastard Operator from Hell stories; see bofh.ntk.net for the archive. (Look under BOFH.)

Chapter 2

1. sort accepts the key specification -k3 (rather than -k3,3), but it probably doesn’t do what you expect. Without the terminating field number, the sort key continues to the end of the line.

2. See Units on page 14 for an introduction to these units.

3. If your shell understands the command helloworld without the ./ prefix, that means the current directory (.) is in your search path. This is bad because it gives other users the opportunity to lay traps for you in the hope that you’ll try to execute certain commands while cd’ed to a directory on which they have write access.

4. The “dot” command is a synonym for source, e.g., . helloworld.

5. Note that the error messages and usage message go to standard output. Shouldn’t they go to standard error instead? That would in fact be more correct, but since this script isn’t intended for use as a filter, the distinction is less important.

6. In reality, these operations are now built into the shell and do not actually run /bin/test.

7. More accurately, the filename expansion is just a little bit magic in that it does maintain a notion of the atomicity of each filename. Filenames that contain spaces will go through the for loop in a single pass.

8. Depending on the invocation, exec can also have the more familiar meaning “stop this script and transfer control to another script or expression.” It’s yet another shell oddity that both functions are accessed through the same statement.

9. Perl guru Tom Christiansen commented, “I don’t know what a ‘scripting language’ is, but I agree that regular expressions are neither procedural nor functional languages. Rather, they are a logic-based or declarative language, a class of languages that also includes Prolog and Makefiles. And BNFs. One might also call them rule-based languages. I prefer to call them declarative languages myself.”

10. Although this section shows HTML excerpts as examples of text to be matched, regular expressions are not really the right tool for this job. Our external reviewers were uniformly aghast. Perl and Python both have excellent add-ons that parse HTML documents the proper way. You can then access the portions you’re interested in with XPath selectors. See the Wikipedia page for XPath and the respective languages’ module repositories for details.

11. Since semicolons are separators and not terminators, the last one in a block is optional.

12. Tom Christiansen commented, “That wouldn’t be my own first choice, but it is a good one. My nominee for the most common error in programs is that they are usually never rewritten. When you take English composition, you are often asked to turn in an initial draft and then a final revision, separately. This process is just as important in programming. You’ve heard the adage ‘Never ship the prototype.’ Well, that’s what’s happening: people hack things out and never rewrite them for clarity and efficiency.”

13. The naming of the scripts themselves is important, too. In this context, dashes are more common than underscores for simulating spaces, as in system-config-printer.

Chapter 3

1. For example, one common use of single-user mode is to reset a lost root password. This operation requires modification of the /etc/shadow file.

2. We once had a corrupted keymap file, and since the keymap file is loaded even in single-user mode, single-user was useless. Setting init=/bin/sh was the only way to boot the system to a usable single-user state to fix the problem. This can also be a useful trick in other situations.

3. YaST is a SUSE-specific graphical configuration utility that maintains many aspects of a SUSE system.

4. In theory, databases should be particularly resistant to this form of corruption, but our experience in practice doesn’t necessarily support this theory.

Chapter 4

1. In fact, the permissions can be set so restrictively that even the owner of a file cannot access it.

2. “Valid” is the operative word here. Certain operations (such as executing a file on which the execute permission bit is not set) are forbidden even to the superuser.

3. One of our technical reviewers commented, “That’s certainly not the intent. In fact, it’s the average sites running basic DNS/web/email service that do best with SELinux. If you’re doing unusual stuff, you end up in policy hell and turn it off. SELinux has actually gotten a lot better in recent times. Of course, I still turn it off…”

4. This FAQ was written for individual users of PGP. In the context of system administration, you should certainly consider the potential for offense. How will your shocking nonsense sound to the jury that’s adjudicating your sexual harassment case?

5. Ubuntu Linux goes even further. By default, the system has no valid root password and requires the use of sudo, detailed later in this section.

6. For the same reason, do not include “.” (the current directory) in your shell’s search path. Although convenient, this configuration makes it easy to inadvertently run “special” versions of system commands that a user or intruder has left lying around as a trap. Naturally, this advice goes double for root.

7. Or even zero people, if you have the right kind of password vault system in place.

Chapter 5

1. Pages are the units in which memory is managed, usually between 1KiB and 8KiB in size.

2. As pointed out by our reviewer Jon Corbet, Linux kernel 2.6.24 introduced process ID namespaces, which allow multiple processes with the same PID to exist concurrently. This feature was implemented to support container-based virtualization.

3. At least initially. If the original parent dies, init (process 1) becomes the new parent. See page 124.

4. Actually, all but one are library routines rather than system calls.

5. The functions of <Control-Z> and <Control-C> can be reassigned to other keys with the stty command, but this is rare in practice. In this chapter we refer to them by their conventional bindings.

6. More specifically, bus errors result from violations of alignment requirements or the use of nonsensical addresses. Segmentation violations represent protection violations such as attempts to write to read-only portions of the address space.

7. Which may be easier said than done. The terminal emulator (e.g., xterm), terminal driver, and user-level commands may all have a role in propagating SIGWINCH. Common problems include sending the signal to a terminal’s foreground process only (rather than to all processes associated with the terminal) and failing to propagate notification of a size change across the network to a remote computer. Protocols such as Telnet and SSH explicitly recognize local terminal size changes and communicate this information to the remote host. Simpler protocols (e.g., direct serial lines) cannot do this.

a. Uses absolute priority, but adds 20 to the value you specify.

8. Actually, it’s worse than this: the stand-alone nice interprets nice -5 to mean a positive increment of 5, whereas the shell built-in nice interprets this same form to mean a negative increment of 5.

9. Well, usually. strace can interrupt system calls. The monitored process must then be prepared to restart them. This is a standard rule of UNIX software hygiene, but it’s not always observed.

10. Most filesystem implementations reserve a portion (about 5%) of the storage space for “breathing room,” but processes running as root can encroach on this space, resulting in a reported usage that is greater than 100%.

Chapter 6

1. It’s perhaps more accurate to say that these entities are represented within the filesystem. In most cases, the filesystem is used as a rendezvous point to connect clients with the drivers they are seeking.

2. Application Programming Interface, a generic term for the set of routines that a library, operating system, or software package provides for programmers to call.

3. We say “in most situations” because Solaris’s ZFS filesystem has adopted a rather different approach to mounting and unmounting, not to mention many other aspects of filesystem administration. Long-time readers may be expecting a snippy comment about gratuitous incompatibility at this point, but the ZFS scheme is a clear improvement and we look forward to the day that it’s adopted by other systems. In the meantime, we must of necessity keep our ZFS coverage somewhat ghettoized. See page 264 for more details.

4. ls -b shows the special characters as octal numbers, which can be helpful if you need to identify them specifically. <Control-A> is 1 (\001 in octal), <Control-B> is 2, and so on.

5. One reviewer commented, “Nagios (see page 887) uses them, and it sometimes needs help.”

6. The file permissions that ls shows for a symbolic link, lrwxrwxrwx, are dummy values. Permission to create, remove, or follow the link is controlled by the containing directory, whereas read, write, and execute permission on the link target are granted by the target’s own permissions. Therefore, symbolic links do not need (and do not have) any permission information of their own.

7. If you think of the owner as “the user,” you can easily remember the order of the permission sets with the word Yugo (like the car). This is also the letter coding used by the mnemonic version of chmod.

8. The kernel understands the #! (“shebang”) syntax and acts on it directly. However, if the interpreter is not specified completely and correctly, the kernel will refuse to execute the file. The shell then makes a second attempt to execute the script by calling sh.

9. Try find mountpoint -xdev -inum inode -print.

10. If myprog were a shell script, it would need both read and execute permission turned on. For the script to be run by an interpreter, it must be opened and read like a text file. Binary files are executed directly by the kernel and therefore do not need read permission turned on.

11. Make sure that your PATH environment variable puts /bin before /usr/gnu/bin so that you get the Solaris-specific versions of ls and chown instead of the GNU versions.

12. In an effort to keep customers disoriented and docile, HP has adopted a strategy of abducting the names of existing filesystems and applying them to proprietary products. For example, HP’s HFS was so called to facilitate confusion with Apple’s Hierarchical File System, also known as HFS. HP calls its VxFS port “JFS” to forestall the possibility that users might distinguish it from IBM’s own unrelated JFS filesystem.

13. In addition to “allow” and “deny”, the NFSv4 specification also allows “audit” and “alarm” entries that do not affect permission calculations but are potentially useful for logging and security control. The exact meaning of these entries is implementation dependent.

Chapter 7

1. At first we could not make this work because we were using sudo (see page 113), and the environment variables set up by sudo command are usually different from those resulting from sudo su - and then running the command in a separate step. The chdev command cares. New versions (1.70 or later) of sudo have the -i flag to address this situation.

2. $1$ is the tag for the BSD MD5 algorithm; Sun uses its own MD5 implementation and tags it $md5$.

3. Except on Solaris, where chfn does not exist. The superuser can change a user’s finger information with passwd -g.

4. This message appears when you log in on the console or on a terminal, but not when you log in through a display manager such as xdm, gdm, or kdm. Not only will you not see the message, but you will generally be logged out immediately because of the display manager’s inability to write to the proper directory (e.g., ~/.gnome).

5. Use the chsec command to change files in /etc/security rather than editing them directly.

6. There are 86,400 seconds in a day: 60 * 60 * 24.

7. Changing to a stronger encryption algorithm should be high on your to-do list for new AIX boxes.

8. To set a group password under Solaris, you have to use passwd and cut and paste into /etc/group. There is no /etc/gshadow or equivalent file.

9. Because the same password can have many encrypted representations, this method verifies only that the user has reset the password, not that it has actually been changed to a different password.

10. PAM is a relatively recent addition to AIX; it should be fully functional in versions 5.3 and later.

11. /etc/security/{passwd,group} on AIX

12. Now that Oracle has purchased Sun, it’s unclear if this system will survive as a product after the merger is complete.

Chapter 8

1. The 512-byte standard for hard disks may not hold out much longer; see lwn.net/Articles/377895.

2. That’s right: for some reason, the power cable is more complicated than the data cable.

3. In the United States, an excellent source for good quality but cheap SATA cables is monoprice.com.

4. “Daisy chaining” is the common description, but it’s perhaps a bit misleading. Parallel SCSI is physically wired as a chain, but it is electrically a single bus.

5. To deal with this issue, Linux uses UUIDs in the /etc/fstab file instead of device names; see page 262.

6. DSF = device special file

7. On the other hand, at $80 for a 1TB drive, why bother?

8. The ATA secure erase command is password-protected to make it more difficult to access. Therefore, you must set the drive password before invoking the command. Don’t bother to record the password, however; you can reset it at will. There is no danger of locking the drive.

9. Solaris and HP-UX even have “dynamic root disk” systems to facilitate the maintenance and use of multiple roots. See the man pages for beadm or lucreate for Solaris and drd for HP-UX.

10. OK, OK, it probably just says “Format” or “OK,” but this is what it should say.

11. Using only the outer cylinders of a disk to improve performance is known as “short stroking,” the stroke in question being the travel of the head armature.

12. EFI has more recently become UEFI, a “unified” EFI effort supported by multiple vendors. However, EFI remains the more common term in general use. UEFI and EFI are essentially interchangeable.

13. RAID is sometimes glossed as “redundant arrays of independent disks,” too. Both versions are historically accurate.

14. Parity data is distributed among all the drives in the array; each stripe has its parity stored on a different drive. Since there’s no dedicated parity disk, it’s unlikely that any single disk will act as a bottleneck.

15. Slogan: “Enough is enough. You can either join BAARF. Or not.”

16. HP-UX limitations require swap space to reside in the first 2GiB of the physical disk and the boot volume to be the first logical volume. The 1.5GB root and 500MB swap shown here were chosen to work around these constraints. You can have a root partition that is larger than these values, but you must then have separate boot and root volumes. See the man page for lvlnboot for more details.

17. In most cases, only metadata changes are journaled. The actual data to be stored is written directly to the filesystem. However, you can change this behavior with the data mount option. See the mount man page for specifics.

18. Some say that the recommendation for ext4 in kernel 2.6.28 was, in retrospect, premature. Current versions are solid, however.

19. Some systems have a mklost+found command you can use to recreate this directory if it is deleted.

20. The noauto mount option excludes a given filesystem from automatic mounting by mount -a.

21. The REFER column shows the amount of data referenced by the active copy of each filesystem. /demo and /demo/new_fs have similar REFER values because they’re both empty filesystems, not because there’s any inherent relationship between the numbers.

22. The reservation and quota properties take into account all storage costs of the filesystem, including the space consumed for snapshots. If you want to limit only the size of the active copy of the filesystem, use the refreservation and refquota properties instead. The ref prefix indicates “amount of data referred to” by the active filesystem, the same total shown in the REFER column in zfs list output.

23. This directory is hidden by default; it does not appear in ls -a output. You can make it visible with zfs set snapdir=visible filesystem.

24. In this example the disks are all the same size, but the virtual devices are not (1TB vs. 500GB).

25. Just to be clear, many such filesystems exist. They are known generically as cluster (or clustered) file-systems. Special locking and synchronization algorithms must be used to implement clustering, so clustered filesystems are typically slower than standard filesystems on local disks. VxFS can operate in clustered or nonclustered mode, so it’s a serviceable option for either situation.

Chapter 9

1. Many sites go further than this and send a text message to an administrator’s phone as soon as a problem is detected. See Chapter 21, Network Management and Debugging, for more details.

2. You can configure cron to use other shells as well.

3. That is, the user after whom the crontab file is named. On most (but not all) systems, the actual owner of crontab files is root.

4. One of our contributors reports having seen a case in which cron consumed 100% of the CPU because the system date had been set to the UNIX epoch. The local time zone was a negative offset from GMT, so with the offset taken into account, the local time appeared to be a negative number, and cron was confused. Most systems power their on-board clocks with a battery, so clock resets are not as unusual as you might think. Time instability is a common symptom of a dead or dying battery.

5. Many systems’ kernels can be configured to put core dumps in a particular directory, or optionally, not generate them at all. For example, see man core on Linux or man coreadm on Solaris.

6. Not all versions of find support the -xdev argument. On some systems, it’s called -x.

7. Bryan Helvey, one of our technical reviewers, has worked in the oil industry and notes that core in that context is as likely to refer to a core sample as to a core dump. More generally, we acknowledge that it’s inherently dangerous to delete files base on their names alone.

Chapter 10

1. A large financial institution located in the World Trade Center kept its “off-site” backups one or two floors below their offices. When the building was bombed (the first time), the backup tapes (as well as the computers) were destroyed. Make sure “off-site” really is.

2. GNU versions of restore include the -C option to verify a dump tape against a directory tree.

3. For example, restore -t reads only the table of contents for the dump, which is stored at the beginning of the tape. When you actually restore a file, you are testing a more extensive region of the medium.

4. It’s helpful to treat users who request the restoration of accidentally deleted files as colleagues who are spot-checking your backup system rather than as incompetent, file-deleting annoyances. A positive attitude makes the experience more pleasant for both of you and increases the number of spot-checks.

5. Actually, most versions of dump do not keep track of files that have been deleted. If you restore from incremental backups, deleted files are recreated.

6. Holes are blocks that have never contained data. If you open a file, write one byte, seek 1MB into the file, then write another byte, the resulting “sparse” file takes up only two disk blocks even though its logical size is much bigger. Files created by Berkeley DB or ndbm contain many holes.

7. dump requires access to raw disk partitions. Anyone allowed to do dumps can read all the files on the system with a little work.

8. Legacy systems may use a separate rdump command to perform dumps to a remote tape drive. Modern dumps accept a -f hostname:tapedevice argument.

9. All the entries for a tape unit use the same major device number. The minor device number tells the tape device driver about special behaviors (rewinding, byte swapping, etc.).

10. The star next to iamlost indicates that it has been marked for extraction.

11. Some versions of dump and restore are rumored to keep track of deletions. We believe Solaris and Linux to be among these.

12. The GNU implementation includes a filename mapping table as one of the files in the archive. Users of the standard tar can extract the contents of the archive and fix it up by hand, but the process is tedious.

13. GNU’s tar handles holes intelligently if you invoke the -S option when creating an archive.

14. On the other hand, if you zfs send -R the /home filesystem and its descendants, there’s currently no way to restore only /home/ned; you must restore /home. As an administrator, you probably don’t want to have to schedule every home directory for independent backups.

Chapter 11

1. More accurately, it uses “netloghost” as one of its hostname aliases. This setup allows the identity of the log host to be modified with little reconfiguration. An alias can be added in /etc/hosts or set up with a CNAME record in DNS. See page 585 for more information about DNS CNAME records.

2. Unless users running X have the xconsole program running, they won’t get these messages.

Chapter 12

1. Google “Custom JumpStart and Advanced Installations” to access Sun’s guide, which contains full details on the rules and profiles files.

2. A diskless client has no hard disk for local filesystems and relies on network services for storage. A dataless client has only has local swap space and perhaps /tmp and /home filesystems.

3. Since vendors all use approximately the same protocols and architectures, aren’t you glad that they cooperated and settled on a standard installation system? :-)

4. Not to be confused with Yum Fish Bait with Live Prey Technology (LPT), yum3x.com.

5. The development team distinguishes between the terms “repository” and “publisher,” but we’ll treat them as equivalent here.

6. Contrast with more than 30,000 in Ubuntu Karmic Koala.

7. Actually, swagentd invokes swagent, but this process is transparent to the user.

8. For example, suppose that sysadmins Alice and Bob both edit the same file and that each makes some changes. Alice saves first. When Bob saves his copy of the file, it overwrites Alice’s version. If Alice has quit from the editor, her changes are completely gone and unrecoverable.

9. Even though this command makes no changes, it must still be run as root because it creates lock files in the .git directory.

10. The explicit call to sh forces the redirection operator to be evaluated in the context of a root shell. If we simply typed echo mtab >> .gitignore, the shell would try to open .gitignore before running sudo.

11. Security patches are a possible exception to this rule. Plug security holes as soon as they are found. On the other hand, security patches do sometimes introduce bugs.

12. Hard-core compile-it-yourselfers should check out the Gentoo Linux distribution, which is designed to be recompiled from scratch on the destination system.

Chapter 13

1. The NDISwrapper project enables Windows drivers for some networking devices to be used under Linux. See sourceforge.net/projects/ndiswrapper for full details.

2. Dan Foster, one of our technical reviewers, commented, “Direct manipulation of the ODM with the odm* tools is not recommended if you don’t know exactly what you’re doing. These commands do no error checking on the data you modify, whereas the normal ch*/mk*/rm* tools validate data prior to making changes. The odm* tools are like a loaded AK-47 with no safety mechanism: one quick touch, and you’ve discharged several rounds into an eviscerated target.”

3. If you’re using really oddball PC hardware, it can be a challenge to create a configuration in which device interrupt request vectors (IRQs) and I/O ports do not overlap. You can view the current assignments on your system by examining the contents of /proc/interrupts and /proc/ioports, respectively. The overlap isn’t typically an issue with current mainstream PC hardware.

Chapter 14

1. A group of Linux enthusiasts from BLUG, the Bergen (Norway) Linux User Group, actually implemented the Carrier Pigeon Internet Protocol (CPIP) as specified in RFC1149. For details, see the web site blug.linux.no/rfc1149.

2. This is actually a little white lie. ARP is not really part of TCP/IP and can be used with other protocol suites. However, it’s an integral part of the way TCP/IP works on most LAN media.

3. A Google study presented at RIPE 57 in October 2008 indicated that overall IPv6 penetration (actual use, not capability) was 0.24%. No country had IPv6 penetration greater than 0.76%.

4. For specificity, RFCs that describe protocols often use the term “octet” instead of “byte.”

5. In general, an IP address identifies a specific and unique destination. However, several special cases muddy the water. NAT (page 462) uses one interface’s IP address to handle traffic for multiple machines. IP private address spaces (page 462) are addresses that multiple sites can use at once, as long as the addresses are not visible to the Internet. Anycast addressing shares one IP address among several machines.

6. You can find a full list of assigned ports at iana.org/assignments/port-numbers.

7. Of course, 0 counts as being divisible by any number…

8. Of course, many routers now run embedded Linux kernels. Even so, these dedicated systems are still generally more proficient and more secure than general-purpose computers that also forward packets.

9. CAIDA, pronounced “kay duh,” is the Cooperative Association for Internet Data Analysis at the San Diego Supercomputer Center on the UCSD campus (caida.org).

10. Many routers also support the Universal Plug and Play (UPnP) standards promoted by Microsoft, one feature of which allows interior hosts to set up their own dynamic NAT tunnels. This can be either a godsend or a security risk, depending on your perspective. The feature is easily disabled at the router if you wish to do so.

11. More specifically, it is the MAC address with the two bytes 0xFFFE inserted in the middle and one bit (bit 6 of the first byte, numbering bits from the left, starting at 0) complemented; see RFC4291. The standard for converting 48-bit MAC addresses into 64-bit IP host numbers is known as EUI-64.

12. The IP source routing feature is an exception to this rule; see page 473.

13. Except on point-to-point links, on which the identity of the destination is sometimes implicit.

14. Routers can in fact be configured to flood broadcast packets to other networks, but this is generally a bad idea. If you find yourself wanting to forward broadcasts, there is most likely something amiss with your network or server architecture.

15. Clients initiate conversations with the DHCP server by using the generic all-ones broadcast address. The clients don’t yet know their subnet masks and therefore can’t use the subnet broadcast address.

16. You can also use a split DNS configuration to achieve this goal; see page 617.

17. Broadcast storms occur because the same link-layer broadcast address must be used to transport packets no matter what the IP broadcast address has been set to. For example, suppose that machine X thinks the broadcast address is A1 and machine Y thinks it is A2. If X sends a packet to address A1, Y will receive the packet (because the link-layer destination address is the broadcast address), will see that the packet is not for itself and also not for the broadcast address (because Y thinks the broadcast address is A2), and may then forward the packet back to the net. If two machines are in Y’s state, the packet circulates until its TTL expires. Broadcast storms can erode your bandwidth, especially on a large switched net.

18. If you try this command in the form sudo echo 1 > icmp_echo_ignore_broadcasts, you just generate a “permission denied” message—your shell attempts to open the output file before it runs sudo. You want the sudo to apply to both the echo command and the redirection. Ergo, you must create a root subshell in which to execute the entire command.

19. Solaris network interfaces must be scoped out with ifconfig plumb to make them accessible. You might have to run this command by hand when performing manual configuration.

20. On HP-UX 11, the hop count field is not required; it defaults to 0 if not explicitly specified. Earlier versions required the count field to be present.

Chapter 15

1. IP packets can also be source-routed—at least in theory—but this is almost never done. The feature is not widely supported because of security considerations.

2. The problem is that changes in topology can lengthen the optimal routes. Some DV protocols such as EIGRP maintain information about multiple possible routes so that they always have a fallback plan. The exact details are not important.

3. Modern versions of IOS support a variety of access methods, including SSH. telnet, of course, is entirely insecure. If your site already uses Cisco routers, contact your network administrator to find out which methods have been enabled.

Chapter 16

1. Did you know that iPhones run a form of embedded UNIX?

2. We have omitted a few goofy Ethernet standards that have withered on the vine, such as 100BaseT4 and 100BaseVG-AnyLAN.

3. No kidding! Attaching a new computer involved boring a hole into the outer sheath of the cable with a special drill to reach the center conductor. A “vampire tap” that bit into the outer conductor was then clamped on with screws.

4. Check with your fire marshall or local fire department to determine the requirements in your area.

5. Sadly, we discovered during technical review that Easy-Bake ovens require a 100 watt light bulb (if they use a bulb at all; some now have heating elements), thus dashing the industry’s hopes for IEEE 802.3at compatibility. And for those of you that are wondering: yes, it is possible to boot a small Linux system off a PoE port. Specific hardware is left as an exercise for the reader.

6. The 600 Mb/s bandwidth of 802.11n is largely theoretical. In practice, bandwidth in the neighborhood of 400 Mb/s is a more realistic expectation for an optimized configuration. The environment and the capabilities and hardware of the client devices explain most of the difference between theoretical and real-life throughput. When it comes to wireless, your mileage may vary!

7. In fact, it will also connect to your stereo to play music wirelessly from your PC or laptop.

8. Like so many popular programs, Wireshark is often the target of attacks by hackers. Make sure you stay up to date with the current version.

9. This type of fire wall is a concrete, brick, or flame-retardant wall that prevents flames from spreading and burning down a building. While much different from a network security firewall, it’s probably just as important.

Chapter 17

1. dig and drill are DNS query tools: dig from the BIND distribution and drill from NLnet Labs.

2. The date convention also includes a two-digit change number, so you can have 99 changes per day.

3. In Windows, the client-side DNS information can be configured through the TCP/IP configuration panel for each network adapter. The exact procedure varies with the version of Windows.

4. Some sites use multiple masters or even no masters; we describe the single-master case.

5. Guaranteed here just means that the answer came from an authoritative server’s in-memory database and not from the cache of a random nonauthoritative server.

6. The in-addr.arpa portion of the name is a fixed suffix.

7. Technically, since you make the rules for your subdomain, one or more will do.

8. If a client’s /etc/resolv.conf file lists multiple name servers, the resolver should fail over to one of the backup servers. But all too often, only a single name server is configured.

9. See k.root-servers.org/statistics/GLOBAL/monthly for current data.

10. MX mail routing records fit in both the zone infrastructure pile and the basic records pile because they can refer to entire zones as well as individual hosts.

11. Actually, any name server for the zone can be listed in the SOA record unless you are using dynamic DNS. In that case, the SOA record must name the master server.

12. But really shouldn’t, for security reasons.

13. This rule for CNAMEs was explicitly relaxed for DNSSEC, which adds digital signatures to each DNS resource record set. The RRSIG record for the CNAME refers to the nickname.

14. Tony Li, an active member of the IETF community, once described IPv6 as “too little, too soon.”

15. This is a little white lie; Google and many other sites implement their SPF records with TXT records because the SPF resource record type is new and is only recently supported by popular name server software. But in looking to the future, we have taken editorial license and shown SPF records instead of TXT records. The digs were also for TXT records, not SPFs.

16. However, the given URL currently redirects to sourceforge.net’s page about DomainKeys, the old standard that has been mostly abandoned.

17. The 0 in the answer is the TTL for the data value. One of our reviewers reported once seeing the answer to this query come up as “Name is Bind, James Bind!”

18. CIDR netmasks are described starting on page 460.

19. Some firewalls are stateful and may be smart enough to recognize the DNS answer as being paired with the corresponding query of a second ago. Such firewalls don’t need help from this option.

20. You also need ingress filtering at your firewall; see page 932. Better yet, use TSIG for authentication.

21. Actually, the whole class A network 127/8 refers to localhost, but most folks just use 127.0.0.1.

22. To display the man pages without installing them, do: groff -man -T ascii man-page-filename | less.

23. The parentheses are here to show grouping; do not include them in an actual value.

24. It also has hooks for the Python scripting language.

25. Don’t make private addresses bogus if you use them and are configuring your internal DNS servers!

26. The number looks random, but it is really just a hash of the TSIG key.

27. The math involved is called the discrete log problem and relies on the fact that for modular arithmetic, taking powers is easy but taking logs to undo the powers is close to impossible.

28. GOST is secure (as far as we know) and has a much shorter key length than other algorithms. Since it is a symmetric cipher and not a public key system, it can replace TSIG but cannot be used for DNSSEC. Proposals to allow GOST are winding their way through the IETF standardization process.

29. In this section, base-64-encoded hashes and keys have all been truncated to save space and better illustrate the structure of the records.

30. 2,048 bits is surely overkill; many sites use 1,500 or fewer.

31. To make it easier to compare the BIND and NSD processes, we finagled the key footprints to make the BIND and NSD sets match. In real life, every key would have a different footprint.

32. Use a command like cat Kexample.com.+*.key >> zonefile. The >> appends to the zonefile rather than replacing it entirely, as > would. (Don’t mess this one up!)

33. The web site keylength.com tabulates a variety of organizations’ recommendations regarding the suggested lengths of cryptographic keys.

34. How can you tell if your parent is signed? Try dig +dnssec or drill -D.

Chapter 18

1. Technically, any transport protocol that implements congestion control can be used, but TCP is the only reasonable choice today.

2. Or its network database equivalent, such as NIS or LDAP.

3. Although the Red Hat NFS server defaults to UID -2, the nobody account in the passwd file uses UID 99. You can leave things as they are, add a passwd entry for UID -2, or change anonuid and anongid to 99 if you wish.

4. Of course, you should never export the root directory.

5. In reality, nfsd simply makes a nonreturning system call to NFS server code embedded in the kernel.

6. Jeff Forys, one of our technical reviewers, remarked, “Most mounts should use hard, intr, and bg, because these options best preserve NFS’s original design goals. soft is an abomination, an ugly Satanic hack! If the user wants to interrupt, cool. Otherwise, wait for the server and all will eventually be well again with no data lost.”

7. A direct map can also be managed as an NIS database or in an LDAP directory, but this is tricky.

8. The other side of this issue is the fact that it takes a certain amount of time to mount a filesystem. System response will be faster and smoother if filesystems aren’t being continually remounted.

Chapter 19

1. Though the old version disappears from the filesystem namespace, it continues to exist until all references have been released. You must also be aware of this effect when managing log files.

2. Although the password is encrypted for transmission across the network, the transferred files are not. If you use ssh as the transport (rsync -gopt -e ssh /etc/passwd /etc/shadow lollipop:/etc – note the single colon), the connection will be encrypted, but sshd will have to be configured not to require a password. Name your poison!

3. Keep in mind that both HTTP and FTP transport data in plaintext. You should consider HTTPS or SFTP, respectively, if the contents of the transferred files are sensitive.

4. Because of LDAP’s tortured history, many sources tend to go into great detail about LDAP’s X.500 and OSI connections. However, this history is not relevant to contemporary use of LDAP. Ignore it.

5. Do not confuse NIS domains with DNS domains. They are completely separate and have nothing to do with each other.

6. An ill-starred successor to the original NIS, now discontinued by Sun but still supported by some systems for historical reasons.

Chapter 20

1. Even as Evi is sailing in the middle of the ocean, she is almost always in email contact through her HAM/SSB radio and a “speedy” packet radio connection that approaches 30 baud at good times.

2. The receiving users’ mailboxes or, sometimes, a database

3. University of Washington, Seattle, WA.

4. Zimbra is not just an access agent, but rather a complete industrial-strength mail system; see page 853.

5. It’s important to note that many of the lines in the header, including the Received lines, may have been forged. Use this data with extreme caution.

6. Technically, aliases are configured only by sysadmins. A user’s control of mail routing through the use of a .forward file is not really aliasing, but we have lumped them together here.

7. We have been inconsistent with terminology in this chapter, sometimes calling a returned message a “bounce” and sometimes calling it an “error.” What we really mean is that a delivery status notification (DSN, a specially formatted email message) has been generated. Such a notification usually means that a message was undeliverable and is therefore being returned to the sender.

8. The ports that sendmail listens on are determined by DAEMON_OPTIONS; port 25 is the default.

9. sendmail can use multiple queues beneath mqueue to increase performance; see page 802.

10. So where is the OSTYPE macro itself defined? In a file in the cf/m4 directory, which is magically prepended to your config file when you run the Build script.

11. This form uses the default LDAP schema defined in the file cf/sendmail.schema; if you want a different schema file, use additional arguments in your FEATURE statement.

12. FEATURE(‘access_db’) must be there too.

13. Bastion hosts are specially hardened hosts intended to withstand attack when placed in a DMZ or outside a firewall.

14. The TRUSTED_USERS feature is typically used to support mailing list software.

15. If you try this command in the form sudo echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout, you just generate a “permission denied” message—your shell attempts to open the output file before it runs sudo. You want the sudo to apply to both the echo command and the redirection. Ergo, you must create a root subshell in which to execute the entire command: sudo sh -c "echo…"

16. Wouldn’t it be nice if standardization efforts could sort out some of these random and apparently meaningless differences so our scripts could be more portable?

17. CDB is Dan Bernstein’s constant database system; it scales well.

18. For CS wizards, it’s Turing-complete; mere mortals can substitute “powerful and complicated.”

19. require means “deny if not matched.”

20. At first glance, the fact that you get to specify a username seems flexible and nice. But because the scanning is done after the DATA command instead of at the RCPT command, the message has already qualified its recipients, and if there are several recipients, whose spam profile should you use?

21. Don’t publish your ADSP record until outbound message signing is set up and working properly, lest other sites start to reject your email.

22. Phil is an active exim-users mailing list contributor.

Chapter 21

1. If a machine hangs at boot time, boots very slowly, or hangs on inbound SSH connections, DNS should be a prime suspect. Solaris and Linux use a sophisticated approach to name resolution that’s configurable in /etc/nsswitch.conf. On these systems, the name service caching daemon (nscd) is of particular interest. If it crashes or is misconfigured, name lookups are affected. With the transition to IPv6 progressing, we find that many DSL routers provide DNS forwarding services that simply drop requests for IPv6 (AAAA) DNS records. This “optimization” causes long timeouts on all name resolution requests. Use the getent command to check whether your resolver and name servers are working properly (e.g., getent hosts google.com).

2. Even Windows has it, but the command is spelled tracert (extra history points if you can guess why).

3. We removed a few fractions of milliseconds from the longer lines to keep them from folding.

4. Connections for “UNIX domain sockets” are also shown, but since they aren’t related to networking, we do not discuss them here.

5. On UNIX systems that don’t support netstat’s -p flag, the lsof command can provide this information (and more). See page 145 for more about lsof.

6. If your filtering needs exceed tcpdump’s capabilities, consider ngrep (ngrep.sourceforge.net), which can filter packets according to their contents.

7. A “talker” is the NetFlow term for a device that creates network traffic.

Chapter 22

1. This survey is conducted yearly and can be found at gocsi.com.

2. Of course, wireless networking technology introduces a whole new set of problems. Air gap in this context means “no networking whatsoever.”

3. As described in Chapter 14, a port is a numbered communication channel. An IP address identifies an entire machine, and an IP address + port number identifies a specific server or network conversation on that machine.

4. Actually, only the privileged ports (those with port numbers under 1,024) and the well-known ports are checked by default. Use the -p option to explicitly specify the range of ports to scan.

5. Especially the passwords of system administrators who have sudo privileges

6. The protected daemons are httpd, dhcpd, mailman, named, portmap, nscd, ntpd, mysqld, postgres, squid, winbindd, and ypbind.

7. In many cases, inetd or xinetd does the actual waiting on their behalf. See page 1188 for details.

8. Port 25 is the SMTP port as defined in /etc/services.

9. This summary describes traditional FTP, also known as “active FTP.” Some systems support “passive FTP,” in which the client initiates both connections.

10. That said, many consumer-oriented networking devices, such as Linksys’s router products, use Linux and iptables at their core.

11. However, you must be careful that reordering the rules for performance doesn’t modify functionality.

12. IBM is an exception. AIX does include a separate packet-filtering suite in its IP Security implementation, although the suite does not do stateful filtering. See the man pages for genfilt to get started.

13. If system backups are not a “normal” activity at your site, you have much bigger problems than the security intrusion.

Chapter 23

1. Non-Linux UNIX distributions refer to this collection simply as “AMP.” Solaris folks call it “SAMP,” and the Windows folks call it “WAMP.” Go figure.

2. Of course, this arrangement also gives Google access to your traffic data, which may or may not be a good thing.

3. A relatively new feature called Server Name Indication (SNI) enables the use of SSL with virtual hosts, but older browsers do not support it.

4. Transport Layer Security (TLS) is the protocol that succeeds SSL and is implemented in all modern browsers. However, the web community still refers to the overall protocol and concept as SSL.

5. Why “Squid”? According to the FAQ, “all the good names were taken.”

6. Unfortunately, some sites mark all their pages as being uncacheable, which prevents Squid from working its magic. In a similar vein, Squid isn’t able to cache dynamically generated pages.

Chapter 24

1. This is not entirely true. Solaris containers have a feature called “branded zones” that allows Linux binaries to run on a Solaris kernel.

2. Estimated by Jonathan Koomey in his excellent study “Estimating total power consumption by servers in the U.S. and the world.”

3. Install the python-virtinst package for virt-install support on Ubuntu.

4. Does your CPU have them? Try egrep ’(vmx|svm)’ /proc/cpuinfo to find out. If the command displays no output, the extensions are not present. On some systems, the extensions must be enabled in the system BIOS before they become visible.

5. You can use virsh to manage Xen domUs as well, if you wish.

6. This is equally true with Xen, but xend does the heavy lifting, creating interfaces in the background.

Chapter 25

1. Xsun included support for Display PostScript, which once upon a time was thought to be the display language of the future.

2. It is possible, but not recommended, to have both environments installed simultaneously. See page 1028 for more information about desktop environments.

3. Or aixterm on AIX. Clever, hmm?

4. Note that ssh also has a -Y flag that trusts all client connections. This feature may solve some forwarding problems, but use it only with extreme caution.

5. Not all options are supported by Xorg. Some options sold separately.

6. Recall that screens identify a monitor/video card combination at a particular color depth.

7. Before using xrandr for the first time, run Xorg -configure to reset the xorg.conf file to a known, clean state.

8. The X server requires the <Control> key to be held down along with the <Alt-Fn> key combination to switch virtual terminals, but the text console does not.

9. We don’t recommend logging into X as root because this operation may create a bunch of default startup files in root’s home directory, which is usually / or /root. It’s also notably insecure. Instead, log in as a regular user and use sudo. Ubuntu enforces this discipline by default.

Chapter 26

1. Older versions of CUPS use enable and disable instead of cupsenable and cupsdisable. Unfortunately, enable is also a bash built-in command, so bash assumes you mean its own enable unless you specify the full pathname of the command. As it happens, bash’s version of enable enables and disables bash built-ins, so you can use it to disable itself with enable -n enable.

2. Keep in mind, though, that many inexpensive printers come with “starter” cartridges that include less ink or toner than a standard replacement.

3. And every program can be shortened. Therefore, as the saying goes, any program can be reduced to a single line that doesn’t work.

Chapter 27

1. At least, if you want to sleep at night…

2. The Kill A Watt meter made by P3 is a popular choice at around $20.

3. See the white paper “Power and Cooling for Ultra-High Density Racks and Blade Servers” at apc.com.

Chapter 28

1. The informational work done by IT equipment is not significant in a thermodynamic sense. Computers are essentially 100% efficient at converting electricity into heat.

2. This product is designed for the North American market, but similar products exist for other markets. A version made for the UK can be found at reuk.co.uk/Buy-UK-Power-Meter.htm.

3. This metric multiplied by 100 yields the percentage of facility power delivered to IT equipment and is known in the industry as “DCiE.” It is a standard metric that can be used to compare organizations. Power usage effectiveness (PUE) is the reciprocal of DCiE and is a common benchmark for very large data centers.

4. See eetd.lbl.gov/emills/PUBS/PDF/ACEEE-datacenters.pdf for lots of gory details.

5. NTP is a special case in that its response latency must be kept low. However, that doesn’t mean you can’t run other services on the same machine. NTP server daemons are commonly niced to give them ready access to the CPU whenever they want it (see page 129). You can achieve similar ends—perhaps even a bit more reliably—through server virtualization.

6. If your current equipment is not energy efficient, you may be better off replacing it immediately to achieve operational energy savings, even when disposal and replacement costs are considered.

7. WordPress developer Mark Jaquith wrote, “It’s like killing a person, and then convincing a murderer to kill one less person. You didn’t negate your murder. You still killed the person. Convincing someone else to reduce their emissions doesn’t make up for your emissions.” We don’t necessarily endorse this view, but it is representative the anti-offset perspective.

Chapter 29

1. Current SSDs have two main weaknesses. First, they are an order of magnitude more expensive per gigabyte than traditional hard disks. Second, they may be rewritten only a limited number of times before wearing out. Their rewrite capacity is high enough to be immaterial for desktop machines (tens of thousands of writes per block), but it’s a potential stumbling block for a high-traffic server. See page 212 for more information about SSDs.

2. Refreshing top’s output too rapidly can itself be quite a CPU hog, so be judicious in your use of top.

3. Ages ago, a second process known as “swapping” could occur by which all pages for a process were pushed out to disk at the same time. Today, demand paging is used in all cases.

4. We assume that about half of disk operations are page-outs.

5. That is, it takes a long time to switch between applications, but performance is acceptable when an application is repeating a simple task.

6. A large virtual address space or resident set used to be a suspicious sign, but shared libraries have made these numbers less useful. ps is not very smart about separating system-wide shared library overhead from the address spaces of individual processes. Many processes wrongly appear to have tens of megabytes of active memory.

7. More granular resource management can be achieved through the Class-based Kernel Resource Management functionality; see ckrm.sourceforge.net.

Chapter 30

1. HP also offers a Samba derivative called the HP CIFS Server, which is available for download from the HP software depot.

2. Windows passes the current user’s credentials to the Samba server when establishing a connection. For this reason, users’ Samba passwords are usually set to match their Windows passwords.

3. Type echo $LANG to see if your system is running in UTF-8 mode.

4. NFSv3 security is based on the idea that the user has no root access on the client and that UIDs match on the client and server. This is not normally the case for self-managed machines. NFSv4 does better UID mapping than NFSv3 and is dramatically more secure.

Chapter 31

1. To be technically correct, this standard should now be referred to as EIA-232-E. However, no one will have the slightest idea what you are talking about.

2. .profile for sh and ksh; .bash_profile and .bashrc for bash; .cshrc and .login for csh/tcsh.

Chapter 32

1. OK, maybe you don’t need this role in your IT group. But it is an industry standard.

2. As of 2005, about half of security breaches originated with insiders.

3. If you plan to do business with a U.S. government agency, you may be required to complete a NIST 800-53 assessment whether you want to or not…

A Brief History of System Administration

1. Although SHARE was originally a vendor-sponsored organization, today it is independent.

2. GECOS was the General Electric Comprehensive Operating System.

3. Minix is a PC-based UNIX clone developed by Andrew S. Tanenbaum, a professor at the Free University in Amsterdam.

4. Just for the record, Windows is indeed less filling.

5. Even Apple’s iPhone runs a stripped-down cousin of UNIX, and Google’s Android operating system includes abstractions from the Linux kernel.