Chapter 5. The Boot Process

Single-user mode
unscheduled in the nighttime?
Something just went “boom”!

In order to properly manage any computing platform, you must understand the boot process. Many systems administration tasks cannot be done while the system is running. OpenBSD specifically requires that certain tasks be done before the boot process has completed. And, of course, on any operating system, sometimes a process starting up prevents the system from completing its boot. The only way to fix these problems is to interrupt the boot partway through.

First, we’ll look at the key to OpenBSD’s booting process: the boot loader. Then we’ll move on to single-user mode, and finally multiuser startup. You can perform useful work at any of these stages.

I recommend playing with the OpenBSD boot process on a test machine before one of your machines won’t boot. That way, when something breaks in the wee hours of the morning, you can spend your time fixing the problem instead of fumbling around with unfamiliar commands.

In general, when a PC-style computer first boots, it starts the BIOS. The BIOS is a small piece of software that figures out things like which drives are attached and what they’re attached to, what sort of CPU is installed, and how much memory is available. After getting that information, the BIOS loads a minimal boot loader from some kind of storage device.[11]

The boot loader is a small program that handles initial system configuration and boots the kernel. It finds and starts the kernel, which in turn detects hardware, attaches device drivers, and performs other core setup. Finally, the kernel calls init(8), which starts processes and enables user programs, network interfaces, server software, and so on.

While most of this process cannot be managed—no one actually configures init!—there’s plenty you can do before the system finishes booting and dumps you at the login screen.

The OpenBSD boot loader lets you interrupt the boot process, configure the system before it boots, adjust kernel settings, and even boot an alternate kernel.

When the hardware hands control of the boot process over to the OpenBSD partition, you’ll see the boot loader prompt, which looks something like this:

>> OpenBSD/amd64 BOOT 3.18
boot>

The boot loader’s main purpose is to find the kernel, load it into memory, and start it. Because it runs before the kernel starts, the boot loader can pass instructions to the kernel itself.

Here are some of the things you can do before booting is complete:

Use built-in help

Use the help function to print a brief list of commands that the boot loader supports.

boot> help
commands: # boot echo env help ls machine reboot set stty time
machine: boot diskinfo memory
Delay the boot process

By default, the loader waits five seconds for instructions, and then boots the kernel. To pause the boot at the prompt, press the spacebar.

We’ll use various permutations of boot to configure the kernel, boot single-user mode, and so on. I’ll cover other boot commands in the appropriate sections. For full details on what you can do at the boot loader prompt, read the boot(8) man page.

Single-user mode is the earliest point when OpenBSD can give you a Unix-style shell prompt. At this point, the kernel has probed all the hardware, attached drivers to all the hardware that it’s going to acknowledge, and started init. The system hasn’t mounted any filesystems except for the root partition, which is mounted in read-only mode. The network isn’t started, no services are running, security is not implemented, and filesystem permissions are ignored.

To boot OpenBSD in single-user mode, enter boot -s at the loader prompt.

boot> boot -s

Why would you want to boot into single-user mode? If your computer has a problem that is preventing it from booting, you should be able to access single-user mode and fix the problem. Suppose a failed disk is preventing the system from booting during a multiuser boot, or you changed your terminal settings in /etc/ttys and now you can’t log on to the system. Or maybe you put a daft setting in rc.conf.local, and the boot process hangs because it’s trying to do something impossible. At times like these, single-user mode is your best friend.

Also, some system administration tasks, such as clearing filesystem flags (see Chapter 8), can be done only in single-user mode.

Mounting Disks in Single-User Mode

Usually, you should have a fully functional filesystem before doing anything in single-user mode. If your system crashed, be sure to check the integrity of your filesystems before mounting them:

# fsck -p
/dev/sd0a (e4bf0318329fe596.a): file system is clean; not checking
/dev/sd0h (e4bf0318329fe596.h): file system is clean; not checking
…
# mount -a

fsck and mount have many more options. We’ll cover them in more detail in Chapter 8.

Once you’ve mounted all of your filesystems, all usual command-line software should be available. You should be able to edit configuration files, start and stop programs, and generally do whatever you like to the system (including destroy it).

Starting the Network in Single-User Mode

Use the shell script /etc/netstart to configure the network while in singleuser mode. (You could run all the appropriate commands by hand, but /etc/netstart will read your system’s configuration files and do the grunt work for you.) You must explicitly run this script through sh:

# sh /etc/netstart

If you’re booting into single-user mode because of network problems, this script will conveniently reproduce the issue for you.

As we’ll cover in tedious detail in Chapter 18, you can configure the OpenBSD kernel, but before you do so, be sure that you can boot alternate kernels. You’ll need to be able to boot a different kernel if, say, you hose your filesystem so badly that it won’t even boot to single-user mode, and you need to recover using the installation kernel.

Booting a Different Kernel File

An OpenBSD installation includes three kernels out of the box: the single-processor kernel /bsd, the multiprocessor kernel /bsd.mp, and the upgrade and install kernel /bsd.rd. (If your machine has multiple processors, the installer renames /bsd to /bsd.sp and /bsd.mp to /bsd.)

To boot a nonstandard kernel, first reboot and interrupt the boot process at the boot loader prompt. Run boot, and give the full path to the kernel you want to boot:

boot> boot /bsd.rd

This will start the system using your chosen kernel. You can use other boot options as well, such as booting the alternate kernel in single-user mode:

boot> boot -s /bsd.sp

This will let you recover from a bad kernel, try a new kernel, or anything in between.

Suppose you’ve really fouled everything up beyond all recognition, and you don’t have a usable kernel on your root partition. Fortunately, if you have a usable kernel on a different hard drive, you can boot from that. (Usually, this kernel lives on an alternate root partition, /altroot, as discussed in Chapter 8.) In this section, I’ll break the task of booting from that alternate kernel into a few steps: finding the hard disk with the partition, finding the partition with the file, and booting the right file on that partition.

To make boot loader options permanent, edit /etc/boot.conf. The boot loader reads and runs entries from this file before giving you the boot> prompt, which means you can use it to automatically run boot loader commands every time your computer boots. (Although if you would rather sit at your computer and enter your settings every time you reboot, don’t let me stop you.)

Any command you might give at the loader prompt is a valid boot.conf entry. For example, if the default boot speed is too slow for your liking, you can set your boot timeout to two seconds by adding this line to boot.conf:

set timeout 2

You can also tell the system to boot a different kernel with the correct boot.conf command.

set image /bsd.mp

By far, boot.conf is most often used to configure a serial console.

All of these nifty boot functions let you do useful stuff when your system is in trouble, but how can you use them when your computer isn’t right in front of you? If your computer is in a data center on the other side of the country, or sitting in the basement behind the last decade of payroll records, a serial console will make your life far more pleasant.

A hardware serial console allows you to run a serial cable between a computer and a terminal server (on another computer) to access BIOS messages and operating system boot and startup messages which simplifies managing remote systems. Serial consoles are invaluable when debugging system crashes, too; error messages come over the serial port, where you can easily capture them.[12]

True UNIX hardware has serial console capabilities, as does most server-grade i386 and amd64 hardware. Most desktop-grade hardware, however, does not. But fortunately, even if you don’t have a hardware serial console, you can access all of OpenBSD’s startup messages with a serial port and a software serial console. While OpenBSD’s software serial console won’t give you access to the hardware BIOS, it will let you interface with the boot loader and remotely access the system console, even when the network is down.

Other Platform Serial Consoles

Every hardware platform has its own standards for serial consoles. If you’re running a less common platform, check your hardware’s documentation.

If your hardware supports a real serial console, you should usually configure it in the BIOS. OpenBSD supports whatever the hardware supports, so your Sparc64 hardware will support OpenBSD’s serial console just as well as it supports any other operating system’s serial console.

Serial Console Physical Setup

A serial console requires a null modem cable, which you should be able to get from any computer store or an online vendor. While gold-plated cables aren’t worth the money, don’t buy the cheapest cable you can find either. If you have an emergency and need the serial console right now, you won’t be in the mood to deal with a defective cable.

Plug one end of the null modem cable into your OpenBSD machine’s first serial port. (The serial console is supported on only the first serial port, or com0 on i386 and amd64 hardware.) Plug the null modem cable’s other end into an open serial port on another system. (For simplicity’s sake, use either another OpenBSD or Unix-like system.)

If you have two OpenBSD machines at a remote location and you want to use serial consoles on both, you can have each machine act as the console client for the other. Attach the first serial port on each server to the second serial port on the other. If you have three machines, you can daisy-chain them in a loop. If you have four or more machines, pick up a used terminal server from your favorite auction site.

You can also use two DB9-to-RJ45 converters, one standard and one crossover, which will allow you to run your console connections over a standard CAT5 cable. If you have a lights-out data center where human beings are forbidden unless they are installing or removing equipment, you can stretch your serial console cables about 12 meters, which should reach into your warm room. (Most modern data facilities are better equipped to handle CAT5 cables than serial cables.)

Now that you have the console physically ready, the next step is to configure your client to access the serial console. Then you can set up the serial console.

Newer serial ports (meaning anything made within the past 10 years) can run at speeds far above 9600 baud. I have servers with serial consoles that run only at 115,200 baud. The BIOS messages display at 115,200 baud, but then the OpenBSD console runs at 9600 baud. My client displays one or the other as gibberish. (A lot of OpenBSD folks think that anything that won’t do serial at 9600 baud is broken, but you won’t always have control over the hardware you work with.)

To use these ports, I can either change my connection speed in my serial console client when switching between the BIOS messages and the OpenBSD messages, or change the speed of my OpenBSD console to match the hardware.

At the boot loader, tell the serial console to run at 115,200 baud:

boot> stty com0 115200
boot> set tty com0

If these settings work, copy them to /etc/boot.conf.

Now configure your serial client. Modify tip to use the higher speed. First, find the entry for tty00 in /etc/remote:

tty00|For hp300,i386,mac68k,macppc,vax:\
        :dv=/dev/tty00:tc=direct:tc=unixhost:

But don’t modify this entry! We’ll use it to illustrate the style of /etc/remote entries.

Backslashes (\) in this entry mean “continued on the next line.” Colons separate fields. Each line after the first must start with a colon, and each field is a key/value pair.

Now, to create a console entry that runs at 115,200 baud, use the following:

console:br#115200:tc=tty00:

The first field in an /etc/remote entry is the name, and every entry must have a unique name. I named this entry console. The second field is the br value. According to remote(5), br stands for bit rate. I’ve set the bit rate to 115,200 baud. The third field is tc, for “table continues,” which is equal to tty00. This means that the description of this entry continues in entry tty00.

Taken as a whole, this entry means “copy the tty00 entry, and add a bit rate of 115,200.”

When the kernel finishes its core setup and hands control over to userland, init(8) runs the shell script /etc/rc. This script handles all system setup, including mounting filesystems, configuring device nodes, identifying shared libraries, and any other task required to make the system usable. Some tasks are delegated to separate scripts; for example, /etc/netstart is used to configure the network.

In this section, we’ll cover how /etc/rc and other startup scripts function, and the flow of the startup process. Armed with this understanding, you should be able to easily configure your OpenBSD machine to start exactly what you need—no more, no less.

The startup system includes the scripts /etc/rc, /etc/rc.conf, /etc/rc.conf.local, /etc/netstart, /etc/rc.securelevel, /etc/rc.local, /etc/rc.shutdown, /etc/rc.firsttime, /etc/fastboot, and the contents of the /etc/rc.d directory.

The /etc/rc.local Script

After /etc/rc does just about everything else, it runs /etc/rc.local. You can put commands to start local daemons in rc.local, but you’re better off writing an rc.d script to start local daemons so you can easily and consistently restart them later. Of course, if you’re lazy, you can get by with rc.local.

The /etc/rc.d Directory

The /etc/rc.d directory contains shell scripts for managing software, as discussed in the next section. While the system comes with scripts for software included in OpenBSD, add-on packages can provide their own scripts (see Chapter 13).

Software Startup Scripts

OpenBSD uses shell scripts to start, stop, restart, check, and reconfigure server software. These scripts are found in the directory /etc/rc.d. Every piece of server software that comes with OpenBSD has a script in this directory, as do most ports and packages that need scripts for proper startup and shutdown. Use these scripts to manage integrated software without rebooting the server.

The rc.d scripts read their configuration from rc.conf and rc.conf.local. Most servers run the SSH daemon sshd, which can be enabled by adding the line sshd_enable="" to rc.conf.local. Look in /etc/rc.d, and you’ll find the shell script sshd.

If you change your sshd configuration, you must restart the daemon. Use the shell script to do this consistently.

# cd /etc/rc.d/
# ./sshd restart
sshd(ok)
sshd(ok)

Of course, you could do the same thing without the shell script simply by identifying the currently running sshd(8) process, reading the man page to see how to shut it down properly, and then restarting it with the same command-line flags. In the case of sshd, that’s easy: Running pkill -1 sshd would tell the daemon to reread its configuration file. But restarting a daemon that requires all sorts of flags is a big deal. Automating these system administration tasks ensures that your daemons run consistently.

To see if a daemon is running, use the check command to check your shell for the return value. The script will return a 0 if the daemon is running and a 1 if it isn’t, as shown here:

# ./nfsd check
# echo $?
1

As you can see by the 1, nfsd is not running.

The most common use for check is in shell scripts. You can start the daemon with the argument start and terminate it with stop. Use the restart argument to tell the daemon to reload its configuration.

In OpenBSD, rc.d scripts run when the system boots and again when it shuts down. (Something needs to unmount all those hard drives, shut down daemons, and clean up.) At shutdown, every script in the /etc/rc.d folder is called with the stop argument.



[11] On i386 and amd64 systems, this is where the MBR comes in.

[12] Granted, a remote keyboard-video-mouse (KVM) system can give you all of this, but very few KVM applications let you copy and paste text from the remote console. That means you’ll need to copy error messages by hand.