If you've done a good job of designing a firewall that fits the needs of your organization, maintaining that firewall should be fairly straightforward. What does it mean to maintain a firewall? Maintenance tasks fall into three major categories:
Housekeeping
Monitoring your system
Keeping up to date
Once you've designed and built your firewall, it really shouldn't take a great deal of effort to keep it going, especially because much of the maintenance work can be automated.
Housekeeping is the eternal round of small tasks that need to be done to keep your firewall clean and safe. There are three main tasks you'll need to deal with again and again:
Backing up your firewall
Managing your accounts
Managing your disk space
Make sure to back up all parts of your firewall. That means not only the general-purpose computers you may be using as bastion hosts or internal servers, but also the routers or other special-purpose devices. Rebuilding router configurations usually isn't easy, and your security depends on having your routers configured correctly.
Put your general-purpose machines on a regular, automated backup system. Preferably, that system should produce confirmation mail when it is running normally and distinctly different messages when it sees errors.
Why not produce mail only when errors occur? If the system produces mail only on errors, you won't notice the system if it fails to run at all. (Silence is not necessarily golden, as any parent of small children knows. If they aren't making noise, they're probably making mischief.)
Why distinctly different messages? If the system produces even vaguely similar messages when it is running normally and when it fails, people who are accustomed to ignoring the success messages will also ignore the failure messages. Ideally, a separate program should check to make sure that the backups have run and to produce messages when they haven't.
Special-purpose machines like routers change much less often and probably don't need an automated backup system. (This is fortunate because such machines rarely support them.) When you do make changes, take advantage of any means available to record the configuration. Most systems write their configuration to flash memory and can transfer configurations via FTP. Some of them also have floppy drives. If you can write the configuration to floppy, do so, and store a floppy separate from the machine. Make backups even if you have downloaded the configuration with FTP; you don't want the router to be completely dependent on another machine. If you didn't download the configuration with FTP, make an FTP copy as well as the floppy disks. Why? Sometimes it's easier to find files than to find small physical objects like floppy disks, and sometimes the floppy drive dies when the rest of the router still works. If you cannot write a floppy disk or another backup that the router can read directly, you should at least be sure that the necessary files for FTP are available from multiple places.
The design of backup systems is outside the scope of this book. This description (along with the section in Chapter 27, called "Backing Up Your Filesystems") provides only a summary. If you're uncertain about your backup system, you'll want to look at a general system administration reference. See Appendix A, for complete information on additional resources.
Account management — adding new accounts, removing old ones, changing passwords, etc. — is one of the most often neglected housekeeping tasks. On firewall systems, it's absolutely crucial that new accounts be added correctly, old accounts removed promptly, and passwords changed appropriately. (See your own system's documentation for how to do all this.)
Establish a procedure for adding accounts; wherever you can, use a program to add them. Even though there shouldn't be many users on your firewall systems, every one of them is a possible danger, and it's worth the effort to ensure they're set up correctly every time. People have an unfortunate tendency to leave out steps or to pause for a few days in the middle of a process. If that gap leaves an account that has no password, you're creating open invitations to intruders.
Make sure your account creation procedure includes dating the account and that accounts are automatically reviewed every few months. You don't need to automatically turn them off, but you do need to automatically inform somebody that they've timed out. It's relatively easy to do on a general-purpose computer; it may be harder on other systems, particularly dedicated systems like routers. If possible, set things up so that the accounts can be watched by an automated system. This can be done by generating account files on a general-purpose computer and then transferring them to the other machine, or by generating the accounts on the machine itself, but automatically copying the account files to a general-purpose computer and examining them.
If your devices support reasonably secure protocols that will allow you to centrally maintain accounts, you should consider using them. However, doing user authentication via NIS or Windows domain authentication on firewall machines is not advisable for security and reliability reasons.
You should also arrange to get termination notices from the appropriate authorities whenever someone leaves your organization. Most companies are able to send notices for full-time employees, and most universities can provide graduation notification for students. It may be much harder to keep track of contractors and students who drop out, so you shouldn't rely on official notifications to tell you about everybody who has left. You may also need to confirm notifications: for example, you may get termination notices for people who are actually converting to contract status, or graduation notices for people who are continuing as graduate students or junior faculty. These people are going to be annoyed if you get rid of all their files (although it's probably acceptable to temporarily disable their accounts if their status is in doubt).
If you can, set up all accounts on the firewall to have nonreusable passwords (see Chapter 21, for more information on nonreusable password systems). This will prevent people from either guessing or sniffing passwords and may help limit people's ability to share passwords with each other.
If you are using reusable passwords, or nonreusable passwords with a fixed component, encourage people to change them regularly to help prevent guessing and limit the usefulness of password sniffing. If your operating system supports password aging, you may want to turn it on. Use a relatively long time period — perhaps three to six months. If you time out passwords more frequently (e.g., every month), users will be willing to go to great lengths to circumvent the timeout, and you probably won't see any real gain in security. Similarly, if your password aging doesn't guarantee that the user will see a notification before the account becomes unusable, don't turn it on. Otherwise, you will annoy your users, and you will run the risk of accidentally locking out administrators who have a critical need to use the machine.
If password aging on your system is going to require users to change their password as they are logging in, you need a password program that strictly enforces good passwords. If you don't do this, people will choose simple passwords in the heat of the moment, honestly intending to change them to better ones later. All in all, you may find it more effective to simply send people notices on a regular basis, even though you'll get less compliance that way.
Data always expands to fill all available space, even on machines that have almost no users. People dump things in odd corners of the filesystem "just for now", and they build up there. This causes more problems than you may realize. Aside from the fact that you may want that disk space, this random junk complicates incident response. You'll end up asking yourself:
Is that a program that you left lying around last time you needed to install a new version, or did an intruder put it in?
Is that truly a random data file, or does it have some deep meaning to an intruder?
Unfortunately, there is no automatic way to find junk; human beings, particularly system administrators who can write anywhere on the disk, are too unpredictable. Another person needs to look around on a regular basis. It's particularly effective to send every new system administrator on a tour of the disks; they'll notice things the old hands have become accustomed to.
Auditing programs like Tripwire, discussed in Chapter 10, will tell you about new files that appear in supposedly static areas, and this information will help you keep things straight. You will still need to check all the areas where you intentionally allow changes, and you should periodically go back and re-check the static areas. You will probably get the alert while you still know why you put that file in that place, and that knowledge may wear off over time.
Aside from accumulating junk, your main disk space problem will be logs. These can and should be rotated automatically, and you may want to compress, encrypt, or digitally sign them as well. A program like trimlog (see Appendix B) can help automate the process. You should also consider making a copy of the files on another system.
It is very important to correctly rotate or truncate logs. If a program is trying to write to a log file while you're trying to move or truncate it, you're obviously going to have problems. In fact, you may run into difficulties even if a program is simply holding the file open in preparation for writing to it later; notably, you may later discover that the program is still logging to the file you renamed.
Under Unix the normal convention for rotating log files is as follows:
Rename the log file.
Create a new and empty file using the original name (and the same file permissions).
Signal the program writing the logs in some way.
Signaling could involve sending a Unix signal or creating a dummy file that the program checks for.
Most Windows NT programs use the Event Logger to deal with logging. As we discussed in Chapter 12, the Event Logger provides no way to rotate logs, which means that it has to be done by hand or with third-party software. The situation with programs that keep their own logs is not so clear; you will normally need to use a special accompanying tool to rotate a program's log files or configure the program to rotate the logs based upon some point (for instance, weekly or when a particular size has been reached).
Under both operating systems, it is unfortunately sometimes necessary, particularly when using add-on products, to stop programs or cause them to suspend logging while you truncate or move logs.