Before we discuss the details of firewalls, it's important to understand some of the basic strategies employed in building firewalls and in enforcing security at your site. These are not staggering revelations; they are straightforward approaches. They're presented here so that you can keep them in mind as you put together a firewall solution for your site.
Perhaps the most fundamental principle of security (any kind of security, not just computer and network security) is that of least privilege. Basically, the principle of least privilege means that any object (user, administrator, program, system, whatever) should have only the privileges the object needs to perform its assigned tasks — and no more. Least privilege is an important principle for limiting your exposure to attacks and for limiting the damage caused by particular attacks.
Some car manufacturers set up their locks so that one key works the doors and the ignition, and a different key works the glove compartment and the trunk; that way, you can enforce least privilege by giving a parking lot attendant the ability to park the car without the ability to get at things stored in the trunk. Many people use splittable key chains, for the same reason. You can enforce least privilege by giving someone the key to your car but not the key to your house as well.
In the Internet context, the examples are endless. Every user probably doesn't need to access every Internet service. Every user probably doesn't need to modify (or even read) every file on your system. Every user probably doesn't need to know the machine's administrative password. Every system administrator probably doesn't need to know the administrative passwords for all systems. Every system probably doesn't need to access every other system's files.
Unlike car manufacturers, most operating system vendors do not configure their operating systems with least privilege by default. It is common for them to be in a "most privileged" mode when connected to a network out of the box or during an operating system installation. Applying the principle of least privilege suggests that you should explore ways to reduce the privileges required for various operations. For example:
Don't give a user administrative rights for a system if all she needs to do is reset the print system. Instead, provide a way to reset the print system without administrative rights (under Unix, it involves a special program of some sort; under NT, it involves giving that user the privileges required, usually by making the account a member of the Print Operators group).
Don't make a program run as a user with general privileges if all it needs to do is write to one protected file. Instead, make the file group-writable to some group and make the program run as a member of that group rather than as a highly privileged user.
Don't have your internal systems trust one of your firewall machines just so it can do backups. Instead, make the firewall machine trust the internal system, or, better yet, put a local tape drive on the firewall machine so that it can do its own backups.
Many of the common security problems on the Internet can be viewed as failures to follow the principle of least privilege. For example, any number of security problems have been and continue to be discovered in Sendmail, which is a big, complex program; any such program is going to have bugs in it. The problem is that Sendmail runs (at least some of the time) setuid to root; many of the attacks against Sendmail take advantage of this. Because it runs as root, Sendmail is a high-value target that gets a lot of attention from attackers; the fact that it's a complex program just makes their jobs easier. This implies both that privileged programs should be as simple as possible and that, if a complex program requires privileges, you should look for ways to separate and isolate the pieces that need privileges from the complex parts.[1]
Many of the solutions you'll employ in protecting your site are tactics for enforcing the strategy of least privilege. For example, a packet filtering system is designed to allow in only packets for the services you want. Running insecure programs in an environment where only the privileges the programs absolutely need are available to them (e.g., a machine that's been stripped down in one way or another) is another example; this is the essence of a bastion host.
There are two problems with trying to enforce least privilege. First, it can be complex to implement when it isn't already a design feature of the programs and protocols you're using. Trying to add it on may be very difficult to get right. Some of the cars that try to implement least privilege with separate keys for the trunk and the ignition have remote trunk release buttons that are accessible without the keys, or fold-down rear seats that allow you to access the trunk without opening it the traditional way at all. You need to be very careful to be sure that you've actually succeeded in implementing least privilege.
Second, you may end up implementing something less than least privilege. Some cars have the gas cap release in the glove compartment. That's intended to keep parking lot attendants from siphoning off your gas, but if you lend a friend your car, you probably want him or her to be able to fill it up with gas. If you give your friend only the ignition key, you're giving your friend less than the minimum privilege you want him or her to have (because your friend won't be able to fill up the gas tank), but adding the key to the trunk and the glove compartment may give your friend more privilege than you want.
You may find similar effects with computer implementations of least privilege. Trying to enforce least privilege on people, rather than programs, can be particularly dangerous. You can predict fairly well what permissions a mail server is going to need to do its job; human beings are less predictable and more likely to become annoyed and dangerous if they can't do what they want. Be very careful to avoid turning your users into your enemies.
[1] It's important to realize that Sendmail is far from the only example we could cite; you can find similar problems in almost any large, complex, privileged piece of software.