When Unix was first developed at the AT&T Labs, the Internet as we now know it was not around. Most computer systems lived in virtual isolation from each other, with people directly connecting to the system that they wanted to use. As the years progressed and the new ARPA network started taking root, systems became more interconnected. In the early days, the only people that were on this new network were scientists and government labs. Because everyone on the network felt they could reasonably trust everyone else, security was not an overriding design goal of the protocols that were being created at the time, such as FTP, SMTP, and Telnet.
Today, with the current level of Internet connectivity, this level of trust is no longer sufficient. The computer world has become a hostile environment; you must now take more care when setting up a public server. Unfortunately, many of these insecure protocols have become deeply rooted and are proving difficult to replace. In this chapter, we will discuss some of the issues you need to be aware of when creating a server, and some of the solutions for securing it.
We will be discussing several utilities and applications in this chapter. By downloading these ahead of time and putting them onto a portable media, such as a CD, you can install them on your servers without having to connect to the network. These programs are listed here.
TIP Many programs, especially ones that deal with security, are delivered with a checksum for integrity checking. Usually these checksums are created with the program md5sum. It is a wise precaution to verify that the programs you download match the checksum provided by the author. See the man page on md5sum for more information on checksums, and be sure to read the web sites for checksum information pertinent to the applications you’re downloading. Once you’ve verified the checksum, copy the file to a read-only media so you do not have to verify it again.
Before proceeding any further with securing your system, you should be 100 percent positive that nobody has installed rogue daemons, Trojan horses, rootkits, or any more nasty surprises on your system. If the system has been connected to a network or had unsupervised users, you cannot make that guarantee.
Always start with a freshly installed operating system. Disconnect your server from the network and boot it from the supplied media from your vendor. If given the option, always choose to do a complete format of the connected drives to be sure that they do not contain malicious content. Then you can install the operating system.
If this is not feasible in your situation, all is not lost, but it will require more diligence and effort on your behalf. You will need to do a complete audit of your system—applications, ports, and daemons—to verify you’re not running any rogue processes or unnecessary services, and to ensure that what you are running is the same as what was installed. Take an unused server, perform a clean install of your operating system on it, and use that server to compare files. Do not put a server out on the network (either internal or external) without making sure there are no back doors open.
The easiest way to compare files on the two machines is to run this command:
on each system (where machine is replaced with the name of the machine) and copy both of these files into the same directory on a third system. Then use the diff command to get a list of differences:
You then have to go through the diff output line by line and verify each difference. It is an exhausting and tedious process and is useful only as a last resort when you cannot create a clean system from scratch.
If diff reports a change between files, use a trusted and verified checksum utility such as md5sum on each file. To ensure that the checksum utility is viable, copy it from a fresh install onto read-only media, and use it from that media only. Compare the checksums provided to determine whether a difference exists. If the trusted host contains a different checksum, replace the file on your server with the file from the trusted host.
Keep a journal for each server you maintain. Any time you make a change to that server, note it in your journal detailing date, time, and what you changed. If your changes included updating, replacing, or otherwise modifying files, keep track of the updated checksums for those files for future reference.
Most modern operating systems are written with the expectation that they will be utilized in a networked environment. To that end, many network protocols, applications, and daemons are included with the systems. While some systems are good about disabling the included services (Mac OS X, for example), others activate all of them and leave it to you to disable the ones you do not want. This setup is inherently insecure, but it is becoming less common.
Most System V (SysV) Unix platforms will scan one or more directories on system startup and will execute all the scripts contained within these directories that match simple patterns. For example, any script that has execute permissions set and begins with the letter S (capitalized) will be run automatically at system startup. The location of these files is a standardized directory tree that varies slightly by vendor, as shown in the following list.
As you can see, the locations are predictable, with minor variations. Go through your startup directories and examine each file. If a script starts up an application or daemon that you are not familiar with, read the associated man pages until you understand the service it provides.
NOTE BSD Unix systems use a variation where the main configuration files are /etc/defaults/rc.conf and /etc/rc.conf. The /etc/rc script runs the /etc/rc.* files in their proper order, loads the configuration, and starts the system boot sequence.
When you have finished taking inventory, make a list of what is being started and then rule out the processes you do not need. To stop a script from executing, rename the script to break the naming convention by prepending “nostart.” to the script name; for example, /etc/rc2.d/nostart.S99dtlogin.
Be careful not to remove scripts that are essential to the operation of your server. If you are not certain whether a script is needed, disable it first and then reboot your server. Watch the startup and make sure that the script did not run; then do stringent testing of your server to verify that it is usable and performs the tasks required. Follow these same steps for each script you are not certain you need.
If your operating system did not ship with any SSL libraries, install OpenSSL. The OpenSSL suite is a set of encryption libraries and some applications to make limited use of them. The main power of OpenSSL comes from the ability of many networking applications and daemons to link the libraries and provide network encryption of your data. For example, Apache uses OpenSSL to serve https web pages, and OpenSSH uses OpenSSL as the foundation it builds upon.
To get OpenSSL, go to www.openssl.org. The latest version as of this writing is 0.9.7b. Download the tar file to /tmp, and then unpack the source and compile it as follows:
OpenSSL will create a /usr/local/ssl directory by default and install all of its files in that directory. Unless you have a good reason why the files should not be placed in that directory, leave them there. Many applications know to look for them in that location.
If the command make test did not produce any errors, you have a good version of OpenSSL installed and ready to be incorporated into your future security plans. OpenSSL provides a secure foundation for many types of protocols, including, but not limited to, ssh, pop3, imap, and https. As such, it is a valuable tool to place into your security arsenal. If you will not be running any of the secured services that it can provide, do not install it. It is one more item you need to keep track of for vulnerabilities and patches.
NOTE The OpenSSL installer as of version 0.9.7b is broken and cannot handle a source directory path with spaces in it. For example, “/tmp/Security Directory/openssl-0.9.7b” would cause the make install command to fail. Make sure there are no spaces in your path.
The Internet, as stated previously, was a much friendlier place when it was first being developed. Security was not at the forefront of anyone’s mind when connection protocols were being created. Many protocols transmit all data without encryption or obfuscation; the data they wish to send is exactly what they send. While this works and is fine in a completely safe environment, it is not a good idea when you’re sending sensitive information between systems.
As an example, let’s suppose we’re using computer A in Chicago and we wish to connect to a remote computer B in London, UK, to check some information we have stored there. We open a shell on A and run telnet:
You are now logged into your server in London. Unfortunately, your login ID and password were just sent in clear, unencrypted text halfway around the world. Anyone who has compromised a router anywhere in your path, or breached security in other ways, has just obtained your information. This scenario unfortunately happens all too often, but it can be avoided.
Many of the unsecured protocols, such as Telnet, FTP, and the r* commands (rsh, rexec, rlogin, etc.), can be replaced with OpenSSH to provide similar functionality but with much higher security. Let’s look at the example again using OpenSSH.
We have just accomplished the same task—logging into a server in London—but none of our personal information was readily available. OpenSSH uses OpenSSL to transparently encrypt and decrypt all information that is sent. While everything looks the same to you, an eavesdropper on the Internet will not get any useful information from this session.
Using OpenSSH requires that the SSH daemon (sshd) is installed and running on the remote server and the SSH client (ssh) is installed on the local system. To get OpenSSH, go to www.openssh.org. Look at the left column and click on your operating system under the “For Other OS’s” heading unless you are running OpenBSD. The current version as of this writing is 3.6.1, and it requires that you have OpenSSL installed before installing OpenSSH.
TIP Before downloading and installing OpenSSH, make sure that your vendor has not already supplied it with the operating system. As more people are starting to consider security, more vendors are starting to include secured protocols.
Download the tar file to /tmp, and unpack and install it.
OpenSSH by default will put sshd in /usr/local/sbin, ssh in /usr/local/bin, and sshd_config in /usr/local/etc. To start the sshd daemon, edit your sshd_config file and check the default settings. Any changes you make in this file will only be read when starting the daemon.
Once sshd_config is to your liking, start sshd as root using the command /usr/local/sbin/sshd. Test your daemon with the command /usr/local/bin/ssh localhost. If you are prompted for your password, OpenSSH is installed and running.
Create a startup script to launch sshd on system startup so it is always available, and then deactivate telnet and the r* commands from /etc/inetd.conf. Send inetd a HUP signal to have it reread the configuration file by finding the PID of inetd and typing “kill –HUP PID” (with PID substituted by the pid of inetd on your system), and your server will no longer accept unencrypted remote logins.
To connect to a remote machine with ssh instead of telnet, the general form of the command is ssh user@machine where user is your user ID on the remote system machine.
To use Secure FTP (SFTP) instead of FTP, first comment out the ftp line in /etc/inetd.conf to disable FTP service. Then use the command sftp user@machine to connect to the remote machine and transfer files. Using SFTP instead of FTP provides not only the benefit of your username and password being encrypted, but all of the data that you transfer is also encrypted. This makes SFTP a good link in your chain for transmitting sensitive information to a remote server.
By using OpenSSH in this manner, you can disable the FTP, Telnet, and r* services, and you can close ports 20, 21, 23, and the many ports used by the r* services on your firewall to stop this type of traffic from ever getting through to your server. Port 22 will be substituted for all of these ports.
Many services running on your server do not need root access to perform their functions. Often, they do not need any special privileges other than the ability to read from—and possibly write to—the data directory. But due to the Unix security measure that states only processes run by root can open a TCP/IP port below 1024, coupled with the fact that most of the well-known ports are below 1024, means that your daemons must be started as root to open their ports.
There are a few workarounds to this dilemma. The first and safest is not to run that service at all. If the daemon isn’t running, then it does not need to run as root. However, this is not always practical. Sometimes you need to run the service provided by the daemon.
In that event, create a dedicated user ID to run the daemon, and make it as restrictive as possible. Make only the directory used by that ID writable by that ID, and give the ID no special elevated permissions. Then change the startup script so that the daemon is owned by this new user ID.
The biggest problem you will run into with this solution is that you will not be able to open a port below 1024. You will have to change the port used to a number greater than 1024. For example, if you create a new ID to run your POP3 service, which is defined at port 110, you could change your service to run at port 1100.
To change the port assignment, first determine whether the daemon is running stand-alone or is called by inetd. To check that, first scan the /etc/services file and look for the port number used by the service by default. The /etc/services file uses the format “service_name port/protocol” where service name is a human readable name assigned to the port (such as telnet or pop3), port is the port number used, and protocol is usually tcp or udp. Edit /etc/services and change the port assigned to the service to be what you want, and then send inetd a HUP signal.
When you have the human name for the service, look at /etc/inetd.conf and search for that name. Let’s look at an example using telnet. The /etc/services file will contain a line similar to “telnet 23/tcp,” which tells us that port 23, using the TCP/IP protocol, is assigned to the service “telnet.” If we look in /etc/inetd.conf for the telnet service, we will see a line similar to “telnet stream tcp nowait root /usr/sbin/in.telnetd in.telnetd.”
Because we found the service in /etc/inetd.conf, we can be reasonably sure that this service is invoked by inetd and is not run as a stand-alone service. If the service is not run by inetd, then it will not be listed in /etc/inetd.conf. For example, when we started SSH in the previous section, we started it on the command line by calling /usr/local/sbin/sshd, and we did not add it to /etc/inetd.conf.
If the service is not called by inetd, then you must change the port when you start the daemon. As this procedure varies from service to service, the best place to start would be to read the man page for the daemon.
NOTE You must use the daemon name to read the man page, and not the service name. For example, telnet has a service name of “telnet,” but a daemon name of “in.telnetd.” If you are unsure what the daemon name is, use the command man -k service where service is what you are looking for. This command will return a list of man pages that relate to what you’re researching.
Most daemons will accept a command-line argument to change the port they listen to. To change the port for sshd, you would use the option -p port. For example, to start sshd listening to port 2200, you would use this command:
Other daemons will read their port assignment from /etc/services. Always change that file when you make port assignment changes. Additionally, netstat and lsof will use the information in that file when listing a report.
If you have complete control over your environment and clients, you can stop at this point and simply configure your clients to use the new port instead of the old. Most e-mail programs, for example, will allow you to specify an SMTP port other than 25 and a POP3 port other than 110. Make the changes to your client machines, and everything should work well again.
Most of the time, though, you are not afforded the luxury of forcing a port change on your clients. In that case, you must use your firewall to make port assignments. You can tell your firewall that any request coming in for machine A at port 110 should be redirected to port 1100. This will make your port-switching transparent to your end users, while still giving you the added security of not running your service as root.
Now if your service gets exploited or your daemon is compromised, the attacker will gain access to an unprivileged account and must do further work to gain root access—giving you more time to track and block them before much damage occurs.
Some daemons were written with this security hole in mind and allow you to specify the user ID and sometimes also the group ID that the daemon should assume after starting. This will allow you to create an unprivileged account to run the daemon, but also will allow the daemon to use the standard port.
The way that this generally works is that the daemon will be started by root. Since it is running as root, it can open a port below 1024. Once the port is opened, the daemon spawns off one or more children, attaches the children to the opened port, and then changes the effective user ID or group ID of the children. This allows you to tighten down the effects of security breaches of the affected daemons. One of the better known daemons that employs this technique is Apache (www.apache.org). Check the configuration guide for your daemons to see if they offer this capability, and use it if available.
Many services, because of practical necessities, cannot be locked down as much as you would like. Maybe they must be run as root, or you cannot change their port assignment, or perhaps there is a completely different reason. In that case, all is not lost—you can still isolate the service to a degree using the chroot command. Please note that you can (and should if at all possible) combine chroot with other forms of security, such as changing user IDs, swapping ports, and using firewalls.
Using chroot causes the command or daemon that you execute to behave as if the directory you specified was the root (/) directory. In practical terms, that means that the daemon, even if completely cracked and exploited, cannot get out of the virtual jail you have assigned it to.
To take a practical example, let’s use chroot to isolate in.ftpd so that you can run an anonymous FTP service without exposing your entire machine. First, we must create a file system to hold our pseudo root; let’s call it /usr/local/ftpd. Once we create the file system, we need to create any directories underneath it that in.ftpd expects. One common directory will be etc, and we will need an etc/passwd and (if your system uses it) an etc/shadow file, as well as etc/group, and so on. We will also need a bin directory containing commands like ls so people can get file listings of the directories, and a dev directory containing the devices that FTP needs to use to read from and write to the network, disks, and the like. Also make sure to set up a directory for the daemon to write logs into.
NOTE Because the process of chroot varies greatly from system to system and daemon to daemon (different systems need different directories, files, permissions, and other things) we will not go into detail here. Please search the Internet for specific pages dedicated to using chroot on your operating system for the daemon you are trying to isolate, and use the instructions presented here as merely a guideline of what needs to be done.
Once we have built up the pseudo root file system so that it contains everything we need to run the daemon, look through it carefully and verify that everything inside has the minimal permissions it needs to function. Every directory and file that should not be changed should have the write bit disabled; everything that does not need to be owned by the daemon should not be, and so on.
The syntax of chroot is chroot newroot command. Any arguments passed to command that start with / will be read from the newroot directory. In our example, we will start up in.ftpd with the root directory /usr/local/ftpd.
Notice that the command path is still relative to the actual root of the system, and not to the newroot path. There is no reason to create a /usr/sbin/in.ftpd file in the pseudo root file system.
Now your FTP service is isolated to one directory on your server. Even if the service is completely cracked, the attacker will only gain control of the chroot file system in.ftpd is running in, and not your entire machine. If you do not have exploitable applications in your pseudo root, then it will be almost impossible for crackers to elevate their permissions. Even if they do, they will still be unable to escape the jail.
TCP Wrappers is a utility that “wraps” TCP/IP connections and allows you to specify who is allowed to connect and who is not. As an example, one site where I worked had an application that required rlogin to be enabled between different machines for communication. Since rlogin is not a service you want visible to most people or systems, we used TCP Wrappers to isolate the service to only a few specific computers, and blocked access to everyone else.
TCP Wrappers is only useful for daemons that are invoked by inetd, unless the application or daemon was compiled with libwrap support. To check whether the daemon was compiled with libwrap support, use the following command:
Replace the word daemon with the path and name of the daemon you are checking. If grep finds a match, the service should support TCP Wrappers via /etc/hosts.allow and /etc/ hosts.deny.
To use TCP Wrappers, download the source from ftp://ftp.porcupine.org/pub/security/index.html. The most recent version as of this writing is 7.6.
NOTE For Solaris versions 8 and higher, you will need to get the version TCP Wrappers IPV6 by Casper Dik. You will also need to uncomment the line in the Makefile that reads IPV6 = -DHAVE_IPV6, along with any other Makefile changes appropriate to your environment.
Unpack the source code and edit the Makefile to make it compatible with your system. At the very least, you need to edit the REAL_DAEMON_DIR variable. (This example will use the “easy” installation, where the REAL_DAEMON_DIR variable is set to /usr/sbin/in.orig, and the original daemons are in /usr/sbin.) Run make to get a list of systems that TCP Wrappers has been written for, and pick your system out of the list. Then run make system where system is your system type; for example, make aix. When finished, you will have five executable programs. The main one is tcpd, but the others are useful as well.
Move tcpd into /usr/sbin, and create a directory /usr/sbin/in.orig. Create a file /etc/ hosts.deny and put this single line in it:
Then create /etc/hosts.allow and put this single line in it:
Move any daemons you want to be under the control of TCP Wrappers to in.orig, and create symbolic links to tcpd with the daemon name, as shown in the following example to migrate in.telnetd.
That’s it. All requests for telnet (or any other services you move) to your server are now filtered by TCP Wrappers. Read the documentation that comes with TCP Wrappers to learn more about setting up the /etc/hosts.allow and /etc/hosts.deny files, along with other useful information.
Modern operating systems come with a myriad of applications and utilities you can install onto your system, in addition to the core operating system itself. When getting started, it is tempting to install most—if not all—of these applications. After all, they might be useful, and they’re included with the system, so it’s probably a good idea to have them, right?
Not necessarily. Most of the applications are harmless and potentially useful, but if you do not need them you are better off not installing them. Keep in mind that every application on your system is potentially another hole that can be exploited by a malicious person. The more applications you have installed, the more vulnerabilities a malicious attacker has the choice of attempting to use. It also means more things you need to track for patching and intrusion detection.
TIP In security, the fewer things you have installed on a system, the easier it will generally be to monitor that system and keep it clean. Always ask yourself when setting up a system, will I need this application to run my server effectively? If the answer is no or probably not, then do not install it.
If you take possession of a server that was created by someone else, do a careful audit of that server and note everything that is installed. If you do not know what something is, research it to determine what it is, what is does, and whether you need it. Keep a list of all of your servers, what applications are installed on each, and which version of each application is installed. When it comes time to do security audits and system patching, this list will save you a lot of time and preparation work. Any time you add, remove, or patch an application, update your list.
Perform a full backup of your server, and then go through and disable or remove all applications that are not necessary for that server. Remove them one at a time and test your server after each change to verify that it still functions correctly.
Do you know what jobs your system is running unattended? Many operating systems come with a variety of automated tasks that are installed and configured for you automatically when the system is installed. Other jobs get added over time by applications that need things run periodically.
In order to stay on top of your system, you need to have a clear idea of what it is running. Periodically audit your crontab files and review what is being run. Many systems store their cron files in /var/spool/cron. Some cron daemons additionally support cron.hourly, cron.weekly, cron.monthly, and cron.yearly files, as well as a cron.d directory. Use the man cron command to determine your cron daemon’s exact capabilities.
Examine all of the files in each directory. Pay attention to who owns each job, and lock down cron to only user IDs that need its capability if your cron daemon (crond) supports that. Make note of each file that is running and the times they run. If something is scheduled that you do not know about, research it to determine exactly what it does and whether you need it. If someone is running something you do not feel they need, contact them and ask for their reasons, and then proceed accordingly.
Keep track of your cron jobs and periodically examine them to see if any changes have been made. If you notice something has changed, investigate it and determine why. Keeping track of what your system is doing is a key step in keeping your system secured.
All systems have SUID (set user ID) and SGID (set group ID) files. These are applications, scripts, and daemons that wish to run as a specific user or group instead of as the user ID or group ID of the person running them. One example is the top command, which runs with elevated permissions so that it can scan kernel space for process information. Since most users cannot read this information with their default permissions, top needs to be run with higher permissions in order to be useful.
Many operating systems allow you to specify that certain disks should not support SUID and SGID, usually by setting an option in your systems mount file. In Solaris, you would specify this with the nosuid option on /etc/vfstab. For example, to mount /users with nosuid on disk c2t0d0s3, the line would look like this:
This would mount /users at boot and disable SUID and SGID applications. The applications would still be permitted to run, but the SUID and SGID bits would be ignored. Disable SUID and SGID on all file systems that you can as a good security practice.
Still, you will need to periodically scan your system and get a list of all SUID and SGID processes that exist. The switch to look for to find SUID is -perms +4000, and for SGID it is -perms +2000. To scan for all SUID files on your entire server, run this command:
The -type f option only looks at “regular” files, not directories or other special files such as named pipes. This will list every file with the SUID bit set on. Carefully review all of the output and verify that everything with SUID or SGID really needs it. Often you will find a surprise that needs further investigation.
As root, you must be positive that the command you think you are running is what you are really running. Consider the following scenario, where you are logged in as root, and your PATH variable is .:/usr/bin:/usr/sbin:/bin:/sbin.
User A creates a script in his directory named ls that contains these commands:
Now user A calls you and informs you that he is having a problem with something in his home directory. You, as root, cd to his directory and run ls -l to take a look around. Suddenly, unbeknownst to you, user A now has a shell he can run to gain root permissions!
Situations like these happen frequently but are easy to avoid. If “.” was not in your path, you would see a script named ls in his directory, instead of executing it.
When you are writing a script, always specify the full path to the application you are using. Consider the following script:
It is only three lines long, and only contains two lines that do anything, yet there are many security holes:
• I did not specify a path
• I did not give the full path to date
• I did not give the full path to find
• I did not give the full path to rm
• I did not do any error checking
• I did not verify that I was in the correct directory
Let’s take another look at the script to see how we can fix some of these problems.
The second line, cd /directory || exit -1, tells ksh to attempt to cd to /directory. If the command fails, it should exit the script with a -1 return code. The ksh command || means “if the previous command fails,” and && is the ksh command that means “if the previous command succeeds.” As an additional example, the command touch /testfile || echo Could not touch will create a file named /testfile, or if the file could not be created (perhaps you do not have enough permissions to create it), then the words “Could not touch” will be displayed on the screen. The command touch/testfile && echo Created file will create the file /testfile and will only display “Created file” if the touch command succeeded. Depending on what type of condition you’re checking for will determine your usage of either || or &&.
If the script proceeds past that line, we’re guaranteed to be in /directory. Now we can explicitly specify our path. This is to lock down where the system searches for commands if we forget to give the full path. It does nothing in this small script, but it is an excellent habit to get into, especially when you are writing long and more complicated scripts; if you forget to specify the full path, there is a smaller chance that the script will invoke a Trojan horse.
Next we call date by its full name, /usr/bin/date. We also fully specify /usr/bin/find and /usr/bin/rm. By doing this, it makes it much harder for a malicious person to insert a Trojan into the system and have us run it unwittingly. After all, if they have high enough permissions to change files in /usr/bin, they probably have enough permissions to do anything else they want.
When writing a script, always follow these simple rules:
• Always specify a path.
• Always use the full path to each application called.
• Always run error-checking, especially before running a potentially destructive command such as rm.
Before you expose a system to the world, you need to know what ports are open and accepting connections. Often you will have something open that you were not aware of, and you should shut it down before letting people access your server. There are several tools that will let you know what your system is exposing.
One tool that is bundled with almost every operating system is netstat. Netstat is a simple tool that shows you network information such as routes, ports, and connections. Netstat will display all ports with their human-equivalent name from /etc/services if the port is defined, making it easier to parse the output. This is a good reason to make sure that /etc/ services is kept up to date on your system. Use the man command to find out all the capabilities of netstat on your system.
We will go through a simple example here.
In this example, someone from the IP address 192.168.3.4 is connected to your server’s SMTP service. Should you be running SMTP? Should this person be allowed to connect to it?
In the preceding output, your system is advertising the NTP service. Is it an NTP server? Should others be allowed to connect to that service? Uh oh, big hole. You have telnet wide open. I hope that you at least have TCP Wrappers protecting it. Should it be deactivated?
Take the time to learn netstat. It will provide you with a wealth of network information if you learn how to ask, and it will let you see exactly who is connected to your system at any given time.
Another very useful command is lsof (list open files). It started out as a simple utility to display what processes have files open, but it has evolved to display ports, pipes, and other communications. It is not included with many operating systems, but it can be found at http://freshmeat.net/projects/lsof.
Download the source and unpack it. The latest version as of this writing is 4.68. Change to the lsof_4.68 directory, verify the authenticity of the included archive, and unpack the lsof_4.68_src.tar file. Change to the lsof_4.68_src directory and then run Configure with your system type.
This will configure lsof for an HP-UX system. Run ./Configure -d for a list of systems that lsof supports. After lsof is built, run the command make install to see how to install lsof. Note that lsof is requesting to be installed with SGID permissions. Spend the time to read the README files that come with lsof. There is a lot of information in them.
Once you have lsof installed, try it out. Just running lsof by itself will show every open file and port on the system. It’s a good way to get a feel for what lsof can do, and also a great way to quickly audit a system. The command lsof | grep TCP will show every open TCP connection on your system. This is a very powerful tool, and also a great aid when you’re trying to unmount a file system and are repeatedly told that it is busy; lsof will quickly show you what processes are using that file system.
The Center for Internet Security (CIS) has created a system security benchmarking tool. This tool, which can be obtained from www.cisecurity.org, will do an audit of your local system and report on its findings. It will look for both good and bad things and give you an overall rank at the end of the test. There are scanning tools for Solaris, HP-UX, Linux, and Windows 2000, as well as Cisco IOS routers.
The nice part about the CIS benchmarks is their explanations. The report will not simply state “You have X, which is bad” but will in fact give you their reasoning behind why they say it is bad, and it will let you make the decision for yourself whether to disable it or keep it. The benchmark tool will check a great many things that you might not have thought of and will give you a quick detailed report of your system.
Download the CIS archive and unpack it. Read the README file and the PDF file. (The PDF file offers great reference material on system security.) Install the package by following the instructions in the README file, and when finished, you should have an /opt/CIS directory with the tool installed.
To get a snapshot of your system, run the command cis-scan. Depending on the speed of your server and the number of drives attached to it, the scan can take a long time to complete. When finished, you will have a file named cis-ruler-log.YYYYMMDD-HH:MM:SS.PID. That file contains the summary of your system, and it shows you the results of all of the tests. There is not a lot of information contained in this file—it is meant to be used as an index to the PDF file that comes with the scanning tool.
Go through the ruler-log file line by line, and if there is a negative result, determine whether you can implement the change suggested in the PDF file. Most of the changes can be implemented without affecting the operation of the server, but not all of them. Beware of false negatives as well; you might have PortSentry watching port 515 for lp exploits, which will cause the CIS tool to erroneously report that you have lp running. The higher the number at the end of the report, the more “hardened” your system is.
This is a great tool to have in your security arsenal and to run periodically on your servers to keep them healthy. Check back on the CIS web site from time to time, as the tools are constantly evolving and changing.
Every operating system has vulnerabilities. Most vendors run audits against the code and remove any that are found, but some are inevitably released into the world. Certain people spend a great deal of time trying to discover the ones that remain; some do it to report them to the vendor, but others do it for their own personal use.
In any event, occasionally exploits are found and patches are released to fix them. Unless the vulnerability is severe, or a known exploit exists in the wild, there is usually not much fanfare announcing the release of these patches. It is up to you to occasionally look and see what patches are available from your vendor, and if any of them apply to you.
Many vendors will supply a tool to help you keep on top of your system patches. HP-UX has the Software Update Manager, Solaris has patchdiag and patchpro, AIX uses smit, and so on. Run your diagnostic tool at least once a month to see what new patches are available for your system, and determine if you need to install them. Set aside at least one hour each Sunday afternoon (more if you are allowed) as dedicated system downtime, and use that time for installing patches and performing other needed maintenance.
You should also make it a habit to go to the web site for each application you have installed to see if there are any bug fixes or security patches released for those applications. Use the list of applications you created earlier to determine whether any of the patches apply to you. Remember to update your list if you apply any patches.
If you are responsible for maintaining multiple servers, then checking the logs on each of them can become unwieldy. To this end, set up a dedicated server to log messages from all of your other servers. By consolidating your logs, you only have to scan one server, saving you time. It also makes a good archive in case a server is compromised; you still have untouched log files elsewhere to read.
To create a central log server, take a machine with a fast CPU and a lot of fast hard drive space available. Shut down all other ports and services except syslogd to minimize the chance of this system being compromised, with the possible exception of a TCP Wrapped SSH daemon restricted to your workstation for remote access. Then verify that syslogd will accept messages from remote systems. This varies from vendor to vendor. Some vendors have the default behavior to accept messages, and you must turn it off if desired; others have the default to not accept, and you must turn it on.
Create a system for archiving older logs and document it. If your logs are ever subpoenaed for evidence, you need to be able to prove that they have not been altered, and you will need to show how they were created. It is suggested that you compress all logs older than one week and replicate them to time-stamped, read-only media, such as a CD.
Once you have a server in place to accept your logs, you need to start pointing your servers to it. Edit /etc/syslog.conf and determine which messages you wish replicated. At the very least, you should replicate the emerg, alert, crit, err, and warning messages, and more if you think it will be beneficial. When you know what you want replicated, add one or more lines like the following to /etc/syslog.conf:
In this example, we are replicating all emerg, alert, crit, err, warning, and notice messages to the remote server.
NOTE You can archive logs onto a remote server and keep them locally at the same time. You can also replicate to more than one log server. Syslog.conf is scanned for all matching entries—the syslogd daemon does not stop after finding the first one.
Although it seems obvious, keeping a log and replicating it is not useful if your daemons are not logging any information in the first place. Some daemons by default create log entries, and others do not. When you audit your system, verify that your daemons are set to log information. This is one of the things that the CIS benchmarking tool will look for and remind you about.
Any daemon publicly available needs to be configured to log, and the log needs to be replicated. Try accessing some of your services and see if logs were collected on your log server. If they were not, read the man page for that service and look for the option to activate logging. Activate it and try using the service again. Keep checking all of your services until you know that everything is logging and replicating.
E-mail is one of the primary reasons many people access the Internet. It has grown from its humble beginnings—initially it was not even its own protocol, but was part of FTP. As more people started to use e-mail, it grew into its own protocol, named SMTP, and sendmail arose to handle this new protocol.
Over time, people started demanding more options and control over their e-mail systems. As a result, sendmail grew in complexity. This cycle continued for many years and resulted in a system that is so complicated that simple changes are beyond the ability of most people who have not devoted a large amount of time to learning sendmail.
This situation has also resulted in a very complicated code base to support the myriad options that sendmail offers. While having these options can be beneficial, one of the results is that there is potential for many different types of exploits against this server. If you look through the past vulnerability alerts from places like CERT, you will see that a high percentage of them deal with sendmail in some manner. Sendmail can be relatively secure if you keep on top of bug-fixes and security releases and know enough about sendmail rules and configurations to lock it down. Because of the complicated process of configuration, though, many people never learn enough to do this effectively.
Another major concern with sendmail is that it must be run as root to work. This restriction, coupled with the large complexity of the code base, makes it a popular target for attackers. If they succeed in exploiting sendmail, they have gained root access to your machine.
Also, ask yourself if your server needs to be running sendmail (or a replacement) at all. Most systems do not—the only reason your system needs this service is if it will accept inbound e-mail for local users from the network. The majority of systems running an SMTP server do not need to, and are doing so simply because that is the way the OS set the system up. If you do not need to accept inbound e-mail, do not run an SMTP service on your machine.
This situation has not gone unnoted in the software community, and several people have written replacements for sendmail that try to address its shortcomings. Some of these replacement programs work quite well, and as such have collected a devoted following of users. We will discuss the two most popular in the following sections: postfix and qmail.
Postfix is a sendmail replacement written by Wietse Venema, the same person who wrote TCP Wrappers and other applications, and it can be found at www.postfix.org. The two overriding goals when writing postfix were ease of use and security, and both have largely been achieved. Getting basic e-mail services running under postfix is easily done by almost every administrator, even if they have not extensively worked with SMTP previously, and there are very few known security exploits.
To use postfix, download the latest version of the source, which is 2.0.16 as of this writing. If your system is one of the supported systems, compiling postfix can be as easy as changing to the directory and running make.
Once postfix is compiled, you have to follow a few simple steps before you can use it. Since your operating system probably came with a version of sendmail, we will go through the steps needed to replace sendmail with postfix. As always, read the README files and the INSTALL file for a lot more information about postfix.
Locate your current sendmail application. If you’re not sure where it is, look in directories such as /usr/sbin, /usr/libexec, and others. You can always run the command find / -type f -name sendmail -print to locate it for you if you cannot find it. Once you have located sendmail, rename it to something else. Wietse’s suggestion is to rename it to “sendmail.OFF.” Locate the applications newaliases and mailq, and rename them in a similar fashion.
Now you will need to create a “postfix” user ID and “postfix” group ID for postfix to use when running. At the same time, create an additional “postdrop” group that is not assigned to any ID, not even the postfix ID. The reason for this was mentioned earlier: if someone manages to exploit postfix they will not gain any elevated privileges. Make sure to use IDs that are not being used by any other process or person. The account does not need a valid shell nor a valid password or home directory.
NOTE An invalid password refers to a password that cannot be used, such as one created by entering “*NP*” in the password field of /etc/passwd or /etc/shadow if your system uses shadow passwords. It doesn’t refer to an account without a password at all.
Edit /etc/aliases to add the alias “postfix: root.” Now you can install postfix by running the command make install in the postfix source directory. When the install is finished, edit the /etc/postfix/main.cf file and make sure you change at least the variables myorigin, mydestination, and mynetworks, and any others you feel are appropriate for your environment. When you are finished, save the file and kill off the current running sendmail daemon, as shown here.
Now restart the old sendmail program in queue mode only to flush out any old e-mail that might be queued for sending, and then start postfix.
Postfix is now installed and running on your system. Any time you make changes to the configuration files, run /usr/sbin/postfix reload to cause postfix to rescan the configuration and incorporate the new changes without affecting mail delivery.
Qmail is a sendmail replacement written by Dan Bernstein. The two overriding goals for qmail were security and speed. Qmail also incorporates a simple mailing list framework to make running your own mailing list a fairly simple process. Dan suggests using djbdns as a DNS replacement, as well. All qmail and related source code can be downloaded from http://cr.yp.to/qmail.html, and the current version of qmail as of this writing is 1.03. Download the archive and unpack it.
Configuring qmail is more complicated than postfix. First you must make the qmail home directory, which by default is /var/qmail, and then create qmail’s group and user IDs. Read INSTALL.ids for more information on creating the necessary IDs. Then edit conf-qmail in the source directory to make any changes needed for your environment.
At this point, run make setup check to compile qmail and create the needed directory structures. Then run config to do some basic qmail configuration. The steps listed in the following code are for a Linux installation, and some of the commands may not be appropriate for your environment.
Because of the many complications involved in setting up qmail, you must read the README and INSTALL* files to make decisions about your environment and perform the necessary steps. Failure to do this will result in an implementation of qmail that does not work. For further information on using qmail, go to www.lifewithqmail.org and www.qmail.org.
To help you stay on top of the latest news in the world of Internet security, there are several web sites and mailing lists that you can subscribe to. Most of them will send timely alerts when an exploit is known to exist, along with steps you can take to temporarily block the exploit, and the locations of patch files (if they exist) to correct the problem.
Some of the more respected groups are the SecurityFocus team at www.securityfocus.com, and the Full Disclosure mailing list available at http://lists.netsys.com/full-disclosure-charter.html. Subscribe to these two lists at a minimum, and pay attention to what they disseminate. Remember that while getting hacked is not a good thing, getting hacked by a known exploit for which there was a patch available can be not only embarrassing, it can get you fired.
There are many aspects to keeping a Unix system secure. In addition to what we discussed here, there are many more options available to you to help lock down your system, such as replacing your FTP server, locking down daemons and services to only a selected group of people, file-scanning your server to detect changes, and so on.
The information in this chapter is intended to get you started on your way to securing your servers. It is not the final word on the matter, and neither is any other book. Security is an ever-changing and evolving field, and it takes time and dedication to stay on top of the game. Use the advice presented here to get started into the world of security, but don’t stop there. Use it, but do not exclusively rely on it.