Chapter 19. Network Monitoring

Network Intrusion Detection Sensors (NIDS) try to detect network intrusions by monitoring the network. NIDS detect attacks only; they do not respond to them. Devices that respond to attacks are known as Network Intrusion Prevention Systems, or NIPS. This chapter will only cover detecting the attacks with NIDS devices that are free and open source.

NIDS are not plug-and-forget devices. They generate logs that need to be analyzed by humans, and then decisions based upon those logs need to be made. Using NIDS and not analyzing the logs is not very useful. So while NIDS solutions for network monitoring have proven themselves to be very effective, they are not a silver bullet solution that you can just plug in and forget.

Snort

Snort is probably to best known network intrusion detection software. Its deployment ranges from home user to financial institutions. Many different books have already been written on Snort. This section offers a quick start and ideas on some usual and some unusual implementation of Snort. Although Snort is specialized, it ends up being useful in a multitude of situations.

Snort can be used in basically three different modes:

When used in the NIDS mode, Snort has several submodes depending on the level of reporting, logging, and stealth required. When used as a NIDS, Snort acquires only information if any of the rules are tripped, and ignores all other network communication without generating packet captures or logs.

When working in NIPS mode, Snort is placed inline with the communication. This configuration requires that Snort be fully configured for your environment since it can not only log attacks, but also block them. In this scenario, a single bad rule that is set to block communications could bring your whole network down, so before thinking of placing Snort in NIPS mode, be sure to control the number of false positives that your sensor sees. Skipping this step might create a denial of service of your own making on your own network.

Many existing tools already provide and exceed Snort's basic packet sniffer functions, so this section will not cover Snort as a normal packet sniffer.

Snort is a signature-based NIDS, so its ability to detect attacks depends on the quality of its signature base. While many signatures are already provided with Snort, you should learn to write your own signatures to fill your own needs. Signature writing requires a decent knowledge of network protocols and regular expressions. Make sure you read and understand all of W. Richard Steven's TCP/IP Illustrated books (Volumes 1 to 3; Addison-Wesley) or other comprehensive TCP/IP books, and perhaps a good regular expression book before you starting writing signatures.

Example 19-1 shows the format of a basic Snort signature; for more advanced features, refer to the Snort manual or web site (http://www.snort.org/docs/snort_manual/node16.html).

Example 19-1. The format of a basic Snort signature

                [Pass, Log, Alert]
                [ip,icmp,tcp,upd]
                [any,'dstIP/netmask']
                [any, 'port']
                [any,'srcIP/netmask']
                [any, 'port']
([content:, msg:, offset:, depth:, content-list:, dsize:,
flags:,nocase,reference:,pcre])

An simple and very bad signature could look like this:

        alert ip any any -> any any (msg: "There is an IP packet... ")

This signature would simply send an alert for every IP packet that the IDS sees.

A more useful signature is:

alert tcp any 80 -> 192.168.1.0/24 any (content-list: "porn-word";
msg: "Porn word matched"; offset: 40; depth: 1000; nocase)

For any incoming packet from port 80 (HTTP), this signature looks inside the first 40 to 1,000 bytes of each packet for strings containing a word from the file porn-word. This could be an easy way to detect whether employees are currently browsing web sites containing words suggesting pornographic content.

Here is a signature using more advanced features, such as Perl regular expressions:

    alert ip any any -> any any
(pcre:"/%[0-9.]*$?(i|n|x|s|h|e|g|a){3}/"; msg: "Format string";)

This simple signature could detect the presence of a format string inside an IP packet. It should detect attacks such as %n%n%n and %213.2n%x%344.123$h.

Regular expressions can provide very efficient ways of detecting attack patterns; however, writing efficient regular expressions can be very difficult because the performance impact of such signatures is usually very high. So using a regular expression match makes sense when one regular expression could replace multiple simple content match rules. There is no easy equation for this ratio because the processing time of a regular expression is not linear.

Tip

Try to avoid the usage of back references in regular expressions because they require over-processing of the data and penalize the performance of the sensor.

The first thing that amazed me when I started consulting in computer security is the lack of knowledge most network administrators have about their network. I'm not talking about knowing the configuration quirks, or version or operating system of a server, but the plain existence of servers in the network. In security, the first thing you have to know is what there is to defend. While a full inventory is ideal, sometimes it is just not possible due to resources, topology, time, or other restrictions. This is where Snort can be very useful.

Snort can be used not only as an intrusion detection device, but also as a passive network discovery tool. Connecting your Snort detector on the span port of a switch and using appropriate signatures can give you a good look at your network in a matter of minutes. How many Windows workstations, printers, web servers, DNS, or even rogue devices are present on the network? The precision of the network discovery you can achieve is pretty much up to you—depending on the precision of the signature, that is.

For example, a rule to discover web servers might look like:

alert tcp any any -> any 80 (content:"(GET|HEAD|POST)"; nocase; msg:
"Web server on port 80";)
alert tcp any any -> any !80 (content:"(GET|HEAD|POST)"; nocase; msg:
"Web server on port other than 80";)

A rule to discover an IIS server would look like:

alert ip any 80 -> any any (content:"Server: Microsoft-IIS"; nocase;
depth:200; msg: "Source is an IIS server";)

A rule to discover an IIS 5.0 server would look like:

alert ip any 80 -> any any (content:"Server: Microsoft-IIS/5.0"; nocase;
depth:200; msg: "Source is an IIS/5.0 server";)

So the level of precision is adjusted by using different types of rules.

Tip

This method of network discovery is limited by what you see is what you get— if a device is down, not using a protocol, or somewhere else so you don't see the communication, then you wont be able to discover it. Only devices having active communications over that channel (switch, router, hub) will be discovered. So, while Snort can help you discover your own network, it is not magically omniscient.

After installing a working version of Snort, you might like it to be stealth or not have an IP address, but still be able to gather the information from the wire.

In most Unix variants (BSD and Linux), simply entering the following command puts the interface up without subsequent configuration such as an IP address or subnet. In this way, it is possible to sniff from that interface without being able to reach it from the IP layer:

ifconfig eth1 up

If you do not receive an IP address listing from the interface you modified, you are good to go.

To run Snort with the specified interface, use the -i flag; for example:

Snort -i eth1

Snort has rules to detect anomalous packets on your network. Maintaining these rule sets can dramatically improve the system's effectiveness, so a few rule-management techniques can help you maintain your Snort IDS:

The Snort rules are split in multiple categories; e.g., www, virus, pop3, oracle, and chat. While doing your configuration, you could disable all the rule categories that do not apply to your environment. That way the level of noise generated by your NIDS will drop significantly. The same idea can be applied inside the categories; for example, if a signature catches a IIS bug but all your servers run Apache, then there is no need to block those attacks or log them, since they have little chance of affecting your server.

The easiest way to disable rules is simply to just place a # in front of the rules or categories that you want to remove, as in to following example:

The example goes from:

alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"SMTP RCPT TO overflow";
pcre:"/^RCPT TO\x3a\s[^\n]{300}/ism"; sid:654; rev:14;)

to:

#alert tcp $EXTERNAL_NET any -> $SMTP_SERVERS 25 (msg:"SMTP RCPT TO overflow";
pcre:"/^RCPT TO\x3a\s[^\n]{300}/ism"; sid:654; rev:14;)

It is possible to change the default port of a service. Since newer rules use the $HTTP_PORTS variable, you simply reset it and rerun the rules for the other ports. All those change occur in the configuration or rules files.

For example:

var HTTP_PORTS 80
include web.rules
var HTTP_PORTS 8080
include web.rules

If you have one network card and two aliases, how can you force Snort to listen on both addresses? Depending on your version, either you can specify an IP list like this:

var HOME_NET [ 192.168.10.0/24, 10.1.1.1/16 ]

Or you can respecify the HOME_NET variable multiple times like this:

var HOME_NET 10.1.1.0/24
include misc.rules
var HOME_NET 192.168.1.0/24
include misc.rules

Preprocessors are a very important part of Snort architecture, as packet fragmentation, URL encoding, and tasteful inspection are all performed through the use of preprocessors. This is why it is very important to learn to use and configure them. Otherwise, a simple fragroute on the side of the attacker could render your entire signature set totally useless.

Preprocessors can also be responsible for parsing more specific protocols such as Telnet, SMTP, and FTP. This preprocessing enables less data to be examined, offers better performance, and protects against evasion techniques. Preprocessors need to be configured before use, so it is important to take a look at the provided configuration since each preprocessor may need its own configuration. Please refer to the Snort manual for more information on the possible configuration of each preprocessor.

It is always a good idea to subscribe to an Internet scanning service to get a second opinion about your network. However, it's not that easy to get a good scan.

With your defenses up, you are essentially leasing a service that has no chance at all of penetrating your network, just overloading your NIDS with logs from their scan. So at the end of the day, all you have is 25,000 logs to look at—you paid these guys to tell you almost nothing, since you blocked all its reconnaissance packets, if not banned its IP address.

So, you might decide to drop your first line of defense for these scanners, to give them (and you) a much better view of the real inside network. This is a better scan of the current vulnerability level because it will find which servers are vulnerable and whether the firewall and IDS fall down.

If you ever want to drop the Snort rules in such a case, just add a pass rule that disables processing packets incoming from the scanner. Here is an example of a pass rule to allow the source IP address 192.168.2.3 to connect to ports in the network without generating logs:

pass ip 192.168.2.3 any -> $HOME_NET any

Another example could be used for a web scanner to access only the web server:

pass tcp 192.168.2.4 any -> $WEB_SERVER 80

Other variations on the same principle could be applied to fit your network.

Once you have a Snort setup working, it will start to generate a lot of information about what is happening on your network. This can be confusing; having too much information is like having not enough—the important information gets lost in the noise level so quickly that you cannot distinguish between what is important and what is not. To solve that issue, there are many products that help parse the information generated by Snort, such as a mySQL server and an ACID interface, or Snortalog. These tools can generate graphics about what is going on in your network. Figure 19-1 shows an example from the main ACID window.

The main ACID window

Figure 19-1. The main ACID window

The other possibilities for log analysis include sending the information to a syslog server and using a tool such as Splunks (http://www.splunks.com/) to manage all of your logs—not those just generated by Snort. Splunk, while not an open source or free tool, is a search engine for your logfiles. It acts like a syslog Google. Figure 19-2 shows the Splunk main window.

Splunk in action

Figure 19-2. Splunk in action

Some might prefer to use the Unix grep to parse logfiles. To GUI-lovers, grep might seem archaic but it remains one of the best log-processing utilities. Scripting with grep also offers you the possibility to create your own reports. Try using grep and you might find it's the right tool for you. When you learn a bit of regular expressions, you might be surprised by what this tool can do.

An NIDS based on signatures is only as good as its signature database. It is very important to keep an up-to-date signature database for Snort. Otherwise, Snort will not be able to catch the new exploits or bugs and will only see valid traffic. However, updating it frequently may also cause a problem if you have customized signatures everywhere. I highly recommend putting your custom signatures in another file so they do not get erased during your next update. The other way of doing it is to keep an original version of the signature database before you modify it, doing a diff of the new database with the original one, and then applying this patch to your personal customized database. This way only new signatures, deleted ones, or modified ones are changed, and your tweaked signature should stay the same.

Example 19-2 lists a simple script to replace the old database with the new one. Note that all changes manually made are lost during this process.

Example 19-2. A script to replace an old database with a new one

#!/bin/sh
# Place of storing your snort rules. Change these variables
# according to your installation.
RULESDIR=/etc/snort
RULESDIRBAK=/etc/snort/bak
# Path to wget program. Modify for your system if needed.
WGETPATH=/usr/bin
# URI for snort rules
RULESURI=http://www.snort.org/downloads/Snortrules.tar.gz
# Get and untar rules.
cd /tmp
rm -rf rules
$WGETPATH/wget $RULESURI
tar -zxf snortrules.tar.gz
rm -f snortrules.tar.gz
# Make a backup copy of existing rules
mv $RULESDIR/*.rules $RULESDIRBAK
# Copy new rules to the location
mv /tmp/rules/*.rules $RULESDIR

Since signature-based detection is not very appropriate for detecting port scans, Snort offers a preprocessor module that does just that. If you want to detect incoming port scans, be sure to add this configuration to your server. The format for this preprocessor is:

preprocessor sfportscan: proto <protocols> \
    scan_type <portscan|portsweep|decoy_portscan|distributed_portscan|all>\
    sense_level <low|medium|high> \
    watch_ip <IP or IP/CIDR> \
    ignore_scanners <IP list>\
    ignore_scanned <IP list> \
    logfile <path and filename>

A normal rule would be:

preprocessor flow: stats_interval 0 hash 2
preprocessor sfportscan: proto {all} scan_type {all} sense_level {low}
ignore_scanners {$Qualys_box, $nessus_box}

The ignore_scanners and ignore_scanned options come into play in weeding out legitimate hosts that are very active on your network. Some of the most common examples are NAT IP addresses, DNS cache servers, syslog servers, and NFS servers. The sfPortscan preprocessor may not generate false positives for these types of hosts, but be aware when first tuning sfPortscan for these IP addresses. Depending on the type of alert that the host generates, the analyst will know which to ignore it as. If the host is generating port sweep events, then add it to the ignore_scanners option. If the host is generating port scan alerts (and is the host that is being scanned), add it to the ignore_scanned option.

ILDS are information leaks detection engines. While network attacks from the Internet are a well-known and discussed subject, information leak detection looks like it is still in its infancy. For those of you that work on large-scale networks, information leaks coming from an employee probably costs much more money to your business than external attackers using your server to propagate worms. Information leak detection tries to target the information that you do not want to get out of your business and fall into the hands of the competition. So what we want to detect is not an attack pattern but a leak pattern. What are leak patterns? How do you look for them? A simple leak pattern could look like this basic list:

Confidential
Secret
Patent
Financial reports
Internal Only
Do not disclose
...

This kind of list could be used to monitor the presence of those words in email, IM, and HTTP communication.

The next example is a rule that detects patterns similar to a social security numbers:

alert tcp any any -> any 25 (pcre:"/\d\d\d-\d\d-\d\d\d\d/"; msg: "SN# in email";)

Other rules (e.g., such as one monitoring people from your staff posting on blogs about your Enterprise) could be used to ensure that the information disclosure rules are applied.

Studies in the field of information leakages, done by companies such as Port Authority (www.portauthoritytech.com), suggest that most information leakages occur through these basic protocols:

These protocols should be first on your hit list, since they are the most used modes of communication to leak information.

Snort was not designed to be an ILDS, so the following features that would be useful in an ILDS are not present in Snort:

Due to these limitations, Snort as an ILDS should only be used to indicate which host or internal leaks should be better monitored by other solutions, such as a full network capture by tcpdump (see Chapter 18). Also, having a full communication capture can provide information about what leak was made by who to whom, under what circumstances, and other parameters.

Tip

Implementing a system that monitors information leaks requires network administrators. These administrators will see part of the leaked information, so be sure not to create another leak by trying to patch the first one.

Snort is network-monitoring software. While it can be used to find an information leak occurring inside your organization, it can also be used to monitor what employees are doing on their system. Sort of like a big brother tool.

Snort can monitor the network in two ways: against attackers trying to get in your network and against employees using the network to look out. While it might be difficult to determine what constitutes a bad web site to look at, rules could be applied to trigger on a different level and catch both types of network usage.

Example 19-3 contains examples of rules to monitor and catch misused corporate resources. Some of these examples are taken from the default Snort rules and are explained; you can also create your own and contribute to the community.

Example 19-3. Example rules for monitoring network usage

# This alert is looking for MSN messenger message
alert tcp $HOME_NET any <> $EXTERNAL_NET 1863 (msg:"CHAT MSN message";
flow:established; content:"MSG "; depth:4; content:"Content-Type|3A|"; nocase;
content:"text/plain"; distance:1; classtype:policy-violation; sid:540; rev:11;)

# This alert is looking for MSN messenger login
alert tcp $HOME_NET any -> $EXTERNAL_NET 1863 (msg:"CHAT MSN login attempt";
flow:to_server,established; content:"USR "; depth:4; nocase; content:" TWN ";
distance:1; nocase; threshold:type limit, track by_src, count 1, seconds 60;
classtype:policy-violation; sid:1991; rev:2;)

# This alert is looking for AIM message
alert tcp $HOME_NET any -> $AIM_SERVERS any (msg:"CHAT AIM send message";
flow:to_server,established; content:"*|02|"; depth:2; content:"|00 04 00 06|";
depth:4; offset:6; classtype:policy-violation; sid:1632; rev:6;)

# This alert is looking for connection to a PCanywhere software.
alert udp $EXTERNAL_NET any -> $HOME_NET 5632 (msg:"POLICY PCAnywhere server
response"; content:"ST"; depth:2; reference:arachnids,239; classtype:
misc-activity; sid:566; rev:4;)

# This alert is looking for keyword "free XXX" in a web site.
alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (msg:"PORN free XXX";
content:"FREE XXX"; nocase; flow:to_client,established; classtype:kickass-porn;
sid:1310; rev:5;)

# This alert is looking for ftp access with no password.
alert tcp $EXTERNAL_NET any -> $HOME_NET 21 (msg:"INFO FTP no password";
flow:from_client,established; content:"PASS"; nocase; pcre:"/^PASS\s*\n/smi";
reference:arachnids,322; classtype:unknown; sid:489; rev:7;)

# This alert is looking for download of video/music over HTTP
alert tcp $EXTERNAL_NET 80 -> $HOME_NET any (msg:"MULTIMEDIA Windows Media
download"; flow:from_server,established; content:"Content-Type|3A|"; nocase;
pcre:"/^Content-Type\x3a\s*(?=[av])(video\/x\-ms\-(w[vm]x|asf)|a(udio\/x\-ms\-
w(m[av]|ax)|pplication\/x\-ms\-wm[zd]))/smi"; classtype:policy-violation;
sid:1437; rev:6;)

# This alert is looking for Shoucast download over http
alert tcp $EXTERNAL_NET 80 -> $HOME_NET any (msg:"MULTIMEDIA Shoutcast playlist
redirection"; flow:from_server,established; content:"Content-type|3A|
audio/x-scpls"; nocase; content:"|0A|"; within:2; classtype:policy-violation;
sid:1439; rev:5;)

# This alert is looking for realmedia player file download over HTTP
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"MULTIMEDIA realplayer
.rmp playlist download attempt"; flow:to_server,established; uricontent:".rmp";
nocase; flowbits:set,realplayer.playlist; flowbits:noalert;
classtype:misc-activity; sid:2420; rev:3;)

# This alert is looking for torrent download over http
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Torrent file download";
flow:to_server,established; uricontent:".torrent"; nocase; flowbits:noalert;
classtype:misc-activity;)

# This alert is looking for MP3 download over http
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Mp3 file download";
flow:to_server,established; uricontent:".mp3"; nocase; flowbits:noalert;
classtype:misc-activity;)

# This alert is looking for Bittorrent transfer
alert tcp $HOME_NET any -> $EXTERNAL_NET 6881:6889 (msg:"P2P BitTorrent transfer";
flow:to_server,established; content:"|13|BitTorrent protocol"; depth:20;
classtype:policy-violation; sid:2181; rev:2;)

# This alert is looking for access to the goaste web site.
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS
(msg: "Not allowed Web access Goaste"; flow: to_server,established;
content:"goaste.cz"; nocase; classtype: policy-violation;)