IP Stack Integrity Checker (ISIC) is a suite of tools to check the network stack of a device (http://isic.sourceforge.net). It contains the following software components:
All these tools work the same way: they send pseudo-random traffic to a target. The user can choose a couple of parameters, such as what percentage of IP packets should be fragmented, what percentage should have a bad IP version, and so on. The tools are useful for detecting crashes and high CPU utilization on the target devices (e.g., due to a weakness in the network stack).
The ISIC suite runs on Linux, BSD, Mac OS X, and most Unix systems. However, the TCP/IP stacks of these systems do not have the same behavior. For example, the FreeBSD network stack will discard most of the malformed packets generated by the ISIC tools. The Linux TCP/IP stack does not discard any single packet. My advice is to run the ISIC tool on a Linux machine.
Tip
Not all Linux distributions have the ISIC package. It can be compiled from source, but this is not always easy. I would recommend a distribution from the Red Hat family (Fedora Core, Red Hat Enterprise, CentOS) or Debian Testing, which provides the latest version of ISIC (version 0.07 at the time of writing).
The device being tested should be plugged directly into the machine that runs the tests. If there is another device in between (even a simple hub or switch), this device could drop packets or crash. The server that runs ISIC requires one Ethernet card to send the traffic, or two cards if you prefer to manage it remotely.
esic, like the other ISIC tools, generates pseudorandom packets. With this tool, you can tweak layer 2. By default, esic creates packets with the source MAC address of the Ethernet card used to send the packets, with the broadcast destination MAC address and Ethernet protocol 2048 (0x800, the IP protocol), and a maximum packet length of 1500 (maximum Ethernet frame size). The only mandatory option of the tool is the interface to send the packets:
[root@replay ˜]#esic -ieth2Seeding with 8127 Maximum packet size (minus header) is 1486 bytes Ethernet protocol will be 2048. Sending to MAC ff:ff:ff:ff:ff:ff Sending from MAC 00:11:43:ec:da:71 Sending... 1000 @ 17868.0 pkts/sec and 13003.6 k/s 2000 @ 16772.6 pkts/sec and 12300.3 k/s 3000 @ 15857.7 pkts/sec and 11843.6 k/s 4000 @ 15372.8 pkts/sec and 11436.6 k/s
If you do a packet capture either on the target or on the server that runs esic, you will see a lot of fragmented and malformed packets (see Figure 17-6) such as wrong IP length, bogus IP headers, and so on.
Here's what you can tweak:
-s<MAC>Source MAC address in the format
aa:bb:cc:dd:ee:ff. You can userandto randomize the address. By default, esic uses the MAC address of the card specified with the-Ioption.-d<MAC>Destination MAC address, can be
rand. By default, esic uses the broadcast addressff:ff:ff:ff:ff:ff. Usually, the destination MAC address is set to the Mac address of the Ethernet card that receives the traffic on the target device.-p<protocol>Protocol number, can be random with the value
rand. The default protocol is 2048 (0x800, IP). You can use 2054 for ARP, 34915 for PPPoE, 36864 for loopback, etc.-r<seed>Random seed. esic generates pseudorandom packets. To be able to reproduce a test, you need to specify a random seed (600, for example) so that esic will replay the same packets in the same order.
-c<packets>Number of packets to send.
-l<length>Maximum packet length must be between 14 and 1,500.
- -m
<interval> Interval of the printouts. By default, esic prints information on the standard output every 1,000 packets.
- -k
<offset> Number of packets to skip. This option was added in version 0.07.
The options -r, -c, and -l are used mainly for automation. See the section "Automation," later in this chapter, for more details.
The other tools are similar to esic, but they work at layers 3 (isic) or 4 (icmpsic, tcpsic, udpsic, multisic for multicast UDP packets). You cannot control the interface used to send the packets, only the destination IP address. This makes the setup a little bit more complicated when the device tested does not have an IP address on the interface that receives the traffic (e.g., in the case of an IPS in transparent mode). Let's say you want to send packets with the source IP address 1.1.1.1 to the destination 1.1.1.2. On Linux, you cannot use two interfaces (interface 1 with the address 1.1.1.1, interface 2 with the address 1.1.1.2), even if you have a static group that forces all IP packets with the destination 1.1.1.2 to be sent through interface 1. Because Linux is aware that interface 2 has the IP address 1.1.1.2, it ignores the static routes and uses the loopback interface to send the packets directly to interface 2. The packets do not leave the server to go through the target system.
One solution is to use two machines, the second one with the IP address 1.1.1.2; however, it is possible to work with a single server. Simply configure the interface that sends the packet thusly:
[root@replay ˜]ifconfigeth1 1.1.1.1netmask255.255.255.0
This creates a static group automatically for interface eth1:
[root@replay ˜]# route -v
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.157.5.128 * 255.255.255.192 U 0 0 0 eth0
1.1.1.0 * 255.255.255.0 U 0 0 0 eth1 169.254.0.0 * 255.255
.0.0 U 0 0 0 eth0
default 10.157.5.129 0.0.0.0 UG 0 0 0 eth0If you run isic, icmpsic,tcpsic, or udpsic at this point with the destination address 1.1.1.2, you will see ARP packets asking only who has the IP address 1.1.1.3, because the receiving interface on the device being tested does not have any IP address. To solve this, you need to assign a fake MAC address or the MAC address of the Ethernet card on the target device to the IP address 1.1.1.2. You can do that by altering the ARP cache of the Linux machine:
[root@replay ˜]#arp -s1.1.1.2 aa:bb:cc:dd:ee:ff
Warning
Do not choose a MAC address of another card of the system running the ISIC suite. Linux would recognize that the Mac address belongs to a physical Ethernet card on the same machine and would use the loopback interface to send the packets directly to the network card.
Now that the setup is complete, you can run the tools:
[root@replay ˜]#isic -s1.1.1.1-d1.1.1.2Compiled against Libnet 1.1.2.1 Installing Signal Handlers. Seeding with 31251 No Maximum traffic limiter Bad IP Version = 10% Odd IP Header Length = 50% Frag'd Pcnt = 30% 1000 @ 15846.6 pkts/sec and 10212.8 k/s 2000 @ 15731.4 pkts/sec and 10059.9 k/s [...] [root@replay ˜]#icmpsic -s1.1.1.1-d1.1.1.2Compiled against Libnet 1.1.2.1 Installing Signal Handlers.s @ 15704.06 pkts/s Seeding with 31796sic -s 1.1.1.1 -d 1.1.1.2 No Maximum traffic limiter Bad IP Version = 10% IP Opts Pcnt = 50% Frag'd Pcnt = 30% Bad ICMP Cksm = 10% 1000 @ 15850.1 pkts/sec and 10162.8 k/s 2000 @ 15309.7 pkts/sec and 9948.5 k/s [...] [root@replay ˜]#tcpsic-s1.1.1.1-d1.1.1.2Compiled against Libnet 1.1.2.1 Installing Signal Handlers. Seeding with 31936 No Maximum traffic limiter Using random source ports. Using random destination ports. Bad IP Version = 10% IP Opts Pcnt = 50% Frag'd Pcnt = 30% Urg Pcnt = 30% Bad TCP Cksm = 10% TCP Opts Pcnt = 50% 1000 @ 14502.2 pkts/sec and 10176.3 k/s 2000 @ 15117.2 pkts/sec and 10133.3 k/s [...] [root@replay ˜]#udpsic -s1.1.1.1-d1.1.1.2Compiled against Libnet 1.1.2.1 Installing Signal Handlers. Seeding with 32085 Using random source ports. Using random destination ports. No Maximum traffic limiter Bad IP Version = 10% IP Opts Pcnt = 50% Frag'd Pcnt = 30% Bad UDP Cksm = 10% 1000 @ 15557.2 pkts/sec and 10105.3 k/s 2000 @ 15132.5 pkts/sec and 10061.4 k/s [...] [root@replay ˜]#multisic -ieth0-s1.1.1.1-d1.1.1.2Compiled against Libnet 1.1.2.1 Seeding with 24965 Using random source ports. Using random destination ports. Sending from MAC 00:0c:f1:cd:a3:a4 No Maximum traffic limiter Bad IP Version = 10% IP Opts Pcnt = 10% Frag'd Pcnt = 10% Bad UDP Cksm = 10% 1000 @ 3163.0 pkts/sec and 2188.6 k/s 2000 @ 6879.0 pkts/sec and 4734.7 k/s [...]
All these tools have the same -c option as esic. They also have several options in common amongst themselves:
-s<IP address>Source IP address. Can be set to random with the value
rand.-d<IP address>Destination IP address. Can be set to random with the value
rand.Be careful with this option. Since most of the destination IP addresses that get generated do not exist, the device will only send ARP packets, not the packets generated by ISIC. If you have the MAC address for all these devices but are using an additional interface for remote access to the server, part of the traffic generated by ISIC will be sent through the management interface according to the routes set up on the device.
-k<packets>Number of packets to skip. Used for automation.
-x<resend>Number of times to send each packet. The default value is
1.-m<throughput>Maximum throughput in kilobytes per second.
-p<packets>Number of packets to generate. Note that it has a different meaning than the
-poption in esic. Mainly used for automation.
You can specify a couple of parameters to generate more or less malformed and fragmented traffic, such as:
-F<fragments>Percentage of fragmented packets. The default is 30 percent.
-V<Bad IP version>Percentage of bad IP versions. The default is 10 percent.
Each utility also has tool-specific options:
- isic
-I<oddIPheaderlength>sets the percentage of odd IP header length. The default is 50 percent.- icmpsic, tcpsic, udpsic
-I<IPoptions>sets the percentage of packets with random IP options. The default value is 50 percent.- icmpsic
-I
<badchecksums>sets the percentage of bad checksums. The default value is 10 percent.- tcpsic
-T<TCPoptions> Sets the percentage of packets with random TCP options. The default value is 50 percent.
-u<urgentdata>Sets the percentage of packets with urgent data. The default is 50 percent.
-t<badchecksums>Sets the percentage of bad TCP checksums. The default is 10 percent.
- udpsic, multisic
-U<badchecksums>sets the percentage of bad TCP checksums. The default is 10 percent.- multisic
-i <interface>sets the interface to use.
With icmpsic, tcpsic, and udpsic it is possible to specify a port number along with the source and destination IP address; for example:
[root@replay ˜]#udpsic -s1.1.1.1,rand-d1.1.1.2,53-r600-F80-V10-I50-U5
The ISIC suite is easy to configure and run, but you will have to generate millions of packets to crash a device once, if it crashes at all. Once the device crashes, it is not easy to go back and determine which of the packets caused a problem.
The best way to proceed is to generate a specific number of packets, test the device, and repeat again.
Here is how it would look if you were running one million packets at a time and doing a ping to test whether the target is still up:
[root@replay ˜]#udpsic -s1.1.1.1-d1.1.1.2-r600 -p 1000000Compiled against Libnet 1.1.2.1 Installing Signal Handlers. Seeding with 600 Using random source ports. No Maximum traffic limiter Bad IP Version = 10% IP Opts Pcnt = 50% Frag'd Pcnt = 30% Bad UDP Cksm = 10% 1000 @ 15195.3 pkts/sec and 10053.8 k/s 2000 @ 15393.9 pkts/sec and 10045.1 k/s [...] [root@replay ˜]#ping -n1 1.1.1.1.2Pinging 1.1.1.2 with 32 bytes of data: Reply from 1.1.1.2: bytes=32 time<1ms TTL=6 Ping statistics for 1.1.1.2: Packets: Sent = 1, Received = 1, Lost = 0 (0% loss) Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms [root@replay ˜]#udpsic -s1.1.1.1-d1.1.1.2-r600 -p 1000000-k1000000Will not transmit first 1000000 packets. Compiled against Libnet 1.1.2.1 Installing Signal Handlers. Seeding with 600 Using random source ports. No Maximum traffic limiter Bad IP Version = 10% IP Opts Pcnt = 50% Frag'd Pcnt = 30% Bad UDP Cksm = 10% 1000 @ 15195.3 pkts/sec and 10053.8 k/s [...] [root@replay ˜]#ping -n1 1.1.1.1.2Pinging 1.1.1.2 with 32 bytes of data: Reply from 1.1.1.2: bytes=32 time<1ms TTL=6 Ping statistics for 1.1.1.2: Packets: Sent = 1, Received = 1, Lost = 0 (0% loss) Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms [root@replay ˜]#udpsic -s1.1.1.1-d1.1.1.2-r600 -p 1000000-k2000000Will not transmit first 2000000 packets. [...]
If the ping fails, let's say between batch 2 or 3, you know that a packet between number 1,000,0000 and 2,0000,0000 caused the problem. You can follow the same process with batches of 10,000 packets, then 1,000 packets, and so on, in order to isolate the right packet.
The test never ends, and random packets are generated indefinitely. The total amount of traffic needed to validate a device is up to you. Obviously, the more traffic, the better.
It is not easy to decide how many packets you should run in a row, and what type of test to execute. If you generate only a few packets before doing a test, it will take much longer to run. However, with a large number of packets, if the device crashes or hangs, it will take much longer to isolate the packet that caused the problem.
So the test should be as short as possible. It should detect a crash or a hang, but also a reboot. If you only ping the machine, you do not know if it rebooted during the test. It should also test the particular area that you are looking for. If you test a DNS server with udpsic, the test should also ensure that the DNS server is still behaving correctly. In this case, a ping is not enough.