Linux kernel rootkits such as Adore or Knark (an LKM rootkit) are hard to detect since they use a kernel module to hide their behavior. Even software such as Tripwire or an anti-virus solution running on the infected system will not see the presence of these rootkits.
Since these rootkits use the power of the kernel to hide themselves, they are only visible from within the kernel. Kstat (Kernel Security Therapy Anti-Trolls) and Zeppoo allow the user to gather important information about the system, especially the functional call address that is often overwritten by these Linux rootkits. Kstat is currently supported only on Linux kernel 2.4.x, and Zeppoo is currently only supported on kernel 2.6.x. You can find Kstat at http://freshmeat.net/projects/kstat/ and Zeppoo at http://www.zeppoo.net.
In this section, we'll cover some of the common tasks with Kstat, including looking up network interfaces, listing processes from memory, investigating individual processes, and examining the syscall table.
To look up the interface, use:
./kstat -i allThe -i option takes as an argument the network interface, which may be specified as all or as a name (e.g., eth0), and Kstat displays information about the queried interface.
What is interesting to notice here is that the interface could be put in promiscuous mode. This means that the interface could be used as a sniffer for the whole hub/switch.
To list the process from memory, use the -p argument, which displays all running processes from a memory dump:
PID PPID UID GID COMMAND
1 0 0 0 init
2 1 0 0 kflushd
3 1 0 0 kupdate
4 1 0 0 kpiod
5 1 0 0 kswapd
6 1 0 0 mdrecoveryd
256 1 1 0 portmap <<hidden process>>Here we are trying to identify processes that are present in this list, but not from standard output such as ps. If such processes exist, they are currently hidden from the user and should be investigated.
The -p argument followed by the wanted process ID returns information about that specifc process. The information is gathered directly from the memory.
Command: Kstat -p 256
Name: portmap
State: S (sleeping)
Pid: 256
Ppid: 1 (init)
Uid: 1 1 1 1
Gid: 0 0 0 0
Flags: PF_FORKNOEXEC PF_SUPERPRIV
Crucial Capabilities Check
Open Files
0 CHAR /dev/null
1 CHAR /dev/null
2 CHAR /dev/null
3 0.0.0.0:111 0.0.0.0:0
4 0.0.0.0:111 0.0.0.0:0
7 FIFO ///
8 FIFO ///
21 CHAR /dev/nullThe -s argument displays info about the system call table. The system call table is the usual hooked place for many rootkits. Hooking (changing the address) of these functions allows the rootkit to replace the real existing function by one of their own, thus affecting the behavior of all "user land" applications using those system calls:
Command : kstat -s
## Output of a normal system with hooking rootkit.
SysCall Address
sys_exit 0xc0117ce4
sys_fork 0xc0108ebc
sys_read 0xc012604c
sys_write 0xc0126110
sys_open 0xc0125c10
sys_close 0xc0125d60
sys_waitpid 0xc0117ff8
sys_creat 0xc0125ca4This is the output on a system where a hooking rootkit is used:
SysCall Address sys_exit 0xc0117ce4 sys_read 0xc012604c sys_open 0xc0125c10 sys_waitpid 0xc0117ff8 sys_creat 0xc0125ca4 sys_fork 0xc4051428 WARNING! Should be at 0xc0108c88 sys_write 0xc4051590 WARNING! Should be at 0xc01269b8 sys_close 0xc405163c WARNING! Should be at 0xc01264a4 sys_kill 0xc40514d0 WARNING! Should be at 0xc011060c sys_mkdir 0xc405172c WARNING! Should be at 0xc012e540
The syscall table is a good place to look for Linux rootkits. Since Linux uses system calls to interface between the user and the kernel, rootkits tend to hijack these addresses to hide their presence from the user. Note however that this method works best when you are able to compare gathered syscall addresses with those that were saved previously in a known clean state. Doing this test after the post-infection would indicate only that the addresses of the system calls look too far from the other ones, but that is all.
Having system calls that are out of bound (i.e., in a range too wide for normal operation) is usually a sign that something unusual is going on. In the compromised example shown here, some system calls are in memory addresses from c0117ce4 to c012604c, and others are in c405xxxx. This range is wide considering that system calls should not be spread around.
Zeppoo uses the device /dev/kmem to perform tasks similar to Kstat. However, Zeppoo requires that you generate a table of fingerprints of your kernel before being able to detect a rootkit. This table must be generated from a clean state; otherwise, a previously installed rootkit could be mistaken as being the clean state. The information output provided by Zeppoo is in the same class as Kstat, so if you understand the output of Kstat, understanding the output of Zeppoo should be very similar. Below is a quick list of Zeppoo's functions.
To generate the table of fingerprints, use:
./zeppoo -f FPTo check the system, use:
zeppoo -z FPTo list processes by /dev/mem dumping, use:
zeppoo -p -d /dev/memTo check for hidden processes, use:
zeppoo -c -pTo get a visualization of the names of IDT with a specific System.map, use:
zeppoo -i -t /boot/System.map-2.6.16To check the fingerprint using /dev/mem, use:
zeppoo -c -f FP -d /dev/kmemChkrootkit uses different techniques to try to identify the presence of rootkits. These techniques include, but are not limited to, hash comparison, string searches, hidden file checks, and some OS-dependent features such as ps versus /proc compare. Doing multiple checks, instead of relying on one technology, offers the best opportunity to find multiple kinds of rootkits. However, since this software comes from the user, rootkits can still evade it.
Here's a sample of Chkrootkit checking:
# ./chkrootkit
ROOTDIR is '/'
Checking 'amd'... not found
Checking 'basename'... not infected
Checking 'biff'... not found
Checking 'chfn'... not infected
....
Checking 'write'... not infected
Checking 'aliens'... no suspect files
Searching for sniffer's logs, it may take a while... nothing found
Searching for HiDrootkit's default dir... nothing found
Searching for t0rn's default files and dirs... nothing found
....
Searching for ESRK rootkit default files... nothing found
Searching for rootedoor... nothing found
Searching for ENYELKM rootkit default files... nothing found
Searching for anomalies in shell history files... nothing found
Checking 'asp'... not infected
....
Checking 'z2'... chklastlog: nothing deleted
Checking 'chkutmp'...
The tty of the following user process(es) were not found in /var/run/utmp !
! RUID PID TTY CMD!
root 1988 tty7 /usr/bin/Xorg :0 -audit 0 -auth /var/gdm/:0.Xauth -nolisten
tcp vt7
chkutmp: nothing deletedIf Chkrootkit cannot find a known signature inside a file, note that it cannot automatically determine whether it has been Trojaned. Run Chkrootkit in expert mode (use the -x option) to examine suspicious strings in the binary programs that may indicate a Trojan. For example, a lot of data can be seen with:
# ./chkrootkit -x | moreOr, to see pathnames inside system commands:
# ./chkrootkit -x | egrep '^/'Chkrootkit uses basic tools such as ls, awk, grep, and sed to perform tests. Since it is not always safe to use these binaries, it is better to use read-only binaries; e.g., those found on a CD-ROM. Follow one of these alternatives shown next:
Use the -p path option to supply an alternate path to binaries you trust:
# ./chkrootkit -p /cdrom/binMount the compromised machine's disk on a machine you trust and specify a new root directory with the -r rootdir option:
# ./chkrootkit -r /mntFinally, Chkrootkit could be run from a cron job. Doing so allows the administrator to have a report from Chkrootkit on a regular basis about the state of the servers, so actions could be taken soon after an intrusion and not months after. You can create a cronjob script like this:
cd/path/to/chkrootkit;./chkrootkit 2>&1 | mail -s "chkrootkit output" root