Because the serial line is a direct line to the kernel, bypassing firewalls and intrusion detectors, you want to set up some barriers against intruders, especially for dial-in administration.
Here are some ways to improve the security of a serial line:
Direct all logging to a remote logging server.
Use a higher-end modem with built-in security features such as automatic callbacks, caller line identification, and approved caller lists.
Don't use a phone line with a publicly listed number.
Disable the SysRq key
with this line in /etc/sysctl.conf: kernel.sysrq=0.
AT S2=255 disables the
modem's command mode, but a remote attacker could possibly reset
it to command mode, then input their own nefarious commands. A
modem that uses DIP switches or jumpers to put it in and out of
command mode is great for preventing this sort of attack.
Your first line of defense is an obscure phone number. Security-through-obscurity has its place in your security architecture; don't make it easy for unsophisticated troublemakers. But this will not foil a war-dialer. If a war-dialer should target your network, it won't take long to discover which phone lines have modems on them. Then, to gain access, they'll need to get past the initial login. A cracker can cause you grief simply by repeatedly dialing the server's phone number—denial-of-service attacks are easy to launch and difficult to defend against. High-end modems like the U.S. Robotics Courier have security features that help against a persistent attacker, such as automatic callback and approved caller lists. You can't stop a cracker from dialing your number, but you can try to stop them from getting to a login prompt.
The SysRq key allows the user
to send commands directly to the kernel. It is used primarily by
kernel developers; otherwise, there is no good reason to leave it
active. Check to see if support for it is compiled into your kernel.
Look in your /boot/config-* file under Kernel
hacking:
CONFIG_MAGIC_SYSRQ=y
This means it is. Another option is to rebuild the kernel and remove support for it; if you're not doing kernel hacking, there is no reason to have it available.
Remote Serial Console HOWTO:
The Serial HOWTO goes extremely in-depth into how serial ports work: