One of the most amazing things about Linux is just how flexible the command line is. Over the years I’ve learned a lot of useful command-line tricks. Some are time-saving and others are life-saving. No matter how long you’ve used Linux, it seems as if you learn new tips like this all the time. In this appendix I have compiled some of my favorite short command-line tips and tricks.
One of the main ways I use grep is when I am looking for a particular process on the system. The problem is that the grep command itself always seems to match and shows up in the output:
$ ps -ef | grep bash
kyle 982 2077 0 19:50 pts/2 00:00:00 grep bash
kyle 2077 6668 0 Apr06 pts/2 00:00:01 bash
kyle 6859 6668 0 Apr03 pts/1 00:00:00 bash
As you can see here, I have two valid bash processes running, but I get the grep command in my output as well. That can be annoying if you want to count the number of Apache processes on the system, for instance, as you always have to subtract one for the grep command—that is, unless you surround the first character of your keyword with brackets:
$ ps -ef | grep [b]ash
kyle 2077 6668 0 Apr06 pts/2 00:00:01 bash
kyle 6859 6668 0 Apr03 pts/1 00:00:00 bash
This works because of the power of regular expressions. When I surround the first character with brackets, I’m telling grep to search within a character class. Usually you use character classes so you can list a number of different characters to match, but in this case, since there is only one character in it, it acts just as if I were grepping for bash. Of course, since the grep command itself has brackets in it, it doesn’t show up in the result.
There are times when you want to look at a shell script you have in your binary path but you can’t quite remember whether it’s in /bin, /sbin, /usr/bin, /usr/local/bin, or somewhere else. While you could just keep hitting the tab key until you find it, the which command will search through your path and output the full path to the command you list:
$ which vim
/usr/bin/vim
If I wanted to run ls -l against the vim binary but I wasn’t sure what directory it was in, I could just type
$ ls -l `which vim`
lrwxrwxrwx 1 root root 21 2008-03-21 22:22 /usr/bin/vim ->
/etc/alternatives/vim
When you surround a command with backticks (`), it will run the command and the output will appear in its place. I especially like to do this when I want to edit a custom shell script I’ve written and don’t know where I saved it.
Be careful with this command! As a sysadmin you often have systems you need to get rid of and need to erase the hard drives so no private company data gets out. A simple way to do this is to use the dd command to read from the special /dev/zero device and output to the drive you want to erase. When the command is done, the entire drive will be all zeros. So if I wanted to completely erase /dev/sdb, I would type
$ sudo dd if=/dev/zero of=/dev/sdb
Often I will be copying a file from one file system or server to another and I want to monitor the copy progress. While I could just type ls -l over and over, a better method is to use the watch command. The watch program takes a command as an argument and then runs that command every two seconds and shows its output on the screen. So, for instance, if I wanted to monitor the size of ubuntu.iso, I would type watch "ls -l ubuntu.iso". The watch command accepts a few arguments such as the -n argument so that you can adjust how many seconds it will wait before it runs the program again.
When I work I usually have more than one thing going on at a time. When I’m working on a server and need to reboot it, usually I want to know right away when it comes back up so I can log in to it and finish whatever work I was doing. Of course, what usually happens is that I start to reboot a server, then get distracted by other work while I wait for it to come back up and completely forget about it until later. A nice solution to this problem is to use the ping program with the -a argument. The -a argument tells ping to play an audible bell (the beep you hear when you hit Tab on the keyboard sometimes) for every ping response. When a system is up, this just means one beep after another. What I like to do is run ping -a against a particular hostname as I reboot it. Then I can go do some other work and once the server comes back up on the network, my terminal will beep over and over until I stop the ping process.
There are many different ways to search and replace text in a file, from sed and awk scripts to opening the file in a text editor. One of my favorite ways is with what I like to call my “Perl pie” script. If I wanted to replace all instances of kyle with Kyle in a file.txt, I would type:
$ perl -pi -e 's/kyle/Kyle/' file.txt
Since this one-liner uses Perl, it also means that you can take advantage of Perl’s advanced regular expression engine.
A very common command-line need is to locate all the files within a directory and all its subdirectories with a certain attribute and run some command on them. For instance, let’s say I wanted to move all files that end in .txt within my current directory and all the directories below it into the ~/Documents directory. If you wanted to just list all of the files, you would run the following find command:
$ find ./ -name "*.txt"
In your output you would get a list of each file that ended with .txt along with its path. To then execute some program against these files you would use the -exec argument to find:
$ find ./ -name "*.txt" -exec mv {} ~/Documents/ \;
The -exec command will replace the {} in each file with the full path to that file. In this case it would run the mv command against the file and move it to the ~/Documents directory. Notice that strange \; text at the end of the command. That is required when you use the -exec argument so that it knows it is at the end of the command.
Be careful when you run this command. People make mistakes, and often you’ll find either your find output is different from what you thought it would be or you have some mistake in your -exec command. I recommend before you run anything risky that you test your -exec command with echo. That way you see what -exec would run before it actually does anything. For instance, I would test the command above with
$ find ./ -name "*.txt" -exec echo mv {} ~/Documents/ \;
As you start to run more and more find -exec commands, you will eventually run across a situation when there are too many files in the output and will get some sort of error message like “too many arguments.” There are a limited number of arguments you can have in your shell, but luckily if this happens to you, there is a good way around it using the xargs program. The xargs program will accept a set of arguments that are piped to it and will run a command against those arguments one by one. So, for instance, if I got that error in my find command, I could instead type
$ find ./ -name "*.txt" -print0 | xargs mv ~/Documents
While it’s always advisable to document all of your procedures, especially when you fix a problem, sometimes you don’t get around to it and have to figure out what you did the last time to fix a system. It can be easy to forget that bash logs all of your previous commands to its history file. One of the first things I do when I can’t remember exactly how I ran a command previously is type history and look through the output for more details.
As you troubleshoot problems on a system, you commonly wonder whether the problem stems from one server having a different configuration file from another working server. While you can definitely open both files and compare them line by line or run diff against them, both methods can be tedious. A quick way to test whether two files are identical is to use the md5sum tool to create a checksum of both. If the output matches, the files are identical:
$ md5sum bar
d41d8cd98f00b204e9800998ecf8427e bar
$ md5sum foo
d41d8cd98f00b204e9800998ecf8427e foo
This is a nice quick tip. When you move around directories on a system, sometimes it would be nice if you could quickly jump back to the directory you were previously in. Just type
$ cd -
Bash will replace the - symbol with the value of $OLDPWD—an environment variable bash uses to keep track of your previous directory.
Linux won’t let you unmount a file system if a file is opened on that file system. Often you will see this if you have a shell in a directory on that file system. In that case you can fix the problem just by changing to a different directory, and then you can unmount the file system; but sometimes you aren’t sure what program is tying it up. In that case you can use the lsof command. The lsof command will list all of the open files on the system. All you need to do is grep for the file system you want to unmount and you will see what processes have open files on that file system. Let’s say, for instance, that I want to unmount /media/cdrom. I would type
$ sudo lsof | grep /media/cdrom
In the lsof output I would see a list of all of the processes with open files on that file system, their process IDs, and even what files they had opened. Then I could close that program or kill the process and unmount the file system.
A mail server listens on port 25 for SMTP connections, and when you connect to that port, if you know what SMTP commands to issue, you can pretend to be another mail server. A very handy trick is to be able to test a mail server just by connecting to port 25 on the machine with telnet and typing a raw e-mail message:
$ telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 minimus ESMTP Postfix (Ubuntu)
HELO kyle
250 minimus
MAIL FROM: kyle@example.net
250 2.1.0 Ok
RCPT TO: kyle@localhost
250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
Subject: test
This is just a test
.
250 2.0.0 Ok: queued as 1ADD057CD0
quit
221 2.0.0 Bye
Connection closed by foreign host.
The first command you type is HELO followed by the name you want the system to know you as. Next type MAIL FROM: followed by the e-mail address you want the e-mail to appear to come from. After that type RCPT TO: followed by the e-mail address you want to send to. Then type DATA. At that point everything else you type will be considered the body of the e-mail. In my example I added a Subject: field but even that is optional. Just continue to type out your message and then when you are finished, type enter then type a period, then type enter again. Finally type quit to end the session.
One of the annoying parts of setting up key authentication on SSH is having to move all the public keys around to each system’s authorized_keys file. For years I would do some fancy shell scripting to copy keys around, but then I found out that the openssh packages included a tool called ssh-copy-id just for that purpose. To use the program just type ssh-copy-id followed by the remote server (and optionally the remote username) you want the key copied to. For instance, if I wanted to copy my current user’s key to a server named web1, I could type:
$ ssh-copy-id web1
Or if I wanted to copy the key to the kyle user’s account on a server named db4, I would type:
$ ssh-copy-id kyle@db4
A number of people (myself included) still use nslookup for a number of DNS queries even though it’s been deprecated for years, just because it’s what we are used to using. Dig, after all, has different syntax, and usually you get a lot of output you have to pore through to find the info you wanted. That is true unless you use dig’s +short option. With that option dig will give you just the output you want:
$ dig www.ubuntu.com +short
91.189.90.40
Another great option for dig is the +trace option. Usually dig will give you the final result you are searching for, but what you may not realize is that there are a number of DNS servers between your query and the answer (if the answers aren’t cached). If you add +trace to a dig query, it will act a lot like traceroute in that it will trace through the dig query and show you all the hops your DNS request took.
$ dig www.ubuntu.com +trace
; <<>> DiG 9.6.1-P2 <<>> www.ubuntu.com +trace
;; global options: +cmd
. 163547 IN NS b.root-servers.net.
. 163547 IN NS c.root-servers.net.
. 163547 IN NS d.root-servers.net.
. 163547 IN NS e.root-servers.net.
. 163547 IN NS f.root-servers.net.
. 163547 IN NS g.root-servers.net.
. 163547 IN NS h.root-servers.net.
. 163547 IN NS i.root-servers.net.
. 163547 IN NS j.root-servers.net.
. 163547 IN NS k.root-servers.net.
. 163547 IN NS l.root-servers.net.
. 163547 IN NS m.root-servers.net.
. 163547 IN NS a.root-servers.net.
;; Received 244 bytes from 192.168.0.1#53(192.168.0.1) in 5 ms
com. 172800 IN NS H.GTLD-SERVERS.NET.
com. 172800 IN NS L.GTLD-SERVERS.NET.
com. 172800 IN NS D.GTLD-SERVERS.NET.
com. 172800 IN NS F.GTLD-SERVERS.NET.
com. 172800 IN NS C.GTLD-SERVERS.NET.
com. 172800 IN NS B.GTLD-SERVERS.NET.
com. 172800 IN NS K.GTLD-SERVERS.NET.
com. 172800 IN NS J.GTLD-SERVERS.NET.
com. 172800 IN NS A.GTLD-SERVERS.NET.
com. 172800 IN NS M.GTLD-SERVERS.NET.
com. 172800 IN NS G.GTLD-SERVERS.NET.
com. 172800 IN NS E.GTLD-SERVERS.NET.
com. 172800 IN NS I.GTLD-SERVERS.NET.
;; Received 504 bytes from 192.58.128.30#53(j.root-servers.net)
in 110 ms
ubuntu.com. 172800 IN NS ns1.canonical.com.
ubuntu.com. 172800 IN NS ns2.canonical.com.
ubuntu.com. 172800 IN NS ns3.canonical.com.
;; Received 144 bytes from 192.41.162.30#53(L.GTLD-SERVERS.NET)
in 91 ms
www.ubuntu.com. 600 IN A 91.189.90.41
ubuntu.com. 172800 IN NS ns1.canonical.com.
ubuntu.com. 172800 IN NS ns2.canonical.com.
ubuntu.com. 172800 IN NS ns3.canonical.com.
;; Received 160 bytes from 91.189.94.173#53(ns1.canonical.com)
in 151 ms