Everyone Should Learn Some Shell Programming

One of the great things about Unix is that it's made up of individual utilities, "building blocks" like cat and grep, that you run from a shell prompt. Using pipes, redirection, filters, and so on, you can combine those utilities to do an incredible number of things. Shell programming lets you take the same commands you'd type at a shell prompt and put them into a file you can run by just typing its name. You can make new programs that combine Unix programs (and other shell scripts) in your own way to do exactly what you need. If you don't like the way a program works, you can write a shell script to do just what you want.

Because many Unix users use the shell every day, they don't need to learn a whole new language for programming, just some tips and techniques. In fact, this chapter covers a lot of programming techniques that you'll want to use even when you aren't programming. For example, loops and tests are handy on the command line.

(This series of articles does assume that you've written programs in some language before or are generally familiar with programming concepts. If you haven't and aren't, you might start with a more comprehensive shell programming book.)

Summary Box

Unix has plenty of other scripting languages — Perl, Python, and Tcl/Tk are some of the best known. So when should you write a script with the shell and when shouldn't you? That's a personal choice; as you learn more languages and their strengths and weaknesses, you're better able to choose the best one for a situation. My rule of thumb is something like this. I write a shell script if:

• It's a script I developed at the command line, so it's easy to just drop those same commands into a file.

• I know some Unix utility that'll do just what I want.

• It has to be portable to a system that might not have another scripting language I'd rather use.

• The (possibly) slower speed of forking processes to run Unix utilities (especially in loops) doesn't matter.

• The script simply has to make a few decisions — like whether standard input is a tty (Section 2.7), checking options and arguments, or something else simple) — before the script ends by running some Unix utility.

• It just feels natural to write a shell script, for whatever reason.

On the other hand, maybe your script needs lots of pipes ( | ) (Section 1.5) or temporary files, or you have out-of-band data that you have to keep passing in to each Unix utility (maybe because you can't shoehorn multiple types of data into a single chain of pipelines between utilities). In that case, you'll be happier with a scripting language that doesn't depend on Unix utilities and pipes.

Some of the topics you need to learn about as a beginning shell programmer have already been covered in other chapters. Here are the articles you'll probably want to read, in an order that makes sense if you're looking for something of a tutorial:

• To see how to write a simple shell program, Section 35.1. To embed scripts from other languages such as sed and awk in a shell script, Section 35.19.

• For explanation of shells in general, Section 27.3.

• To read about environment and shell variables, Section 35.3 and Section 35.9, respectively.

• Shell quoting is explained in Section 27.12.

• Stepping through arguments or any list of words with a for loop is discussed in Section 28.9 (as well as in Section 35.21, later in this chapter).

Then, once you've had your refresher, come on back and read the following articles:

• Test strings with a case statement, Section 35.10. Match patterns in a case statement, Section 35.11.

• Use the output of one command as arguments to another command with command substitution, Section 28.14.

• Find out whether a program worked or failed with its exit status, Section 35.12.

• Loop through a set of commands and use another command to control that loop, Section 35.15.

• Set exit status of a shell (shell script), Section 35.16.

• Handle interrupts (like CTRL-c) and other signals, Section 35.17.

• Read input from the keyboard, Section 35.18.

• Handle command-line arguments (options, filenames, etc.), Section 35.20.

• Test a program's exit status and do different things if it worked or failed, Section 35.13 and Section 35.14.

• Handle arguments with the while and shift commands, Section 35.22.

• Handle command-line arguments in a more standard and portable way with getopt, Section 35.24.

• Set shell options and command-line arguments with the set command, Section 35.25.

• Test files and strings of characters with the test command, Section 35.26.

• Pick a name for a new command with no conflict, Section 35.27.

• Find the name of a program and use it in the script, Section 35.28.

• Use "subprograms" that can change the current environment, Section 35.29.

This chapter discusses only Bourne shell programming. We don't cover many features from more advanced Bourne-type shells, like bash and zsh, because those can make your shell scripts nonportable; we stick to concepts that should work almost anywhere. Also, in most cases, the C shell isn't great for shell programming.

A note about command versions: unfortunately, the same commands on different versions of Unix can have different options. Some Bourne shells are a little different from others. For instance, some test (Section 35.26) commands have a -x option to test for an executable file; others don't. Some echo commands use a -n option to mean "no newline at the end of this string"; others have you put \c at the end of the string. And so on. Where there are differences, these articles generally use the commands in original Berkeley Unix from the 1980s. If a command doesn't seem to work on your system, check its online manual page or the sh manual page.

— JP