The fragmentation of the Unix marketplace has had its advantages and disadvantages. The advantages came mostly in the early days: lack of standardization and proliferation among technically savvy academics and professionals contributed to a healthy “free market” for Unix software, in which several programs of the same type (e.g., shells, text editors, system administration tools) would often compete for popularity. The best programs would usually become the most widespread, while inferior software tended to fade away.
But often there was no single “best” program in a given category, so several would prevail. This led to the current situation, where multiplicity of similar software has led to confusion, lack of compatibility, and — most unfortunate of all — Unix’s inability to capture as big a share of the market as other operating platforms. In particular, Unix has been relegated to its current position as a very popular operating system for servers, but it’s a rarity on desktop machines.
The “shell” category has probably suffered in this way more than any other type of software. As we said in the preface and Chapter 1, it is one of the strengths of Unix that the shell is replaceable, and thus a number of shells are currently available; the differences between them are often not all that great. We believe that the Korn shell is one of the best of the most widely used shells, but other shells certainly have their staunch adherents, so they aren’t likely to fade into obscurity. In fact, it seems that shells, Bourne-compatible or not, continue to proliferate.
Therefore we felt it necessary to include information on shells similar to the Korn shell. This Appendix summarizes the differences between the Korn shell and the following shells:
The System V Release 4 Bourne shell, as a kind of baseline
The 1988 version of the Korn shell
The IEEE POSIX 1003.2 Shell Standard, to which the Korn shell and other shells adhere
The Desk Top Korn shell (dtksh), a Korn shell with enhancements for X Window System programming, as part of the Common Desktop Environment (CDE)
The tksh shell, an interesting blend of ksh93 with Tcl/Tk
pdksh, a widely used public domain version of the Korn shell
The bash shell, another enhanced Bourne shell with some C shell and Korn shell features
The Z shell, zsh, yet another enhanced Bourne shell with some C shell and Korn shell features and many, many more of its own
Korn shell workalikes on desktop PC platforms
The Korn shell is almost completely backward-compatible
with the Bourne shell.
The only significant feature of the latter that
the Korn shell doesn’t support is ^ (caret) as a synonym
for the pipe (|) character.[131]
This is an archaic feature that
the Bourne shell includes for its own backward compatibility
with earlier shells. No modern Unix version has any shell code
that uses ^ as a pipe.
To describe the differences between the Bourne shell and the Korn shell, we’ll go through each chapter of this book and enumerate the features discussed in the chapter that the Bourne shell does not support.
The cd
old new
and cd - forms of the cd command.
Tilde (~) expansion.
The Bourne shell always follows the physical file layout, which affects
what happens when you cd .. out of somewhere that
was a symbolic link.
The built-in commands don’t have online help.
Some older versions of the Bourne shell don’t support the jobs command and job control, or they may require being invoked as jsh in order to enable job control features.
All (i.e., the Bourne shell doesn’t support any of the history and editing features discussed in Chapter 2).
Aliases are not supported.
set -o options don’t work. The Bourne shell supports the abbreviations listed in the “Options” table in Appendix B, except -A, -b, -C, -m, -p, and -s.
Environment files aren’t supported; neither is the print command (use echo instead). The following built-in variables aren’t supported:
.sh.edchar
|
.sh.version
|
HISTEDIT
|
LINENO
|
PS4
|
.sh.edcol
|
COLUMNS
|
HISTFILE
|
LINES
|
PWD
|
.sh.edmode
|
EDITOR
|
HISTSIZE
|
OLDPWD
|
RANDOM
|
.sh.edtext
|
ENV
|
LANG
|
OPTARG
|
REPLY
|
.sh.match
|
FCEDIT
|
LC_ALL
|
OPTIND
|
SECONDS
|
.sh.name
|
FIGNORE
|
LC_COLLATE
|
PPID
|
TMOUT
|
.sh.subscript
|
FPATH
|
LC_CTYPE
|
PS3
|
VISUAL
|
.sh.value
|
HISTCMD
|
LC_NUMERIC
|
Some of these variables (e.g., EDITOR and VISUAL)
are still used by other programs, like mail and news readers.
Extended variable names (those with a dot in them), as well as compound
variable assignment are not available, nor is string concatenation
with the += operator.
Indirect variables (namerefs) are not available.
The whence command is not available; use type instead.
The pattern-matching variable operators
(%, %%, #, ##, etc.)
and advanced (regular expression) wildcards are not available; use
the external command expr instead.
Autoloaded functions are not available, and
only POSIX-style functions (those defined using the name
()
syntax) may be used.
Command substitution syntax is different: use the older
`
command
`
instead of
$(
command
).
(Some vendors have enhanced their Bourne shells to support the
$(
command
)
notation, since it’s defined by the POSIX standard.)
return may only be used from within a function.
Conditional tests use older syntax:
[
condition
] or
test
condition
instead of [[
condition
]].
These are actually two forms of the same command
(see the test(1) manual page).
The logical operators
&& and ||
are -a and -o instead.
Supported test operators differ from system to system.[132]
The ! keyword
to reverse the exit status of a command
was not in the SVR4 Bourne shell,
although it may be available on your system, since it is
required by POSIX.
The select construct isn’t supported.
Neither is the arithmetic for loop,
and there is no way to fall through from one case to another
inside a case statement.
There is no equivalent for TMOUT.
The SVR4 Bourne shell getopts command is similar, but not identical to, that in ksh. It does not allow options that begin with a plus sign, nor any of the advanced features described in Appendix B.
Arithmetic isn’t supported: use the external command
expr instead of
the $((...)) syntax.
For numeric conditionals, use the old condition test syntax
and relational operators -lt, -eq, etc., instead of
((...)).
let isn’t supported.
Array variables and the typeset command are not supported.
The following I/O redirectors are not supported:[133]
>|
|
<>
|
<<<
|
<&p
|
>&p
|
<&
n
-
|
>&
n
-
|
|&
|
print isn’t supported (use echo instead). printf is generally available as an external command.
None of the options to read are supported, nor is the ability to supply a prompt with the first variable’s name.
The Bourne shell does not do special interpretations of pathnames, such as /dev/fd/2 or /dev/tcp/ftp.oreilly.com/ftp.
The $"..."
and $'...'
quoting mechanisms are not available.
Job control is supported, but only if the shell is invoked with the -j option or as jsh.
The - option to trap (reset trap to
the default for that signal) is not available.
Instead, a missing trap indicates that the supplied traps should
be reset.
trap accepts only signal
numbers, not logical names.
Coroutines aren’t supported.
The wait command only accepts process IDs.
The ERR and DEBUG fake signals are not available. The EXIT fake signal is supported, as signal 0.
set -n takes effect even in interactive shells.
The output from tracing with set -x is
not customizable (no PS4 variable).
Discipline functions are not available.
Privileged mode isn’t supported. (Some Bourne shells do have it.)
The ulimit and umask commands are not as capable.
Perhaps the most obvious shell with which to compare ksh93 is ksh88, the 1988 version of the Korn shell. This section briefly describes those features of ksh93 that are either different or nonexistent in ksh88. As with the presentation for the Bourne shell, the topics are covered in the same order as they’re presented in the rest of the book.
The built-in help facilities (such as -?, --man, and so on) are not available.
Tilde substitution does not occur during variable expansion
(${
var
op
word
}).
CTRL-T works differently in emacs editing mode. In ksh88, it transposes the two characters to the right of point and moves point forward by one character.
In emacs-mode,
ESC ESC, ESC *, and ESC = don’t work on the
first word of a command line, i.e., there is no command completion facility.
The ESC # command always prepends a #
character. It never removes them.
Similarly, in vi-mode, the *, \
and = commands don’t work on the first word of a command line,
and the # command always prepends a #
character. It never removes them.
Finally, there is no variable completion, the hist
command is called fc,
FCEDIT is used in place of HISTEDIT,
and the HISTCMD
variable is not available.
The ENV file is
sourced for all shells, and ksh88 only does
variable substitution on the value of $ENV.
Alias tracking can be turned off in ksh88.
The -p option to alias is not present in ksh88, nor is the -a option to unalias.
The following built-in variables aren’t supported:
.sh.edchar
|
.sh.match
|
.sh.version
|
LANG
|
.sh.edcol
|
.sh.name
|
FIGNORE
|
LC_ALL
|
.sh.edmode
|
.sh.subscript
|
HISTCMD
|
LC_COLLATE
|
.sh.edtext
|
.sh.value
|
HISTEDIT
|
LC_CTYPE
|
In ksh88, both syntaxes for defining functions produce functions with Korn shell semantics. You cannot apply the dot command to a function name.
The command search order in ksh88 is keywords, aliases, all built-in commands, functions, and then external commands and scripts. The order was changed in ksh93 for POSIX compliance.
In ksh88, undefined (autoloaded) functions are searched
for exclusively along the list in the FPATH variable, and
PATH is not involved.
The .paths file feature is not available.
Many of the variable substitution features described in the main text are new
to ksh93. Only the following are available in ksh88:
${
name
:-
string
},
${
name
:=
string
},
${
name
:?
string
},
${
name
:+
string
},
${
name
#
pattern
},
${
name
##
pattern
},
${
name
%
pattern
},
${
name
%%
pattern
},
${#
name
},
${#
name
[*]},
and
${#
name
[@]}.
Compound variables, namerefs and
the += operator for appending to a variable
are not in ksh88.
The \n notation in patterns is not available,
nor are
non-greedy matching, the
use of backslash escapes in patterns,
options in subpatterns, nor any of the [:...:]
character classes.
The .sh.match array is not available.
Exit values in ksh88 only go up to 128. Programs that die due to a signal have exit status 128 plus signal number.
There is no command built-in command. To replace a built-in, you have to use an awkward combination of aliasing, functions, and quoting.
The set -o pipefail option and the
the ! keyword for reversing the sense of a test are
not available. Instead of the
== operator for [[...]],
ksh88 uses =.
The arithmetic for loop is not in ksh88.
The use of ;& for falling through cases existed in
ksh88 but was undocumented.
The TMOUT variable existed in ksh88,
but it only applied to the shell itself, not to the select
loop or the read command.
The ksh88 getopts did not have the ability to specify numeric arguments to options, nor a way to specify optional arguments to options.
The built-in arithmetic only supports integers, and the
++, --,
?: and comma operators aren’t available.
Numeric constants of the form
base
#
number
can only go up to base 36. There are no built-in arithmetic functions.
Only indexed arrays exist, and the maximum index is 1023.
The following I/O redirectors are not supported:
<&
n
-
|
>&
n
-
|
<<<
|
TCP/IP networking is available starting with ksh88e, but you must use IP addresses.
ksh88 does not have the printf command,
nor the -A, -d,
-n, and -t options to read.
TMOUT does not affect read.
Brace expansion and process substitution are compile-time options that are generally not available in ksh88.
Locale translation with $"..." and
ANSI C strings with $'...' are not available.
The disown command is not available, and neither are the -n and -s options to kill. kill -l can only be used by itself to list the available signals.
true and false are predefined aliases instead of built-in commands.
Functions and aliases are exported to shell subprocesses; this is not true of ksh93.
Traps for the DEBUG fake signal are executed after each command is executed, not before.
Discipline functions are not available.
The umask command only works with octal masks.
You cannot customize the built-in editors; the KEYBD fake signal does not exist.
There have been many attempts to standardize Unix. Hardware companies’ monolithic attempts at market domination, fragile industry coalitions, marketing failures, and other such efforts are the stuff of history — and the stuff of frustration.
Only one standardization effort has not been tied to commercial interests: the Portable Operating System Interface, known as POSIX. This effort started in 1981 with the /usr/group (now UniForum) Standards Committee, which produced the /usr/group Standard three years later. The list of contributors grew. Eventually, the effort to create a formal standard moved under the umbrella of the Institute of Electrical and Electronic Engineers (IEEE) and the International Organization for Standardization (ISO).
The first POSIX standard was published in 1988 and revised in 1996. This one, called IEEE Standard 1003.1, covered low-level issues at the system call level. IEEE Standard 1003.2, covering the shell, utility programs, and user interface issues, was ratified in September 1992 after a six-year effort. In September 2001, a joint revision of both standards was approved. The new standard, covering all the material in the two earlier separate documents, is known as IEEE Standard 1003.1-2001.
The POSIX standards were never meant to be rigid and absolute. The committee members certainly weren’t about to put guns to the heads of operating system implementers and force them to adhere. Instead, the standards are designed to be flexible enough to allow for both coexistence of similar available software, so that existing code isn’t in danger of obsolescence, and the addition of new features, so that vendors have the incentive to innovate. In other words, they are supposed to be the kind of third-party standards that vendors might actually be interested in following.
As a result, most Unix vendors currently comply with both standards. The Korn shell is no exception; it is intended to be 100% POSIX-compliant. It pays to be familiar with what’s in the standard if you want to write code that is portable to different systems.
The shell part of the standard describes utilities that must be present on all systems, and others that are optional, depending upon the nature of the system. One such option is the User Portability Utilities option, which defines standards for interactive shell use and interactive utilities like the vi editor. The standard — on the order of 2000 pages — is available through the IEEE; for information, contact the IEEE:
IEEE Customer Service
445 Hoes Lane, PO Box 1331
Piscataway, NJ 08855-1331
(800) 678-IEEE (United States and Canada)
(732) 981-0060 (international/local)
(732) 981-9667 (fax)
customer.service@ieee.org http://www.standards.ieee.org/catalog/ordering.html
The committee members had two motivating factors to weigh when they designed the shell standard. On the one hand, the design had to accomodate, as much as possible, existing shell code written under various Bourne-derived shells (the Version 7, System V, BSD, and Korn shells). These shells are different in several extremely subtle ways, most of which have to do with the ways certain syntactic elements interact with each other.
It must have been quite difficult and tedious to spell out these differences, let alone to reach compromises among them. Throw in biases of some committee members towards particular shells, and you might understand why it took six years to ratify the first 1003.2 standard and another five years to merge the two standards.
On the other hand, the shell design had to serve as a standard on which to base future shell implementations. This implied goals of simplicity, clarity, and precision — objectives that seem especially elusive in the context of the above problems.
The designers found one way of ameliorating this dilemma: they decided that the standard should include not only the features included in the shell, but also those explicitly omitted and those included but with unspecified functionality. The latter category allows some of the existing shells’ innovations to “sneak through” without becoming part of the standard, while listing omitted features helps programmers determine which features in existing shell scripts won’t be portable to future shells.
The POSIX standard is primarily based on the System V Bourne shell. Therefore you should assume that Korn shell features that aren’t present in the Bourne shell also aren’t included in the POSIX standard.
However, the Korn shell did contribute a few of its features to the POSIX standard, including:
$((...)) syntax for arithmetic expressions.
$(...) syntax for command substitution, except
that the $(<
filename
)
shorthand for
$(cat
filename
) isn’t supported.
Tilde expansion (originally derived from the C shell).
The following Korn shell features are left “unspecified” in the standard, meaning that their syntax is acceptable but their functionality is not standardized:
The ((...)) syntax for arithmetic conditionals.
The arithmetic test operators introduced in Chapter 5
(e.g., -eq, -lt), however, are included.
The [[...]] syntax for conditional tests.
The external test or [...]
utility should be used instead.
The Korn shell’s version of test is POSIX-compliant when
used with no more than three arguments.
(It also complies with four arguments, if the first argument is !.)
The syntax for defining functions that this book uses.
The other syntax shown in Chapter 4 (fname
() instead of
function
fname) is supported,
with what we described as “POSIX semantics;”
see below.
The select control structure.
Signal numbers are only allowed if the numbers for certain key signals (INT, TERM, and a few others) are the same as on the most important historical versions of Unix. In general, shell scripts should use symbolic names for signals.
The POSIX standard supports functions, but the
semantics are weaker than the Korn shell’s
function-style functions: functions do not have local
traps or options, and it is not possible to define local
variables.
(For this reason, ksh93 has two different syntaxes
for defining functions, with different semantics.)
Code blocks ({...; }) are
supported. For maximum portability, when you want literal
curly braces, you should quote them
(for reasons too complicated to go into here).
The POSIX standard introduced the following features, which are different from traditional Bourne shell behavior. ksh93 supports them all:
The command lookup order has been changed to allow
certain built-in commands to be overridden by functions — since
aliases aren’t included in the standard.
Built-in commands are divided into two
sets by their positions in the command lookup order:
some are processed before functions, some after.
Specifically, the built-in commands
break, : (do nothing),
continue, . (dot), eval,
exec, exit,
export, readonly, return,
set, shift,
trap, and unset take priority over functions.
A new built-in command, command, allows you to use built-in commands other than the above even if there are functions of the same name.[134]
A new keyword, !, takes the logical
“not” of a command’s exit status: if command returns
exit status 0, !
command returns 1;
if command
returns a non-zero value,
!
command returns 0.
! can be used with
&&, ||, and parentheses (for nested subshells)
to create logical combinations of exit statuses in conditionals.
The command unset -v is used instead of unset
(without an option) to remove the definition of a variable.
This provides a better syntactic match with unset -f,
for unsetting functions.
Finally, because the POSIX standard is meant to promote shell script portability, it explicitly avoids mention of features that only apply to interactive shell use — including aliases, editing modes, control keys, and so on. The User Portability Utilities option covers these. It also avoids mentioning certain key implementation issues: in particular, there is no requirement that multitasking be used for background jobs, subshells, etc. This was done to allow portability to non-multitasking systems like MS-DOS, so that, for example, the MKS Toolkit (see later in this appendix) can be POSIX compliant.
The Desk Top Korn Shell (dtksh) is a standard part of the Common Desktop Environment (CDE), available on commercial Unix systems such as Solaris, HP-UX, and AIX. It is based on a somewhat older version of ksh93. It evolved from the earlier program wksh, the Windowing Korn shell, released by Unix System Laboratories in late 1992. It’s a full Korn shell, compatible with the version that this book describes,[135] that has extensions for graphical user interface (GUI) programming in the X Window System environment. It is typically found in /usr/dt/bin/dtksh.
dtksh supports the OSF/Motif graphical Toolkit by making its routines available as built-in commands. This allows programmers to combine the Korn shell’s strength as a Unix systems programming environment with the power and abstraction of the Toolkit. The result is a unified environment for quick and easy development of graphics-based software.
There are various GUI development tools that allow you to construct user interfaces with a graphics-based editor rather than with programming language code. But such tools are typically huge, expensive, and complex. dtksh, on the other hand, is inexpensive and unbeatable for the smoothness of its integration with Unix — it’s the only such tool that you can use as your login shell! (Well, almost; see the next section.) It is a definite option for systems programmers who use X-based workstations and need a rapid prototyping tool.
To give you the flavor of dtksh code, here is a script that implements the canonical “Hello World” program by displaying a small box with an “OK” button. It is from the article Graphical Desktop Korn Shell, in the July, 1998 issue of Linux Journal, by George Kraft IV. (See http://www.linuxjournal.com/article.php?sid=2643.) This code should hold no surprises for X and Motif programmers:
#!/usr/dt/bin/dtksh
XtInitialize TOPLEVEL dtHello DtHello "$@"
XmCreateMessageDialog HELLO $TOPLEVEL hello \
dialogTitle:"DtHello" \
messageString:"$(print "Hello\nWorld")"
XmMessageBoxGetChild HELP $HELLO \
DIALOG_HELP_BUTTON
XtUnmanageChild $HELP
XmMessageBoxGetChild CANCEL $HELLO \
DIALOG_CANCEL_BUTTON
XtUnmanageChild $CANCEL
XtAddCallback $HELLO okCallback exit
XtManageChild $HELLO
XtMainLoop
http://www.cactus.org/~gk4/kraft/george/papers/dtksh/ is Mr. Kraft’s web presentation on dtksh.
The following book is devoted to dtksh:
Desktop KornShell Graphical Programming by
J. Stephen Pendergast, Jr., published by Addison-Wesley, 1995 (ISBN: 0-201-63375-2).
Examples from the book are online at
ftp://ftp.aw.com/aw.prof.comp.series/pendergrast.examples.tar.Z.
Also available is
ftp://ftp.aw.com/aw.prof.comp.series/pend.dtksh1,
a text file that
provides an overview of the book.
Unfortunately, as of this writing, this book is out of print.
Back in 1996, while a computer science graduate student at Princeton University, Dr. Jeffrey L. Korn[136] wrote tksh. This is an integration of ksh93 with Tcl/Tk. The following quote (from Dr. Korn’s research web page) summarizes it well:
Tksh is a graphical language (similar to Visual Basic or Tcl/Tk) that uses KornShell (ksh93) for scripting and Tk for graphical user interface. Tksh is implemented as a ksh93 extension, and allows Tcl libraries such as Tk to run on top of ksh93 unchanged. ksh93 is well suited for graphical scripting because it is backward compatible with sh, making it both easy to learn and easy to extend existing scripts to provide user interface. Tksh also allows Tcl scripts to run without modification, making it possible to mix and match components written in either Tcl or ksh93.
The tksh home page is still at Princeton: http://www.cs.princeton.edu/~jlk/tksh/. It has links to papers and documentation that can be downloaded and printed. However, the link to tksh executables is out-of-date. The tksh source is available from AT&T Research as part of the ast-open package, which also contains ksh93 and reimplementations of many other Unix tools. See Appendix C for more information.
The following example script, from the USENIX paper on tksh, is called watchdir:
# Tksh Demo
# Jeff Korn
#
# This script keeps track of visited directories and shows the files
# in the current directory. You can double-click on files and
# directories. The script should be used interactively, so to run:
# $ tksh
# $ . scripts/watchdir
function winsetup
{
pack $(frame .f)
frame .f.dirname -relief raised -bd 1
pack .f.dirname -side top -fill x
pack $(frame .f.ls) $(frame .f.dirs) -side left
label .f.dirname.label -text "Current directory: "
label .f.dirname.pwd -textvariable PWD
pack .f.dirname.label .f.dirname.pwd -side left
scrollbar .f.ls.scroll -command ".f.ls.list yview"
listbox .f.ls.list -yscroll ".f.ls.scroll set" -width 20 -setgrid 1
pack $(label .f.ls.label -text "Directory Contents") -side top
pack .f.ls.list .f.ls.scroll -side left -fill y -expand 1
scrollbar .f.dirs.scroll -command ".f.dirs.list yview"
listbox .f.dirs.list -yscroll ".f.dirs.scroll set" -width 20 -setgrid 1
pack $(label .f.dirs.label -text "Visited Directories") -side top
pack .f.dirs.list .f.dirs.scroll -side left -fill y -expand 1
bind .f.dirs.list "<Double-1>" 'cd $(selection get)'
bind .f.ls.list "<Double-1>" 'tkfileselect $(selection get)'
}
function tkfileselect
{
[[ -d "$1" ]] && tkcd "$1"
[[ -f "$1" ]] && ${EDITOR-${VISUAL-emacs}} "$1"
}
function tkcd
{
cd $1 > /dev/null || return
.f.ls.list delete 0 end
set -o markdirs
.f.ls.list insert end .. *
[[ ${VisitedDir["$PWD"]} == "" ]] && .f.dirs.list insert end "$PWD"
VisitedDir["$PWD"]=1
}
typeset -A VisitedDir
winsetup > /dev/null
alias cd=tkcd
tkcd .What’s nice about tksh, besides the interesting blend of complementary technologies, is that it brings Tk programming out to the shell level. Graphics programming with Tk is much higher level than with the Motif toolkit; thus the learning curve is easier to climb, and the scripts are easier to read and write.
Many of the Open Source Unix-like systems, such as GNU/Linux, come with the Public Domain Korn Shell, pdksh. pdksh is available as source code; start at its home page: http://www.cs.mun.ca/~michael/pdksh/. It comes with instructions for building and installing on various Unix platforms.
pdksh was originally written by Eric Gisin, who based it on Charles Forsyth’s public-domain clone of the Version 7 Bourne shell. It is mostly compatible with the 1988 Korn shell and POSIX, with some extensions of its own.
Its emacs editing mode is actually more powerful than that of the 1988 Korn shell. Like the full Emacs editor, you can customize the keystrokes that invoke editing commands (known as key bindings in Emacs terminology). Editing commands have full names that you can associate with keystrokes by using the bind command.
For example, if you want to set up CTRL-U to do the same thing as CTRL-P (i.e., go back to the previous command in the history file), you could put this command in your .profile:
bind '^U'=up-history
You can even set up two-character escape sequences, which (for example) allow you to use ANSI arrow keys as well as control characters, and you can define macros, i.e., abbreviations for sequences of editing commands.
The public domain Korn shell’s additional features include
alternation wildcards (borrowed from the C shell) and
user-definable tilde notation, in which you can set up ~
as an abbreviation for anything, not just user names.
There are also a few subtle differences in integer expression
evaluation and aliasing.
pdksh lacks the following features of the official version:
The built-in variable LINES.
The DEBUG fake signal. The fake signals ERR and EXIT within functions.
Functions inherit the trap settings of the main script.
The POSIX file expansion character classes ([[:alpha:]], etc.)
are not available.
The read command and select
loop do not use the command-line editing modes.
The last command of a pipeline is not run in the parent shell.
Thus, echo hi | read x; print $x doesn’t work
the same as in ksh. (Most Bourne-style shells
work this same way.)
The set -o option form of ksh is set -X option in pdksh.
Although it lacks most ksh93 extensions, pdksh is a worthwhile alternative to the C and Bourne shells.
bash is probably the most popular “third-party” shell that is
available for Unix and other systems.
In particular, it is the default shell on GNU/Linux systems.
(It also comes on the “freeware” CD with Solaris 8.)
You can get it from
the Internet, via anonymous FTP to
ftp.gnu.org
in the directory /pub/gnu/bash.
You can also order it from
its maker at the address listed here.
The Free Software Foundation
59 Temple Place - Suite 330
Boston, MA 02111-1307
(617) 542-2652
(617) 542-5942 (fax)
gnu@gnu.org
http://www.gnu.org
Bash was written by Brian Fox and Chet Ramey. Chet Ramey currently maintains it. Its name is in line with the FSF’s penchant for bad puns: it stands for Bourne-Again Shell. Although bash is easily available and you don’t have to pay for it (other than the cost of media, phone calls, etc.), it’s not really public domain software. While public domain software doesn’t have licensing restrictions, the FSF’s software does. But those restrictions are diametrically opposed to those in a commercial license:[137] instead of agreeing not to distribute the software further, you agree not to prevent it from being distributed further! In other words, you enjoy unrestricted use of the software as long as you agree not to inhibit others from doing the same. Richard Stallman, the founder of the FSF, invented this intriguing and admirable concept.
These days, the ideals of the Free Software and Open Source movements, the GNU project, and the quality of GNU software are all well known. The most popular GNU system is GNU/Linux, which uses the Linux kernel and GNU utilities to make a complete, fully functional, Unix- and POSIX-compatible computing environment.
bash is fully compatible with the 1992 POSIX standard. It has several of the most important Korn shell features and the C shell features that the Korn shell has appropriated, including aliases, functions, tilde notation, emacs and vi editing modes, arithmetic expressions, job control, etc.
The overlap of features between bash and the Korn shell has increased in recent years. It includes many ksh93 features. But it is not an exact ksh clone. The bash FAQ, published monthly by Chet Ramey, lists the following differences between bash and ksh93. Items enclosed in square brackets ([...]) are listed in this book, but not in the FAQ.
The following new things in ksh93 are not available in bash 2.05:
Associative arrays
Floating-point arithmetic and variables
Math library functions
${!name[sub]} name of subscript for associative array
“.” is allowed in variable names to create a hierarchical namespace
More extensive compound assignment syntax
Discipline functions
sleep and getconf built-ins (bash has loadable versions)
typeset -n and nameref variables
The KEYBD trap
The variables:
.sh.edchar,
.sh.edmode,
.sh.edcol,
.sh.edtext,
.sh.version,
.sh.name,
.sh.subscript,
.sh.value,
HISTEDIT
[The .sh.match variable also]
Backreferences in pattern matching (\N)
The & operator in pattern lists for matching
print -f (bash uses printf)
fc has been renamed to hist
The dot command (.) can execute shell functions
Exit statuses between 0 and 255
The set -o pipefail option
The += variable assignment operator
TMOUT is default timeout for read
and select
<&
n
-
and
>&
n
-
redirections (combination dup and
close) [Here-strings with <<<]
FPATH and PATH mixing
getopts -a
The -R invocation option
DEBUG trap now executed before each simple command, instead of after
The printf
%H,
%P,
%T
modifiers, and an output base for
%d
[Also %Z.]
The following new things in ksh93 are present in bash 2.05:
The
for ((...;...;...)) ; do list; done
arithmetic for command
The ?:, ++, --,
and comma (,) arithmetic operators
The shell variable expansions:
${!
param
},
${
param
:
offset[:
len]},
${
param
/
pat[/
str]},
${!
param
*}
Compound array assignment
The ! reserved word
Loadable built-ins — but ksh uses builtin while bash uses enable
The command, builtin, and disown built-ins
New $'...' and $"..." quoting
FIGNORE (but bash
uses GLOBIGNORE), HISTCMD
set -o notify, set -C
Changes to kill built-in
read -A (bash uses read -a)
read -t/read -d
trap -p
exec -a/exec -c
The dot command (.) restores the positional parameters when it completes
The test command conforms to POSIX.
umask -S
unalias -a
Command and arithmetic substitution performed on
PS1,
PS4,
and
ENV
Command name completion
ENV processed only for interactive shells
bash has many features of its own that make it a very powerful and flexible environment. Here are some of the highlights:
You can put backslash-escapes in the primary prompt string
(PS1) for which bash substitutes things like
the date, time, current working directory, machine name,
user name, shell, etc.
The commands builtin, command, and enable give you more control over the steps bash goes through to look for commands — i.e., bash’s analogue to the list of command search steps in Chapter 7.
The emacs editing mode is customizable, even more so than its equivalent in pdksh. You can use the bind command to set up your own keystroke preferences, and there are several more commands available — including the ability to undo your last command.
You can also rebind keystrokes in vi editing mode.
pushd and popd are built-in, as they are in the C shell.
Indexed arrays may be of unlimited size.
Many new options and variables let you customize your environment
with unprecedented flexibility.
This includes set -o posix for strict POSIX conformance.
We’re compelled to say that many users prefer bash to the Korn shell. With the increasing popularity of GNU/Linux and various BSD-derived systems, it’s not clear which shell has the larger user base. In any case, bash is definitely an excellent choice. We recommend the book Learning the bash Shell by Cameron Newham and Bill Rosenblatt, published by O’Reilly & Associates. (It is based on the first edition of this book.)
zsh is a powerful interactive shell and scripting language with many features found in ksh, bash, and tcsh, as well as several unique features. zsh has most of the features of ksh88 but few of ksh93. It is freely available and should compile and run on just about any modern version of Unix. Ports for other operating systems are also available. The zsh home page is http://www.zsh.org. The current version is 4.0.2.
In this section we cover:
Extended globbing
Completion
Command-line editing
Prompts and prompt themes
Differences between zsh and ksh
A very useful feature is the recursive glob[138]
operator, **.
For example,
a recursive grep is simple to construct:
grep foo **/*.c
Or to recursively find all files or directories named core, try:
print **/core
Another useful feature is glob qualifiers. There are many of these, for example, to print out only regular files in the current directory:
print *(.)
or just the directories:
print *(/)
Combining these with the recursive glob operator can be handy. We can improve the above example of finding core files by limiting the search to regular files only:
print **/core(.)
Another qualifier is U for file system objects you own. The following
prints all files you own in /tmp and below:
print /tmp/**/*(U)
The glob qualifiers can also be combined. For example, using the
socket file keyword =
in combination with U, it’s easy to find
socket files in /tmp and below that you own:
print /tmp/**/*(U=)
File size qualifiers are also available. For example, to find all files in your home directory that are greater than 10 megabytes in size:
print ~/**/*(Lm+10)
And file permission qualifiers are also available. For example, the W
qualifier selects world-writable file system objects. You can
use it to find all directories in your home directory and below that
are owned by you and that are world-writeable:
print ~/**/*(UW/)
See zshexpn(1) for more information.
The zsh completion system is extremely sophisticated. The main idea is that any time you are about to type something on the command line, if you hit TAB, zsh will try to complete it. zsh comes with many defaults for completion and is also fully customizable.
To get a full set of default completion features, run the following commands (normally in your ~/.zshrc startup file):
autoload -U compinit compinit
Now let’s look at some examples.
We represent the TAB key in examples as [TAB].
First, zsh is smart about doing completions.
For example, cd[TAB] only
expands directories, thus eliminating completion noise.
Have you ever been frustrated because you can’t think of exactly the name of the command you want to find more information on, and man -k [139] hasn’t been configured on your system? Well, zsh will complete available man pages for you:
g@sifl:pts/7% man zsh[TAB]
zsh zshcompctl zshcontrib zshmodules zshzftpsys
zshall zshcompsys zshexpn zshoptions zshzle
zshbuiltins zshcompwid zshmisc zshparamOr maybe you want to find out a process name or PID you want to kill:
g@sifl:pts/2% kill [TAB]
9652 pts/2 00:00:00 zsh
9653 pts/2 00:00:00 psFor finger, it expands users:
g@sifl:pts/7% finger o[TAB]
odin omg oof operator orband hosts:
g@sifl:pts/7% finger oof@[TAB]
brak localhost sifl zorakUsing the distributed compdef function, you can define your own completions, using either your own custom functions or the completion functions that come with zsh. For example, the distribution defines the kill command to use the _pids distribution function to provide process identifiers. You can also use it to define completion for other commands, such as the Solaris pstack command:
compdef _pids pstack
Once this is done, you can apply completion to the pstack command like so:
g@sifl:pts/7% pstack [TAB]
13606 pts/7 00:00:00 zsh
13607 pts/7 00:00:00 ps
Another very useful distribution completion function is
_gnu_generic. This can be applied to any command
that uses the GNU --long-option
command-line option conventions. The zsh
distribution specifies many GNU commands to complete with this function
(such as tar):
g@sifl:pts/7% tar --v[TAB]
--verbose --verify --version --volno-fileAnd this is just the tip of the proverbial iceberg. There is much more to the zsh completion system; see zshcompsys(1) for the (gory) details.
The zsh command-line editor is extremely powerful. It has several unique features, including multiline editing and an input buffer stack. The multiline command editor makes composing small scripts on the command line much easier then just having one line to edit with.
The input buffer stack comes in very handy. While you are typing a command, you can type ESC q, and the current line is pushed onto the buffer stack. The input line is then cleared, and you can type another command. When that has been executed, the previous line is popped off the stack and you can continue with that command. See zshzle(1) for more details.
While most modern shells have customizable prompts, zsh raises it
to an art form. One of the unique features is a right side
prompt, RPROMPT, which is very useful for
holding the current directory. This in turn removes
clutter from the left hand prompt:
g@sifl:pts/2% RPROMPT='%~'
g@sifl:pts/2% ~/src/xemacs-21.1.14Also, you can define colors and bold fonts, and the prompt can be more than one line.
And as the notion of themes [140] has become popular in GUIs such as GNOME, zsh prompt themes can be defined; the distribution ships with several to choose from. To enable prompt themes, add these lines to your ~/.zshrc:
autoload -U promptinit promptinit
To see what themes are available, run:
g@sifl:pts/2% prompt -l ~
Currently available prompt themes:
adam1 adam2 bart bigfade clint elite2 elite fade fire off oliver
redhat suse zeframTo enable a theme, use the -s option. For example:
g@sifl:pts/7% prompt -s bart ~
sifl [prompt -s bart] ~ 01-10-04 11:58PM
g@sifl:pts/7% ~You can see that bart is a two-line prompt with several components such as the host name, the previous command, the current directory, and the date and time. See zshcontrib(1) for more details on prompt themes.
This section is derived from information in the zsh FAQ.
Most features of ksh88 (and hence also of the Bourne shell, sh) are implemented in zsh; problems can arise because the implementation is slightly different. Note also that not all ksh’s are the same either. This is based on the 11/16/88f version of ksh; differences from ksh93 are more substantial.
As a summary of the status:
Because of all the options, it is not safe to assume a general zsh run by a user will behave as if sh or ksh compatible.
Invoking zsh as sh
or ksh (or if either is a symbolic link to
zsh) sets appropriate options and improves
compatibility (from within zsh itself, calling
ARGV0=sh zsh will also work).
From Version 3.0 onward, the degree of compatibility with sh under these circumstances is very high: zsh can now be used with GNU configure or perl’s Configure, for example.
The degree of compatibility with ksh is also high, but a few things are missing: for example, the more sophisticated pattern-matching expressions are different for versions before 3.1.3 — see the detailed list below.
Also from 3.0, the command emulate is
available: emulate ksh and emulate sh set various options as well as changing the effect
of single-letter option flags, as if the shell had been invoked
with the appropriate name. Including the command emulate sh; setopt localoptions in a shell function turns on
sh emulation for that function only. In 4.0 (and
in 3.0.6 through 8), this can be abbreviated as emulate -L sh.
The classic difference is word splitting:
zsh keeps the result of plain $variable
as one word, even if the variable contains white space.
This trips up many beginning zsh users.
The answer is to set SH_WORD_SPLIT
for backward compatibility. The next most classic difference is that
unmatched glob patterns cause the command to abort; set NO_NOMATCH
for those.
zsh has a large set of options which increase ksh compatibility, though maybe decreasing zsh’s abilities: see the manual entries for the details. If invoked as ksh, the shell sets suitable options.
Here are some differences from ksh which might
prove significant for ksh programmers, some of
which may be interpreted as bugs. Note that this
list is deliberately rather full and that most of the items are fairly
minor. Those marked with † perform in a ksh-like
manner if the shell is invoked with the name ksh
or if emulate ksh is in effect. Capitalized words
with underscores in their names refer to shell options.
† Shell word splitting: see above.
† Arrays are (by default) more csh-like
than ksh-like: subscripts start at 1, not 0;
array[0] refers to array[1];
$array refers to the whole array, not
$array[0]; braces are unnecessary: $a[1]
is the same as ${a[1]}, etc. Set the
KSH_ARRAYS option for compatibility.
Coprocesses are established by coproc;
|& behaves like csh.[141]
Handling of coprocess file descriptors is also different.
For cmd1
&&
cmd2
&,
only cmd2, instead of the whole expression, is run in
the background in zsh. The manual implies that this is a bug.
Use {
cmd1
&&
cmd2
} & as a workaround.
† Failure to match a globbing pattern causes
an error (use NO_NOMATCH).
† The results of parameter substitutions are treated as plain text:
foo="*"; print $foo prints all files
in ksh but prints * in zsh
(use GLOB_SUBST).
† The prompt variables (e.g., PS1)
do not undergo parameter substitution by default (use PROMPT_SUBST).
† Standard globbing does not allow ksh-style pattern-lists. Table A-1 shows equivalent patterns.
The ^, ~ and
# (but not |) forms
require setting EXTENDED_GLOB. From version 3.1.3, the
ksh forms are fully supported when the option
KSH_GLOB is in effect; for previous versions you
must use the equivalents given in Table A-1.
Unquoted assignments do file expansion after colons (intended
for PATH-style variables).
integer does not allow -i.
typeset and integer have special behavior for assignments in ksh, but not in zsh. For example, this doesn’t work in zsh:
integer k=$(wc -l ~/.zshrc)
because the return value from wc includes leading whitespace, which causes wordsplitting. ksh handles the assignment specially, as a single word.
| ksh | zsh | Meaning |
!(foo)
|
^foo
|
Anything but |
foo1~foo2
|
Anything matching | |
@(foo1|foo2|...)
|
(foo1|foo2|...)
|
One of |
?(foo)
|
(foo|)
|
Zero or one occurrences of |
*(foo)
|
(foo)#
|
Zero or more occurrences of |
+(foo)
|
(foo)##
|
One or more occurrences of |
[a]
Note that | ||
† There is no ENV variable
(use /etc/zshrc, ~/.zshrc;
note also $ZDOTDIR).
$PATH is not searched for
commands specified at invocation without -c.
The order in which aliases and functions are defined is significant:
function definitions with () expand aliases.
Aliases and functions cannot be exported.
There are no tracked aliases: command hashing replaces these.
The use of aliases for key bindings is replaced by bindkey.
† Options are not local to functions
(use LOCAL_OPTIONS; note this
may always be unset locally to propagate options settings from a function to the calling level).
Functions defined with
function
funcname
{
body
;}
behave the same
way as those defined with
funcname
() {
body
;}.
In ksh93, only the former
behave as true functions,
and the latter
behave as if the body were read from a file with the dot command.
† Traps are not local to functions.
The option LOCAL_TRAPS is available from 3.1.6.
TRAPERR has become TRAPZERR (this was forced by UNICOS which has SIGERR).
The options
emacs,
gmacs,
and
viraw
are not supported. Use bindkey to change
the editing behavior:
set -o emacs becomes bindkey -e
and
set -o vi becomes bindkey -v;
for
gmacs, go to emacs-mode
and use bindkey \^t gosmacs-transpose-characters.
The keyword option does not exist and set -k is instead interactivecomments.
† Management of histories in multiple shells is different: the
history list is not saved and restored after each command. The option
SHARE_HISTORY appeared in 3.1.6 and is set in
ksh compatibility mode to remedy this.
\ does not escape editing chars (use CTRL-V).
Not all ksh bindings are set (e.g. ESC #; try ESC q).
† # in an interactive shell is not treated as a comment by default.
Some built-ins (r, autoload, history, integer, ...) are aliases in ksh.
There is no built-in command newgrp:
use alias newgrp="exec newgrp".
jobs has no -n flag.
read has no -s flag.
select always redisplays the list of selections on each loop.
The proliferation of the Korn shell has not stopped at the boundaries of Unix-dom. Many programmers who got their initial experience on Unix systems and subsequently crossed over into the PC world wished for a nice Unix-like environment (especially when faced with the horrors of the MS-DOS command line!), so it’s not surprising that several Unix shell-style interfaces to small-computer operating systems have appeared, Korn shell emulations among them.
In the past several years, not just shell clones have appeared, but entire Unix “environments.” Two of them use shells that we’ve already discussed. Two others provide their own shell reimplementations. Providing lists of major and minor differences is counterproductive. Instead, this section describes each environment in turn (in alphabetical order), along with contact and Internet download information.
Cygnus Consulting (now Red Hat), created the cygwin environment. First creating cgywin.dll, a shared library that provides Unix system call emulation, they ported a large number of GNU utilities to various versions of Microsoft Windows. The emulation includes TCP/IP networking with the Berkeley socket API. The greatest functionality comes under Windows/NT, Windows 2000, and Windows XP, although the environment can and does work under Windows 95/98/ME, as well.
The cygwin environment uses bash
for its shell, GCC for its C compiler, and the rest of the GNU utilities
for its Unix toolset. A sophisticated mount command
provides a mapping of the Windows C:\path notation
to Unix filenames.
The starting point for the cygwin project is http://www.cygwin.com. The first thing to download is an installer program. Upon running it, you choose what additional packages you wish to install. Installation is entirely Internet-based; there are no official cygwin CD’s, at least not from the project maintainers.
The DJGPP suite provides 32-bit GNU tools for the MS-DOS environment. To quote the web page:
DJGPP is a complete 32-bit C/C++ development system for Intel 80386 (and higher) PCs running MS-DOS. It includes ports of many GNU development utilities. The development tools require a 80386 or newer computer to run, as do the programs they produce. In most cases, the programs it produces can be sold commercially without license or royalties.
The name comes from the initials of D.J. Delorie, who ported the GNU C++ compiler, g++ to MS-DOS, and the text initials of g++, GPP. It grew into essentially a full Unix environment on top of MS-DOS, with all the GNU tools and bash as its shell. Unlike cygwin or UWIN (see later in this chapter), you don’t need a version of Windows, just a full 32-bit processor and MS-DOS. (Although, of course, you can use DJGPP from within a Windows MS-DOS window.) The web site is http://www.delorie.com/djgpp/.
Perhaps the most established Unix environment for the PC world is the MKS Toolkit from Mortice Kern Systems:
| MKS Canada - Corporate Headquarters |
| 410 Albert Street |
| Waterloo, ON N2L 3V3 |
| Canada |
| (519) 884-2251 |
| (519) 884-8861 (fax) |
| (800) 265-2797 (sales) |
| http://www.mks.com |
The MKS Toolkit comes in various versions depending upon the development environment and the number of developers who will be using it. It includes a shell that is POSIX compliant, along with just about all the features of the 1988 Korn shell, as well as over 300 utilities, such as awk, perl, vi, make, and so on. Their library supports over 1500 Unix APIs, making it extremely complete and easing porting to the Windows environment. More information is available at http://www.mkssoftware.com/products/tk/ds_tkpdev.asp.
Thompson Automation Software provides the Thompson Toolkit, which includes a shell and over 100 utilities. The toolkit is available MS-DOS 2.1 and higher, OS/2 1.2 or WARP, and for Microsoft Windows 95 and higher. The contact information is:
Thompson Automation Software
5616 SW Jefferson
Portland, OR 97221
1-800-944-0139 (U.S. and Canada)
1-503-224-1639 (international/local)
1-503-224-3230 (FAX)
sales@tasoft.com
http://www.tasoft.com/toolkit.html/
Thompson software is best known for their implementation of awk, which is both fast and reliable, with many powerful extensions to the awk language. The toolkit shell is compatible with POSIX and the 1988 version of the Korn shell.
The UWIN package is a project by David Korn and his colleagues to make a Unix environment available under Microsoft Windows. It is similar in structure to cygwin, discussed earlier. A shared library, posix.dll, provides emulation of the Unix system call APIs. The system call emulation is quite complete. An interesting twist is that the Windows registry can be accessed as a filesystem under /reg. On top of the Unix API emulation, ksh93 and over 200 Unix utilities (or rather, reimplementations) have been compiled and run. The UWIN environment relies on the native Microsoft Visual C/C++ compiler, although the GNU development tools are available for download and use with UWIN.
http://www.research.att.com/sw/tools/uwin/ is the web page for the project. It describes what is available, with links for downloading binaries, as well as information on commercial licensing of the UWIN package. Also included are links to various papers on UWIN, additional useful software, and links to other, similar packages.
The most notable advantage to the UWIN package is that its shell is the authentic ksh93. Thus, compatibility with the Unix version of ksh93 isn’t an issue.
[131] There are also a few differences in how the two shells react to certain extremely pathological input. Usually, the Korn shell processes correctly what causes the Bourne shell to “choke.”
[132]
In the original Version 7 Bourne Shell
(and in Berkeley Unix systems through 4.3BSD),
test and
[
condition
]
were actually external commands. (They were hard links to each
other in /bin.) However, they’ve been built into
the Bourne shell in all systems since System III (circa 1981).
[133]
The <> operator was in the original Version 7 Bourne shell,
but not documented, since it didn’t always work correctly across all Unix systems.
Its availability should not be relied upon for Bourne shell programming.
[134] But note that it’s not a special built-in! Design by committee shows through here.
[135] Features listed throughout the book as being introduced for “recent” versions won’t be in dtksh.
[136] Yes, David Korn’s son. He now works in the same research center as his father at AT&T Laboratories, although in a different area.
[137] Accordingly, the document that spells out these restrictions is called a copyleft.
[138] Globbing is technical slang for wildcard expansion.
[139] This does a keyword search of an online database extracted from the man pages.
[140] Some popular GUIs, such as GNOME, support themes. Rather than having one immutable look and feel, they can be changed to different styles, or themes. The distributions of these GUIs often contain several to choose from. Some of these tend to emulate other GUI’s, while others are new and are mostly fun window dressing.
[141]
In csh, |& sends both
standard output and standard error down the same pipeline; it is
equivalent to ... 2>&1 | ....