The sed stream editor

The sed program is an interpreter for sed, the stream editor, a simple data-driven programming language that accepts data on input (normally strings), and acts on and transforms it conditionally. The programs end up being very short, as the commands are all only one letter long. This terse approach means they fit very well on a command line and are quick to type.

We'll use the following example file, named manual, for our sed demonstration:

$ cat manual
The sed utility is a stream editor that shall read one or more text files,
make editing changes according to a script of editing commands,
and write the results to standard output.

With an empty program, sed simply reproduces all of the output without changing it; this is because by default it prints all the lines it filters implicitly, unless they are deleted:

$ sed '' manual
The sed utility is a stream editor that shall read one or more text files,
make editing changes according to a script of editing commands,
and write the results to standard output.
Note the empty string is specified with ''; if we don't use the quotes to show an empty argument, sed will only be passed manual as an argument, and will think it's a script!

We can delete individual lines with the d command, by prefixing them with the line number:

$ sed '1d' manual
make editing changes according to a script of editing commands,
and write the results to standard output.

We can delete a range of lines with a comma-separated pair of numbers:

$ sed '1,2d' manual
and write the results to standard output.

Lines can be selected not just with line numbers, but also with pattern matches. For example, to delete all lines from "1" up to and including the next occurrence of the word "commands," we could write the following:

$ sed '1,/commands/d' manual
and write the results to standard output.

We can transform lines with the s command, which substitutes a regular expression pattern into a given replacement:

$ sed 's/stream/river/' manual
The sed utility is a river editor that shall read one or more text files,
...

The s/pattern/replacement/ pattern-replacement syntax is the function of sed that is best known to programmers even outside of shell scripting; it shows up very often in older shell scripts, and in various other contexts in Unix, such as in the vi editor.

These are just a few examples of the text transformations possible with sed; you should check your operating system's manual page for more details.

While powerful, the terseness of these examples shows sed programs can be a little hard to write at first. Fortunately, many of the most common filtering functions for which programmers often use sed can be done in Bash itself, using parameter expansion for common string operations such as removing extensions or leading paths from paths. We will discuss that in Chapter 5, Variables and Patterns .

Don't be tempted to use sed for tasks such as parsing programming languages or markup languages such as HTML; it's just not powerful enough to do it correctly! Use a proper parser such as HTML's tidy instead, and keep sed programs for simple and well-defined line-based text translations. For XML, try xpath; for JSON, try jq.