Unix Power Tools by Peek, Jerry -- Read -- Imperial Library of Trantor

Index

UNIX Power Tools, 3rd Edition

SPECIAL OFFER: Upgrade this ebook with O’Reilly A Note Regarding Supplemental Files How to Use This Book Preface

A Book for Browsing Like an Almanac Like a News Magazine Like a Hypertext Database Programs on the Web About Unix Versions Cross-References What's New in the Third Edition Typefaces and Other Conventions The Authors The Fine Print Request for Comments Acknowledgments for the First Edition Acknowledgments for the Second Edition Acknowledgments for the Third Edition

I. Basic Unix Environment

1. Introduction

1.1. What's Special About Unix? 1.2. Power Grows on You 1.3. The Core of Unix 1.4. Communication with Unix 1.5. Programs Are Designed to Work Together 1.6. There Are Many Shells 1.7. Which Shell Am I Running? 1.8. Anyone Can Program the Shell 1.9. Internal and External Commands 1.10. The Kernel and Daemons 1.11. Filenames 1.12. Filename Extensions 1.13. Wildcards 1.14. The Tree Structure of the Filesystem 1.15. Your Home Directory 1.16. Making Pathnames 1.17. File Access Permissions 1.18. The Superuser (Root) 1.19. When Is a File Not a File? 1.20. Scripting 1.21. Unix Networking and Communications 1.22. The X Window System

2. Getting Help

2.1. The man Command 2.2. whatis: One-Line Command Summaries 2.3. whereis: Finding Where a Command Is Located 2.4. Searching Online Manual Pages 2.5. How Unix Systems Remember Their Names 2.6. Which Version Am I Using? 2.7. What tty Am I On? 2.8. Who's On? 2.9. The info Command

II. Customizing Your Environment

3. Setting Up Your Unix Shell

3.1. What Happens When You Log In 3.2. The Mac OS X Terminal Application 3.3. Shell Setup Files — Which, Where, and Why 3.4. Login Shells, Interactive Shells

3.4.1. Login Shells 3.4.2. Interactive Shells

3.5. What Goes in Shell Setup Files? 3.6. Tip for Changing Account Setup: Keep a Shell Ready 3.7. Use Absolute Pathnames in Shell Setup Files 3.8. Setup Files Aren't Read When You Want? 3.9. Gotchas in set prompt Test 3.10. Automatic Setups for Different Terminals 3.11. Terminal Setup: Testing TERM 3.12. Terminal Setup: Testing Remote Hostname and X Display 3.13. Terminal Setup: Testing Port 3.14. Terminal Setup: Testing Environment Variables 3.15. Terminal Setup: Searching Terminal Table 3.16. Terminal Setup: Testing Window Size 3.17. Terminal Setup: Setting and Testing Window Name 3.18. A .cshrc.$HOST File for Per Host Setup 3.19. Making a "Login" Shell 3.20. RC Files 3.21. Make Your Own Manpages Without Learning troff 3.22. Writing a Simple Manpage with the -man Macros

4. Interacting with Your Environment

4.1. Basics of Setting the Prompt 4.2. Static Prompts 4.3. Dynamic Prompts 4.4. Simulating Dynamic Prompts 4.5. C-Shell Prompt Causes Problems in vi, rsh, etc. 4.6. Faster Prompt Setting with Built-ins 4.7. Multiline Shell Prompts 4.8. Session Info in Window Title or Status Line 4.9. A "Menu Prompt" for Naive Users 4.10. Highlighting and Color in Shell Prompts 4.11. Right-Side Prompts 4.12. Show Subshell Level with $SHLVL 4.13. What Good Is a Blank Shell Prompt? 4.14. dirs in Your Prompt: Better Than $cwd 4.15. External Commands Send Signals to Set Variables 4.16. Preprompt, Pre-execution, and Periodic Commands 4.17. Running Commands When You Log Out 4.18. Running Commands at Bourne/Korn Shell Logout 4.19. Stop Accidental Bourne-Shell Logouts

5. Getting the Most out of Terminals, xterm, and X Windows

5.1. There's a Lot to Know About Terminals 5.2. The Idea of a Terminal Database 5.3. Setting the Terminal Type When You Log In 5.4. Querying Your Terminal Type: qterm 5.5. Querying Your xterm Size: resize 5.6. Checklist: Terminal Hangs When I Log In

5.6.1. Output Stopped? 5.6.2. Job Stopped? 5.6.3. Program Waiting for Input? 5.6.4. Stalled Data Connection? 5.6.5. Aborting Programs

5.7. Find Out Terminal Settings with stty 5.8. Setting Your Erase, Kill, and Interrupt Characters 5.9. Working with xterm and Friends 5.10. Login xterms and rxvts 5.11. Working with Scrollbars 5.12. How Many Lines to Save? 5.13. Simple Copy and Paste in xterm 5.14. Defining What Makes Up a Word for Selection Purposes 5.15. Setting the Titlebar and Icon Text 5.16. The Simple Way to Pick a Font 5.17. The xterm Menus 5.18. Changing Fonts Dynamically

5.18.1. VT Fonts Menu 5.18.2. Enabling Escape Sequence and Selection

5.19. Working with xclipboard 5.20. Problems with Large Selections 5.21. Tips for Copy and Paste Between Windows 5.22. Running a Single Command with xterm -e 5.23. Don't Quote Arguments to xterm -e

6. Your X Environment

6.1. Defining Keys and Button Presses with xmodmap 6.2. Using xev to Learn Keysym Mappings 6.3. X Resource Syntax 6.4. X Event Translations 6.5. Setting X Resources: Overview 6.6. Setting Resources with the -xrm Option 6.7. How -name Affects Resources 6.8. Setting Resources with xrdb 6.9. Listing the Current Resources for a Client: appres 6.10. Starting Remote X Clients

6.10.1. Starting Remote X Clients from Interactive Logins 6.10.2. Starting a Remote Client with rsh and ssh

III. Working with Files and Directories

7. Directory Organization

7.1. What? Me, Organized? 7.2. Many Homes 7.3. Access to Directories 7.4. A bin Directory for Your Programs and Scripts 7.5. Private (Personal) Directories 7.6. Naming Files 7.7. Make More Directories! 7.8. Making Directories Made Easier

8. Directories and Files

8.1. Everything but the find Command 8.2. The Three Unix File Times 8.3. Finding Oldest or Newest Files with ls -t and ls -u 8.4. List All Subdirectories with ls -R 8.5. The ls -d Option 8.6. Color ls

8.6.1. Trying It 8.6.2. Configuring It 8.6.3. The -- color Option 8.6.4. Another color ls

8.7. Some GNU ls Features 8.8. A csh Alias to List Recently Changed Files 8.9. Showing Hidden Files with ls -A and -a 8.10. Useful ls Aliases 8.11. Can't Access a File? Look for Spaces in the Name 8.12. Showing Nonprintable Characters in Filenames 8.13. Counting Files by Types 8.14. Listing Files by Age and Size 8.15. newer: Print the Name of the Newest File 8.16. oldlinks: Find Unconnected Symbolic Links 8.17. Picking a Unique Filename Automatically

9. Finding Files with find

9.1. How to Use find 9.2. Delving Through a Deep Directory Tree 9.3. Don't Forget -print 9.4. Looking for Files with Particular Names 9.5. Searching for Old Files 9.6. Be an Expert on find Search Operators 9.7. The Times That find Finds 9.8. Exact File-Time Comparisons 9.9. Running Commands on What You Find 9.10. Using -exec to Create Custom Tests 9.11. Custom -exec Tests Applied 9.12. Finding Many Things with One Command 9.13. Searching for Files by Type 9.14. Searching for Files by Size 9.15. Searching for Files by Permission 9.16. Searching by Owner and Group 9.17. Duplicating a Directory Tree 9.18. Using "Fast find" Databases 9.19. Wildcards with "Fast find" Database 9.20. Finding Files (Much) Faster with a find Database 9.21. grepping a Directory Tree 9.22. lookfor: Which File Has That Word? 9.23. Using Shell Arrays to Browse Directories

9.23.1. Using the Stored Lists 9.23.2. Expanding Ranges

9.24. Finding the (Hard) Links to a File 9.25. Finding Files with -prune 9.26. Quick finds in the Current Directory 9.27. Skipping Parts of a Tree in find 9.28. Keeping find from Searching Networked Filesystem

10. Linking, Renaming, and Copying Files

10.1. What's So Complicated About Copying Files 10.2. What's Really in a Directory? 10.3. Files with Two or More Names 10.4. More About Links

10.4.1. Differences Between Hard and Symbolic Links 10.4.2. Links to a Directory

10.5. Creating and Removing Links 10.6. Stale Symbolic Links 10.7. Linking Directories 10.8. Showing the Actual Filenames for Symbolic Links 10.9. Renaming, Copying, or Comparing a Set of Files 10.10. Renaming a List of Files Interactively 10.11. One More Way to Do It 10.12. Copying Directory Trees with cp -r 10.13. Copying Directory Trees with tar and Pipes

11. Comparing Files

11.1. Checking Differences with diff 11.2. Comparing Three Different Versions with diff3 11.3. Context diffs 11.4. Side-by-Side diffs: sdiff 11.5. Choosing Sides with sdiff 11.6. Problems with diff and Tabstops 11.7. cmp and diff 11.8. Comparing Two Files with comm 11.9. More Friendly comm Output 11.10. make Isn't Just for Programmers! 11.11. Even More Uses for make

12. Showing What's in a File

12.1. Cracking the Nut 12.2. What Good Is a cat? 12.3. "less" is More 12.4. Show Nonprinting Characters with cat -v or od -c 12.5. What's in That Whitespace? 12.6. Finding File Types 12.7. Squash Extra Blank Lines 12.8. How to Look at the End of a File: tail 12.9. Finer Control on tail 12.10. How to Look at Files as They Grow 12.11. GNU tail File Following 12.12. Printing the Top of a File 12.13. Numbering Lines

13. Searching Through Files

13.1. Different Versions of grep 13.2. Searching for Text with grep 13.3. Finding Text That Doesn't Match 13.4. Extended Searching for Text with egrep 13.5. grepping for a List of Patterns 13.6. Approximate grep: agrep 13.7. Search RCS Files with rcsgrep

13.7.1. rcsgrep, rcsegrep, rcsfgrep 13.7.2. rcsegrep.fast

13.8. GNU Context greps 13.9. A Multiline Context grep Using sed 13.10. Compound Searches 13.11. Narrowing a Search Quickly 13.12. Faking Case-Insensitive Searches 13.13. Finding a Character in a Column 13.14. Fast Searches and Spelling Checks with "look" 13.15. Finding Words Inside Binary Files 13.16. A Highlighting grep

14. Removing Files

14.1. The Cycle of Creation and Destruction 14.2. How Unix Keeps Track of Files: Inodes 14.3. rm and Its Dangers 14.4. Tricks for Making rm Safer 14.5. Answer "Yes" or "No" Forever with yes 14.6. Remove Some, Leave Some 14.7. A Faster Way to Remove Files Interactively 14.8. Safer File Deletion in Some Directories 14.9. Safe Delete: Pros and Cons 14.10. Deletion with Prejudice: rm -f 14.11. Deleting Files with Odd Names 14.12. Using Wildcards to Delete Files with Strange Names 14.13. Handling a Filename Starting with a Dash (-) 14.14. Using unlink to Remove a File with a Strange Name 14.15. Removing a Strange File by its i-number 14.16. Problems Deleting Directories 14.17. Deleting Stale Files 14.18. Removing Every File but One 14.19. Using find to Clear Out Unneeded Files

15. Optimizing Disk Space

15.1. Disk Space Is Cheap 15.2. Instead of Removing a File, Empty It 15.3. Save Space with "Bit Bucket" Log Files and Mailboxes 15.4. Save Space with a Link 15.5. Limiting File Sizes

15.5.1. limit and ulimit 15.5.2. Other Ideas

15.6. Compressing Files to Save Space 15.7. Save Space: tar and compress a Directory Tree 15.8. How Much Disk Space? 15.9. Compressing a Directory Tree: Fine-Tuning 15.10. Save Space in Executable Files with strip 15.11. Disk Quotas

IV. Basic Editing

16. Spell Checking, Word Counting, and Textual Analysis

16.1. The Unix spell Command 16.2. Check Spelling Interactively with ispell 16.3. How Do I Spell That Word? 16.4. Inside spell 16.5. Adding Words to ispell's Dictionary 16.6. Counting Lines, Words, and Characters: wc 16.7. Find a a Doubled Word 16.8. Looking for Closure 16.9. Just the Words, Please

17. vi Tips and Tricks

17.1. The vi Editor: Why So Much Material? 17.2. What We Cover 17.3. Editing Multiple Files with vi 17.4. Edits Between Files 17.5. Local Settings for vi 17.6. Using Buffers to Move or Copy Text 17.7. Get Back What You Deleted with Numbered Buffers 17.8. Using Search Patterns and Global Commands

17.8.1. Global Searches

17.9. Confirming Substitutions in vi 17.10. Keep Your Original File, Write to a New File 17.11. Saving Part of a File 17.12. Appending to an Existing File 17.13. Moving Blocks of Text by Patterns 17.14. Useful Global Commands (with Pattern Matches) 17.15. Counting Occurrences; Stopping Search Wraps 17.16. Capitalizing Every Word on a Line 17.17. Per-File Setups in Separate Files 17.18. Filtering Text Through a Unix Command 17.19. vi File Recovery Versus Networked Filesystems 17.20. Be Careful with vi -r Recovered Buffers 17.21. Shell Escapes: Running One UnixCommand While Using Another 17.22. vi Compound Searches 17.23. vi Word Abbreviation 17.24. Using vi Abbreviations as Commands (Cut and Paste Between vi's) 17.25. Fixing Typos with vi Abbreviations 17.26. vi Line Commands Versus Character Commands 17.27. Out of Temporary Space? Use Another Directory 17.28. Neatening Lines 17.29. Finding Your Place with Undo 17.30. Setting Up vi with the .exrc File

18. Creating Custom Commands in vi

18.1. Why Type More Than You Have To? 18.2. Save Time and Typing with the vi map Commands

18.2.1. Command Mode Maps 18.2.2. Text-Input Mode Maps

18.3. What You Lose When You Use map! 18.4. vi @-Functions

18.4.1. Defining and Using Simple @-Functions 18.4.2. Combining @-Functions 18.4.3. Reusing a Definition 18.4.4. Newlines in an @-Function

18.5. Keymaps for Pasting into a Window Running vi 18.6. Protecting Keys from Interpretation by ex 18.7. Maps for Repeated Edits 18.8. More Examples of Mapping Keys in vi 18.9. Repeating a vi Keymap 18.10. Typing in Uppercase Without CAPS LOCK 18.11. Text-Input Mode Cursor Motion with No Arrow Keys 18.12. Don't Lose Important Functions with vi Maps: Use noremap 18.13. vi Macro for Splitting Long Lines 18.14. File-Backup Macros

19. GNU Emacs

19.1. Emacs: The Other Editor 19.2. Emacs Features: A Laundry List 19.3. Customizations and How to Avoid Them 19.4. Backup and Auto-Save Files 19.5. Putting Emacs in Overwrite Mode 19.6. Command Completion 19.7. Mike's Favorite Timesavers 19.8. Rational Searches 19.9. Unset PWD Before Using Emacs 19.10. Inserting Binary Characters into Files 19.11. Using Word-Abbreviation Mode

19.11.1. Trying Word Abbreviations for One Session 19.11.2. Making Word Abbreviations Part of Your Startup

19.12. Directories for Emacs Hacks 19.13. An Absurd Amusement

20. Batch Editing

20.1. Why Line Editors Aren't Dinosaurs 20.2. Writing Editing Scripts 20.3. Line Addressing 20.4. Useful ex Commands 20.5. Running Editing Scripts Within vi 20.6. Change Many Files by Editing Just One 20.7. ed/ex Batch Edits: A Typical Example 20.8. Batch Editing Gotcha: Editors Fail on Big Files 20.9. patch: Generalized Updating of Files That Differ 20.10. Quick Reference: awk

20.10.1. Command-Line Syntax 20.10.2. Patterns and Procedures

20.10.2.1. Patterns 20.10.2.2. Procedures 20.10.2.3. Simple pattern-procedure examples

20.10.3. awk System Variables 20.10.4. Operators 20.10.5. Variables and Array Assignments 20.10.6. Group Listing of awk Commands 20.10.7. Alphabetical Summary of Commands

20.11. Versions of awk

21. You Can't Quite Call This Editing

21.1. And Why Not? 21.2. Neatening Text with fmt 21.3. Alternatives to fmt 21.4. Clean Up Program Comment Blocks

21.4.1. The recomment Script 21.4.2. fmt -p

21.5. Remove Mail/News Headers with behead 21.6. Low-Level File Butchery with dd 21.7. offset: Indent Text 21.8. Centering Lines in a File 21.9. Splitting Files at Fixed Points: split 21.10. Splitting Files by Context: csplit 21.11. Hacking on Characters with tr 21.12. Encoding "Binary" Files into ASCII

21.12.1. uuencoding 21.12.2. MIME Encoding

21.13. Text Conversion with dd 21.14. Cutting Columns or Fields 21.15. Making Text in Columns with pr

21.15.1. One File per Column: -m 21.15.2. One File, Several Columns: -number 21.15.3. Order Lines Across Columns: -l

21.16. Make Columns Automatically with column 21.17. Straightening Jagged Columns 21.18. Pasting Things in Columns 21.19. Joining Lines with join 21.20. What Is (or Isn't) Unique? 21.21. Rotating Text

22. Sorting

22.1. Putting Things in Order 22.2. Sort Fields: How sort Sorts 22.3. Changing the sort Field Delimiter 22.4. Confusion with Whitespace Field Delimiters 22.5. Alphabetic and Numeric Sorting 22.6. Miscellaneous sort Hints

22.6.1. Dealing with Repeated Lines 22.6.2. Ignoring Blanks 22.6.3. Case-Insensitive Sorts 22.6.4. Dictionary Order 22.6.5. Month Order 22.6.6. Reverse Sort

22.7. lensort: Sort Lines by Length 22.8. Sorting a List of People by Last Name

V. Processes and the Kernel

23. Job Control

23.1. Job Control in a Nutshell 23.2. Job Control Basics

23.2.1. How Job Control Works 23.2.2. Using Job Control from Your Shell

23.3. Using jobs Effectively 23.4. Some Gotchas with Job Control 23.5. The "Current Job" Isn't Always What You Expect 23.6. Job Control and autowrite: Real Timesavers! 23.7. System Overloaded? Try Stopping Some Jobs 23.8. Notification When Jobs Change State 23.9. Stop Background Output with stty tostop 23.10. nohup 23.11. Disowning Processes 23.12. Linux Virtual Consoles

23.12.1. What Are They? 23.12.2. Scrolling, Using a Mouse

23.13. Stopping Remote Login Sessions

24. Starting, Stopping, and Killing Processes

24.1. What's in This Chapter 24.2. fork and exec 24.3. Managing Processes: Overall Concepts 24.4. Subshells 24.5. The ps Command 24.6. The Controlling Terminal 24.7. Tracking Down Processes

24.7.1. System V 24.7.2. BSD

24.8. Why ps Prints Some Commands in Parentheses 24.9. The /proc Filesystem

24.9.1. Memory Information 24.9.2. Kernel and System Statistics 24.9.3. Statistics of the Current Process 24.9.4. Statistics of Processes by PID 24.9.5. A Glimpse at Hardware

24.10. What Are Signals? 24.11. Killing Foreground Jobs 24.12. Destroying Processes with kill 24.13. Printer Queue Watcher: A Restartable Daemon Shell Script 24.14. Killing All Your Processes 24.15. Killing Processes by Name? 24.16. Kill Processes Interactively

24.16.1. killall -i 24.16.2. zap

24.17. Processes Out of Control? Just STOP Them 24.18. Cleaning Up an Unkillable Process 24.19. Why You Can't Kill a Zombie 24.20. The Process Chain to Your Window 24.21. Terminal Windows Without Shells 24.22. Close a Window by Killing Its Process(es)

24.22.1. Example #1: An xterm Window 24.22.2. Example #2: A Web Browser 24.22.3. Closing a Window from a Shell Script

25. Delayed Execution

25.1. Building Software Robots the Easy Way 25.2. Periodic Program Execution: The cron Facility

25.2.1. Execution Scheduling 25.2.2. A Little Help, etc.

25.3. Adding crontab Entries 25.4. Including Standard Input Within a cron Entry 25.5. The at Command 25.6. Making Your at Jobs Quiet 25.7. Checking and Removing Jobs 25.8. Avoiding Other at and cron Jobs 25.9. Waiting a Little While: sleep

26. System Performance and Profiling

26.1. Timing Is Everything 26.2. Timing Programs 26.3. What Commands Are Running and How Long Do They Take? 26.4. Checking System Load: uptime 26.5. Know When to Be "nice" to Other Users...and When Not To

26.5.1. BSD C Shell nice 26.5.2. BSD Standalone nice 26.5.3. System V C Shell nice 26.5.4. System V Standalone nice

26.6. A nice Gotcha 26.7. Changing a Running Job's Niceness

VI. Scripting

27. Shell Interpretation

27.1. What the Shell Does 27.2. How the Shell Executes Other Commands 27.3. What's a Shell, Anyway?

27.3.1. How Shells Run Other Programs 27.3.2. Interactive Use Versus Shell Scripts 27.3.3. Types of Shells 27.3.4. Shell Search Paths 27.3.5. Bourne Shell Used Here 27.3.6. Default Commands

27.4. Command Evaluation and Accidentally Overwriting Files 27.5. Output Command-Line Arguments One by One 27.6. Controlling Shell Command Searches 27.7. Wildcards Inside Aliases 27.8. eval: When You Need Another Chance 27.9. Which One Will bash Use? 27.10. Which One Will the C Shell Use? 27.11. Is It "2>&1 file" or "> file 2>&1"? Why? 27.12. Bourne Shell Quoting

27.12.1. Special Characters 27.12.2. How Quoting Works 27.12.3. Single Quotes Inside Single Quotes? 27.12.4. Multiline Quoting

27.13. Differences Between Bourne and C Shell Quoting

27.13.1. Special Characters 27.13.2. How Quoting Works

27.14. Quoting Special Characters in Filenames 27.15. Verbose and Echo Settings Show Quoting 27.16. Here Documents 27.17. "Special" Characters and Operators 27.18. How Many Backslashes?

28. Saving Time on the Command Line

28.1. What's Special About the Unix Command Line 28.2. Reprinting Your Command Line with CTRL-r 28.3. Use Wildcards to Create Files? 28.4. Build Strings with { } 28.5. String Editing (Colon) Operators 28.6. Automatic Completion

28.6.1. General Example: Filename Completion 28.6.2. Menu Completion 28.6.3. Command-Specific Completion 28.6.4. Editor Functions for Completion

28.7. Don't Match Useless Files in Filename Completion 28.8. Repeating Commands 28.9. Repeating and Varying Commands

28.9.1. A foreach Loop 28.9.2. A for Loop

28.10. Repeating a Command with Copy-and-Paste 28.11. Repeating a Time-Varying Command 28.12. Multiline Commands, Secondary Prompts 28.13. Here Document Example #1: Unformatted Form Letters 28.14. Command Substitution 28.15. Handling Lots of Text with Temporary Files 28.16. Separating Commands with Semicolons 28.17. Dealing with Too Many Arguments 28.18. Expect

28.18.1. Dialback 28.18.2. Automating /bin/passwd 28.18.3. Testing: A Story 28.18.4. Other Problems

29. Custom Commands

29.1. Creating Custom Commands 29.2. Introduction to Shell Aliases 29.3. C-Shell Aliases with Command-Line Arguments 29.4. Setting and Unsetting Bourne-Type Aliases 29.5. Korn-Shell Aliases 29.6. zsh Aliases 29.7. Sourceable Scripts 29.8. Avoiding C-Shell Alias Loops 29.9. How to Put if-then-else in a C-Shell Alias 29.10. Fix Quoting in csh Aliases with makealias and quote 29.11. Shell Function Basics

29.11.1. Simple Functions: ls with Options 29.11.2. Functions with Loops: Internet Lookup 29.11.3. Setting Current Shell Environment: The work Function 29.11.4. Functions Calling Functions: Factorials 29.11.5. Conclusion

29.12. Shell Function Specifics 29.13. Propagating Shell Functions

29.13.1. Exporting bash Functions 29.13.2. FPATH Search Path

29.13.2.1. Korn shell 29.13.2.2. zsh

29.14. Simulated Bourne Shell Functions and Aliases

30. The Use of History

30.1. The Lessons of History 30.2. History in a Nutshell 30.3. My Favorite Is !$ 30.4. My Favorite Is !:n* 30.5. My Favorite Is ^^ 30.6. Using !$ for Safety with Wildcards 30.7. History by Number 30.8. History Substitutions 30.9. Repeating a Cycle of Commands 30.10. Running a Series of Commands on a File 30.11. Check Your History First with :p 30.12. Picking Up Where You Left Off

30.12.1. bash, ksh, zsh 30.12.2. C Shells

30.13. Pass History to Another Shell 30.14. Shell Command-Line Editing

30.14.1. vi Editing Mode 30.14.2. Emacs Editing Mode 30.14.3. tcsh Editing 30.14.4. ksh Editing 30.14.5. bash Editing 30.14.6. zsh Editing

30.15. Changing History Characters with histchars 30.16. Instead of Changing History Characters

31. Moving Around in a Hurry

31.1. Getting Around the Filesystem 31.2. Using Relative and Absolute Pathnames 31.3. What Good Is a Current Directory? 31.4. How Does Unix Find Your Current Directory? 31.5. Saving Time When You Change Directories: cdpath 31.6. Loop Control: break and continue 31.7. The Shells' pushd and popd Commands 31.8. Nice Aliases for pushd 31.9. Quick cds with Aliases 31.10. cd by Directory Initials 31.11. Finding (Anyone's) Home Directory, Quickly 31.12. Marking Your Place with a Shell Variable 31.13. Automatic Setup When You Enter/Exit a Directory

32. Regular Expressions (Pattern Matching)

32.21.1. Examples of Searching 32.21.2. Examples of Searching and Replacing

33. Wildcards

33.1. File-Naming Wildcards 33.2. Filename Wildcards in a Nutshell 33.3. Who Handles Wildcards? 33.4. What if a Wildcard Doesn't Match? 33.5. Maybe You Shouldn't Use Wildcards in Pathnames 33.6. Getting a List of Matching Files with grep -l 33.7. Getting a List of Nonmatching Files

33.7.1. Using grep -c 33.7.2. The vgrep Script

33.8. nom: List Files That Don't Match a Wildcard

34. The sed Stream Editor

34.1. sed Sermon^H^H^H^H^H^HSummary 34.2. Two Things You Must Know About sed 34.3. Invoking sed 34.4. Testing and Using a sed Script: checksed, runsed

34.4.1. checksed 34.4.2. runsed

34.5. sed Addressing Basics 34.6. Order of Commands in a Script 34.7. One Thing at a Time 34.8. Delimiting a Regular Expression 34.9. Newlines in a sed Replacement 34.10. Referencing the Search String in a Replacement 34.11. Referencing Portions of a Search String 34.12. Search and Replacement: One Match Among Many 34.13. Transformations on Text 34.14. Hold Space: The Set-Aside Buffer 34.15. Transforming Part of a Line 34.16. Making Edits Across Line Boundaries 34.17. The Deliberate Scrivener 34.18. Searching for Patterns Split Across Lines 34.19. Multiline Delete 34.20. Making Edits Everywhere Except... 34.21. The sed Test Command 34.22. Uses of the sed Quit Command 34.23. Dangers of the sed Quit Command 34.24. sed Newlines, Quoting, and Backslashes in a Shell Script

35. Shell Programming for the Uninitiated

35.1. Writing a Simple Shell Program 35.2. Everyone Should Learn Some Shell Programming 35.3. What Environment Variables Are Good For 35.4. Parent-Child Relationships 35.5. Predefined Environment Variables 35.6. The PATH Environment Variable 35.7. PATH and path 35.8. The DISPLAY Environment Variable 35.9. Shell Variables 35.10. Test String Values with Bourne-Shell case 35.11. Pattern Matching in case Statements 35.12. Exit Status of Unix Processes 35.13. Test Exit Status with the if Statement 35.14. Testing Your Success 35.15. Loops That Test Exit Status

35.15.1. Looping Until a Command Succeeds 35.15.2. Looping Until a Command Fails

35.16. Set Exit Status of a Shell (Script) 35.17. Trapping Exits Caused by Interrupts 35.18. read: Reading from the Keyboard 35.19. Shell Script "Wrappers" for awk, sed, etc. 35.20. Handling Command-Line Arguments in Shell Scripts

35.20.1. With the "$@" Parameter 35.20.2. With a Loop 35.20.3. Counting Arguments with $#

35.21. Handling Command-Line Arguments with a for Loop 35.22. Handling Arguments with while and shift 35.23. Loop Control: break and continue 35.24. Standard Command-Line Parsing 35.25. The Bourne Shell set Command

35.25.1. Setting Options 35.25.2. Setting (and Parsing) Parameters 35.25.3. (Avoiding?) set with No Arguments 35.25.4. Watch Your Quoting 35.25.5. Can't Set $0

35.26. test: Testing Files and Strings 35.27. Picking a Name for a New Command 35.28. Finding a Program Name and Giving Your Program Multiple Names 35.29. Reading Files with the . and source Commands 35.30. Using Shell Functions in Shell Scripts

36. Shell Programming for the Initiated

36.1. Beyond the Basics 36.2. The Story of : # #! 36.3. Don't Need a Shell for Your Script? Don't Use One 36.4. Making #! Search the PATH 36.5. The exec Command 36.6. The Unappreciated Bourne Shell ":" Operator 36.7. Parameter Substitution 36.8. Save Disk Space and Programming: Multiple Names for a Program 36.9. Finding the Last Command-Line Argument 36.10. How to Unset All Command-Line Parameters 36.11. Standard Input to a for Loop 36.12. Making a for Loop with Multiple Variables 36.13. Using basename and dirname

36.13.1. Introduction to basename and dirname 36.13.2. Use with Loops

36.14. A while Loop with Several Loop Control Commands 36.15. Overview: Open Files and File Descriptors 36.16. n>&m: Swap Standard Output and Standard Error 36.17. A Shell Can Read a Script from Its Standard Input, but... 36.18. Shell Scripts On-the-Fly from Standard Input 36.19. Quoted hereis Document Terminators: sh Versus csh 36.20. Turn Off echo for "Secret" Answers 36.21. Quick Reference: expr

36.21.1. Syntax 36.21.2. Examples

36.22. Testing Characters in a String with expr 36.23. Grabbing Parts of a String

36.23.1. Matching with expr 36.23.2. Using echo with awk or cut 36.23.3. Using set and IFS 36.23.4. Using sed

36.24. Nested Command Substitution 36.25. Testing Two Strings with One case Statement 36.26. Outputting Text to an X Window 36.27. Shell Lockfile

37. Shell Script Debugging and Gotchas

37.1. Tips for Debugging Shell Scripts

37.1.1. Use -xv 37.1.2. Unmatched Operators 37.1.3. Exit Early 37.1.4. Missing or Extra esac, ;;, fi, etc. 37.1.5. Line Numbers Reset Inside Redirected Loops

37.2. Bourne Shell Debugger Shows a Shell Variable 37.3. Stop Syntax Errors in Numeric Tests 37.4. Stop Syntax Errors in String Tests 37.5. Quoting and Command-Line Parameters 37.6. How Unix Keeps Time 37.7. Copy What You Do with script 37.8. Cleaning script Files 37.9. Making an Arbitrary-Size File for Testing

VII. Extending and Managing Your Environment

38. Backing Up Files

38.1. What Is This "Backup" Thing? 38.2. tar in a Nutshell 38.3. Make Your Own Backups 38.4. More Ways to Back Up 38.5. How to Make Backups to a Local Device

38.5.1. What to Back Up 38.5.2. Backing Up to Tape 38.5.3. Backing Up to Floppies or Zip Disks 38.5.4. To gzip, or Not to gzip?

38.6. Restoring Files from Tape with tar

38.6.1. Restoring a Few Files 38.6.2. Remote Restoring

38.7. Using tar to a Remote Tape Drive 38.8. Using GNU tar with a Remote Tape Drive 38.9. On-Demand Incremental Backups of a Project 38.10. Using Wildcards with tar

38.10.1. Without GNU tar 38.10.2. With GNU tar 38.10.3. Wildcard Gotchas in GNU tar

38.11. Avoid Absolute Paths with tar 38.12. Getting tar's Arguments in the Right Order 38.13. The cpio Tape Archiver 38.14. Industrial Strength Backups

39. Creating and Reading Archives

39.1. Packing Up and Moving 39.2. Using tar to Create and Unpack Archives 39.3. GNU tar Sampler 39.4. Managing and Sharing Files with RCS and CVS 39.5. RCS Basics 39.6. List RCS Revision Numbers with rcsrevs 39.7. CVS Basics 39.8. More CVS

40. Software Installation

40.1. /usr/bin and Other Software Directories 40.2. The Challenges of Software Installation on Unix 40.3. Which make? 40.4. Simplifying the make Process 40.5. Using Debian's dselect

40.5.1. Choosing the Access Method 40.5.2. Updating Information on Available Packages 40.5.3. Choosing Packages for Installation or Removal 40.5.4. Exiting the Select Function 40.5.5. Installing Packages 40.5.6. Configuring Packages 40.5.7. Removing Packages 40.5.8. Exiting dselect

40.6. Installing Software with Debian's Apt-Get

40.6.1. Configuring the sources.list File 40.6.2. Using apt-get

40.6.2.1. Updating information on available packages 40.6.2.2. Installing a package 40.6.2.3. Upgrading installed packages

40.7. Interruptable gets with wget 40.8. The curl Application and One-Step GNU-Darwin Auto-Installer for OS X 40.9. Installation with FreeBSD Ports 40.10. Installing with FreeBSD Packages 40.11. Finding and Installing RPM Packaged Software

41. Perl

41.1. High-Octane Shell Scripting 41.2. Checking your Perl Installation 41.3. Compiling Perl from Scratch 41.4. Perl Boot Camp, Part 1: Typical Script Anatomy 41.5. Perl Boot Camp, Part 2: Variables and Data Types

41.5.1. Scalars 41.5.2. Arrays 41.5.3. Hashes 41.5.4. References

41.6. Perl Boot Camp, Part 3: Branching and Looping 41.7. Perl Boot Camp, Part 4: Pattern Matching 41.8. Perl Boot Camp, Part 5: Perl Knows Unix 41.9. Perl Boot Camp, Part 6: Modules 41.10. Perl Boot Camp, Part 7: perldoc 41.11. CPAN

41.11.1. Installing Modules the Easy Way 41.11.2. Installing Modules the Hard Way 41.11.3. Browsing the CPAN Web Site

41.12. Make Custom grep Commands (etc.) with Perl 41.13. Perl and the Internet

41.13.1. Be Your Own Web Browser with LWP 41.13.2. Sending Mail with Mail::Sendmail 41.13.3. CGI Teaser

42. Python

42.1. What Is Python? 42.2. Installation and Distutils 42.3. Python Basics

42.3.1. Indentation 42.3.2. Functions 42.3.3. Everything's an Object 42.3.4. Modules and Packages 42.3.5. I/O and Formatting 42.3.6. wxPython

42.4. Python and the Web 42.5. urllib 42.6. urllib2 42.7. htmllib and HTMLParser 42.8. cgi 42.9. mod_python 42.10. What About Perl?

VIII. Communication and Connectivity

43. Redirecting Input and Output

43.1. Using Standard Input and Output 43.2. One Argument with a cat Isn't Enough 43.3. Send (Only) Standard Error Down a Pipe 43.4. Problems Piping to a Pager 43.5. Redirection in C Shell: Capture Errors, Too? 43.6. Safe I/O Redirection with noclobber 43.7. The ( ) Subshell Operators

43.7.1. Combining Several Commands 43.7.2. Temporary Change of Directory and Environment

43.8. Send Output Two or More Places 43.9. How to tee Several Commands into One Place 43.10. Redirecting Output to More Than One Place 43.11. Named Pipes: FIFOs 43.12. What Can You Do with an Empty File?

44. Devices

44.1. Quick Introduction to Hardware 44.2. Reading Kernel Boot Output 44.3. Basic Kernel Configuration 44.4. Disk Partitioning 44.5. Filesystem Types and /etc/fstab 44.6. Mounting and Unmounting Removable Filesystems 44.7. Loopback Mounts 44.8. Network Devices — ifconfig 44.9. Mounting Network Filesystems — NFS, SMBFS 44.10. Win Is a Modem Not a Modem? 44.11. Setting Up a Dialup PPP Session 44.12. USB Configuration 44.13. Dealing with Sound Cards and Other Annoying Hardware 44.14. Decapitating Your Machine — Serial Consoles

45. Printing

45.1. Introduction to Printing 45.2. Introduction to Printing on Unix

45.2.1. lpr-Style Printing Commands 45.2.2. lp-Style Printing Commands

45.3. Printer Control with lpc 45.4. Using Different Printers 45.5. Using Symbolic Links for Spooling 45.6. Formatting Plain Text: pr 45.7. Formatting Plain Text: enscript 45.8. Printing Over a Network 45.9. Printing Over Samba

45.9.1. Printing to Unix Printers from Windows 45.9.2. Printing to Windows Printers from Unix

45.10. Introduction to Typesetting 45.11. A Bit of Unix Typesetting History 45.12. Typesetting Manpages: nroff 45.13. Formatting Markup Languages — troff, LATEX, HTML, and So On 45.14. Printing Languages — PostScript, PCL, DVI, PDF 45.15. Converting Text Files into a Printing Language 45.16. Converting Typeset Files into a Printing Language 45.17. Converting Source Files Automagically Within the Spooler 45.18. The Common Unix Printing System (CUPS) 45.19. The Portable Bitmap Package

46. Connectivity

46.1. TCP/IP — IP Addresses and Ports

46.1.1. Internet Protocol (IP) 46.1.2. Layer 4 Protocols: TCP, UDP, and ICMP

46.2. /etc/services Is Your Friend 46.3. Status and Troubleshooting 46.4. Where, Oh Where Did That Packet Go? 46.5. The Director of Operations: inetd 46.6. Secure Shell (SSH) 46.7. Configuring an Anonymous FTP Server 46.8. Mail — SMTP, POP, and IMAP 46.9. Domain Name Service (DNS) 46.10. Dynamic Host Configuration Protocol (DHCP) 46.11. Gateways and NAT 46.12. Firewalls 46.13. Gatewaying from a Personal LAN over a Modem

47. Connecting to MS Windows

47.1. Building Bridges 47.2. Installing and Configuring Samba 47.3. Securing Samba 47.4. SWAT and GUI SMB Browsers 47.5. Printing with Samba 47.6. Connecting to SMB Shares from Unix 47.7. Sharing Desktops with VNC

47.7.1. Connecting to a Windows VNC server 47.7.2. Setting up VNC on Unix

47.8. Of Emulators and APIs

47.8.1. VMWare 47.8.2. Wine

47.9. Citrix: Making Windows Multiuser

47.9.1. Citrix Metaframe 47.9.2. rdesktop 47.9.3. Hob

IX. Security

48. Security Basics

48.1. Understanding Points of Vulnerability 48.2. CERT Security Checklists 48.3. Keeping Up with Security Alerts 48.4. What We Mean by Buffer Overflow 48.5. What We Mean by DoS 48.6. Beware of Sluggish Performance

48.6.1. Check Processes 48.6.2. Checking Swap Space 48.6.3. Check Network Connections 48.6.4. Other Checks

48.7. Intruder Detection 48.8. Importance of MOTD 48.9. The Linux proc Filesystem 48.10. Disabling inetd 48.11. Disallow rlogin and rsh 48.12. TCP Wrappers

49. Root, Group, and User Management

49.1. Unix User/Group Infrastructure 49.2. When Does a User Become a User 49.3. Forgetting the root Password 49.4. Setting an Exact umask 49.5. Group Permissions in a Directory with the setgid Bit 49.6. Groups and Group Ownership 49.7. Add Users to a Group to Deny Permissions 49.8. Care and Feeding of SUID and SGID Scripts 49.9. Substitute Identity with su 49.10. Never Log In as root 49.11. Providing Superpowers with sudo 49.12. Enabling Root in Darwin 49.13. Disable logins

50. File Security, Ownership, and Sharing

50.1. Introduction to File Ownership and Security 50.2. Tutorial on File and Directory Permissions

50.2.1. User, Group, and World 50.2.2. Which Group is Which?

50.3. Who Will Own a New File? 50.4. Protecting Files with the Sticky Bit 50.5. Using chmod to Change File Permission 50.6. The Handy chmod = Operator 50.7. Protect Important Files: Make Them Unwritable 50.8. cx, cw, c-w: Quick File Permission Changes 50.9. A Loophole: Modifying Files Without Write Access 50.10. A Directory That People Can Access but Can't List 50.11. Juggling Permissions 50.12. File Verification with md5sum 50.13. Shell Scripts Must Be Readable and (Usually) Executable 50.14. Why Can't You Change File Ownership? 50.15. How to Change File Ownership Without chown

51. SSH

51.1. Enabling Remote Access on Mac OS X 51.2. Protecting Access Through SSH 51.3. Free SSH with OpenSSH 51.4. SSH Problems and Solutions 51.5. General and Authentication Problems 51.6. Key and Agent Problems 51.7. Server and Client Problems

Glossary Index About the Authors Colophon SPECIAL OFFER: Upgrade this ebook with O’Reilly

← Prev
Back
Next →

← Prev
Back
Next →