<< notation, 35
* (asterisk)
in cron entry, 157
for globbing files, 306, 346–348
^ operator, 87
$(( )) notation, 34–36
. (period)
escaping for grep command, 70
for hidden files, 12
to source script, 43
“404 Not Found” errors, preventing, 220
2001: A Space Odyssey (movie), 274
-a (logical AND), 25–26
access_log file, 235–239
splitting from error_log, 243
Acey Deucey, 290–297
addagenda script, 90–95
adduser script, 131–133
admin account, for apm, 226, 228
agenda script, 90–95
album script, 211–213
aliases, 104
alphanumeric input, validating, 15–17
alternatives system (Debian), 104
AND, logical (-a), 25–26
ANSI color sequences, 40–42
for region highlighting, 107–109
Apache web server
access_log file, 235–239
error_log file, 242–246
installing, 201
managing passwords, 223–229
apm-footer.html, 227–228
apm script, 223–229
apt package manager, installing with, 201
archive
file, emailing or copying to cloud storage, 166
remote, for backups, 246–249
of removed files, 58–62
archivedir script, 169–171
area codes, looking up, 183–185
askvalue() function, 148–149
asterisk (*), in cron entry, 157
Atkin, sieve of, 287
automated screencapture, 263–266
for disk capacity, 125–126
for displaying random text, 213–214
general format for script, 189
printf command, 33–34
backups
automated, based on sftp, 106, 229–233
of directories, 169–171
of files as they’re removed, 55–58
managing, 166–169
.bash_profile (login script), 4–5, 12
.bashrc (login script), 4–5, 12
bash shell, 1–2
customizing, 4
installing on Windows, 341–344
running commands, 3–4
running shell scripts, 5–7
batch files, 175
bc program, wrappers for, 34–36, 82–85
bestcompress script, 113–115
Bitcoin, address information retrieval, 192–194
blank phrase, vs. zero-character quoted phrase, 17
bold type, 41–42
border for image, 318–322
brew package manager, installing with, 329
broken internal links, identifying, 217–220
bugs. See debugging
bulkrename script, 346–348
bulkrun script, 348–351
calc script, 82–85
calculations
currency values, 190–192
loan payments, 87–90
calculators
floating-point, 34–36
interactive, 82–85
calendar program, 90–95
capitals of states quiz, 282–284
carriage return, tr command to replace with newline, 262
case sensitivity, of Unix, 72
case statements, as regular expressions, 72
cat command
alternative to, 101–103
compressed files and, 109–112
-n flag, 98–99
OS X files and, 262
printing file contents to screen, 4
reading user data with, 81
Celsius units, translating between Fahrenheit or Kelvin and, 85–87
CentOS, cgi-bin directory for, 201
CGI (Common Gateway Interface) scripts, 199
running, 201
viewing environment, 202–203
cgrep script, 107–109
change mode (chmod) command, 7, 57
changetrack script, 194–197
chattr command, 64
checkexternal script, 220–222
checklinks script, 217–220
chmod (change mode) command, 7, 57
Chrome operating system (Google), 299
city, checking time in, 76
cleaning up after guest user, 141–143
cloud storage, 299
creating slide shows from photo streams, 304–306
emailing or copying archive file to, 166
keeping Dropbox running, 300–301
syncing Dropbox, 301–304
syncing files with Google Drive, 307–309
color sequences, ANSI, 40–42
for region highlighting, 107–109
command line interface, shell as, 2
commands
count of those in PATH, 51–52
running, 3–4
running in bulk, 348–351
Common Gateway Interface. See CGI (Common Gateway Interface) scripts
Common Log Format, 235
composite utility, 317
compressed files, 109–112, 113–115
convertatemp script, 85–87
convertcurrency script, 190–192
Coordinated Universal Time (UTC), 73
copyright
header in bc program, silencing, 36
issues, 207
cron
archivedir in, 171
ensuring jobs are run, 159–162
scheduling jobs with, 154
crontab
entry for netstat log generation, 254
for office document synced with Google Drive, 308
validating user entries, 154–159
currency, calculating values, 190–192
Darwin (Unix core), 261
database
checking size of, 70
searching with locate, 68–71
secure search, 127–131
data storage. See cloud storage; disk usage
date formats
normalizing, 17–20
validating, 29–32
variations in operating systems, 95, 149
dates
calculating days between, 332–335
calculating days until specified, 335–339
finding day for specific past date, 330–332
finding moon phase for, 351–353
setting on system, 148–150
date utility, 95, 329–330, 334
dayinpast script, 330–332
daysago script, 332–335
daysuntil script, 335–339
Debian, cgi-bin directory for, 201
debugging
shell scripts, 45–49
using shell scripts for, 199
decimal separator, 22–23
deleted files, restoring, 55
.deleted-files archive, 55, 57
pruning by timestamp, 62
deleteuser script, 136–138
deleting user accounts, 136–138
development folder, creating, 4–5
df utility
improving readability of output, 123–125
reports on disk usage, 125
dice, 287–290
diff command, 196–197
directories
Apache support for password protection, 223
backing up, 169–171
bash search of, 3
displaying contents of, 65–68
name as Terminal window title, 266–267
for thumbnail images, 322
DIR script, 71–73
disabling user account, 133–136
diskhogs script, 121–123
diskspace script, 125–127
disk usage
analysis of, 119–120
available space, 125–127
managing quotas, 121–123
docron script, 159–162
domain names, requesting list from website, 179
dot (.) notation. See . (period)
double quotes, nested, 61
downloading files with FTP, 174–177
Dropbox
keeping running, 300–301
syncing, 301–304
du command, 119
echo command, 3
echon command, 33–34
of archive file to cloud storage, 166
warnings about disk space consumption, 121–123
web page as, 209–211
emulating
GNU-style flags with quota, 103–104
MS-DOS environment, 71–73
encoded strings, transmitting, 201
end of file (EOF), unexpected, 47
end-of-line character, in OS X files, 262–263
env command, 202
environment variables, 11, 47. See also PATH environment variable
EOF (end of file), unexpected, 47
Eratosthenes, sieve of, 287
error messages
“404 Not Found,” preventing, 220
from date --version, 329
File does not exist, 243
from ImageMagick, 321
permission denied, 129
unable to read font, 318
esc variable, 42
/etc/crontab file, 160
/etc/daily directory, 160, 161
/etc/group file, 131
/etc/monthly directory, 160, 161
/etc/shadow file, 131
/etc/skel directory, 132–133
/etc/weekly directory, 160, 161
events
calendar, tracking, 90–95
web, logging, 203–206
exec call, 105
executable file, making, 6–7
EXIF (exchangeable image file format) information, removing, 324
EXIT (SIGEXIT) signal, 109
exit 0 command, 43
Extensible Markup Language (XML), 174
external links, report on broken, 220–222
extracting URL, from web page, 177–180
Fahrenheit units, translating between Celsius or Kelvin and, 85–87
file command, 314
File does not exist error, 243
file extension, for scripts, 5
filelock script, 37–40
filenames, user-specific database of, 129
file permissions, for executing script, 6–7
files
backing up as removed, 55–58
displaying
with additional information, 101–103
with line numbers, 98–99
by printing to console screen, 4
downloading with FTP, 174–177
finding by filename, 68–71
identify tool for information, 314–315
locking, 37–40
logging removals, 62–65
multiple, with same name, 57
renaming, in bulk, 346–348
sourcing, 42
storage. See cloud storage
syncing with SFTP, 229–233
uploading to FTP server, 177
File Transfer Protocol (FTP). See FTP (File Transfer Protocol)
FileZilla, 174
find command, 165
-xdev argument, 120
finding
files by filename, 68–71
programs in PATH, 11–15
specific commands, 3
findsuid script, 146–148
fixguest script, 141–143
flags
default for commands, 79
GNU-style, emulating with quota, 103–104
floating-point
calculations, 84
calculator, 34–36
input, validating, 26–29
limitation of, 99
formatdir script, 65–68
formatting long lines, 53–55
<form> tag (HTML), 204
fquota script, 119–120
frameit script, 318–322
framing images, 318–322
FreeBSD
command count, 52
ps output, 150
FTP (File Transfer Protocol)
for downloading files, 174–177
limitations, 229
uploading to server, 177
ftpget script, 174–177
ftp program, secure version, 104–106
fuzzy matching, 284
games, 273
Acey Deucey, 290–297
dice, 287–290
hangman, 277–281
number-guessing game, 45–49
prime numbers, 285–287
state capitals quiz, 282–284
unscramble, 275–277
gedit, 5
geoloc script, 325–327
getbtcaddr script, 192–194
getdope script, 209–211
gethubuser script, 180–182
getlinks script, 178–180
getstats script, 250–251
GitHub, getting user information, 180–182
globbing
for bulk-renaming files, 346–348
disabling, 306
GMT (Greenwich Mean Time), 73
GNU-style flags, emulating with quota, 103–104
Chrome operating system, 299
Currency Converter, 190–192
Drive, syncing files with, 307–309
GPS geolocation, interpreting information, 325–327
Grab utility, 263
graphics. See ImageMagick tool; images; photos
grep command, fixing, 107–109
compressed files and, 109–112
gsync subdirectory, 308
Guenther, Philip, 37
guest user, cleaning up after, 141–143
hangman game, 277–281
hangman script, 277–281
hard links, 111
hidden files, period (.) for, 12
Holbrook, Bill, 207
home directory, identifying, 5
.htaccess data file, 223, 226, 228
HTML
for formatting online photo album, 212
parsing with lynx tool, 177
HTTP_USER_AGENT string, for Safari web browser, 202–203
HUP (SIGHUP) hang-up signal, 135
identify tool, 314–315, 325–327
IEEE (Institute for Electrical and Electronic Engineers), 10
IFS (input field separator), 226
ImageMagick tool, 213, 313. See also images
convert tool
for animated GIF, 265
for borders, 318–322
display command and, 304–306
error messages from, 321–322
GPS geolocation interpretation, 325–327
identify tool, 314–315, 325–327
images. See also ImageMagick tool; photos
EXIF information, stripping, 324
framing, 318–322
inclusion from other website, 207–208
report on reference errors, 220
scaled versions, creating, 213, 306, 322–325
size analyzer, 314–315
watermarking, 316–318
imagesize script, 314–315
IMDb (Internet Movie Database), movie info, accessing, 187–190
#include feature, shell alternative to, 42
incremental backup, 166–169
initializeANSI script function, 40–42
input
extracting specified line from, 61
getting and passing to function, 13
validating
alphanumeric, 15–17
floating-point, 26–29
integer, 23–26
phone number, 17
input field separator (IFS), 226
Institute for Electrical and Electronic Engineers (IEEE), 10
integer input, validating, 23–26
internal links, identifying broken, 217–220
Internet Movie Database (IMDb), movie info, accessing, 187–190
Internet tools. See also web pages
Apache access_log, 235–239
Apache error_log, 242–246
area code lookup, 183–185
Bitcoin address information retrieval, 192–194
currency value calculation, 190–192
extracting URL from web page, 177–180
FTP for downloading files, 174–177
GitHub user information, 180–182
identifying broken internal links, 217–220
IMDb, accessing movie info from, 187–190
logging web events, 203–206
monitoring network status, 249–255
photo album creation, 211–213
random text display, 213–215
scriptable, 173
search engine traffic, 239–242
tracking web page changes, 194–197
watermarking images, 316–318
weather, 185–186
ZIP code lookup, 182–183
isprime script, 285–287
italic type, 41–42
iTunes libraries, summary listings in OS X, 267–269
JPEG files, finding dimensions of, 314–315
Kelvin units, translating between Fahrenheit or Celsius and, 85–87
kevin-and-kell script, 207–208
keywords, use in search engines, 239
KILL (SIGKILL) signal, 135
killall script, 150–154
killing processes by name, 150–154
large numbers, presenting attractively, 20–23
leading slash, 13
left-rooting pattern, 153
length, in bc, 36
library
iTunes, summary listings in OS X, 267–269
of scripts, building, 42–45
library-test script, 43–45
line feeds, for Unix, 262
lines of text
context in file, displaying with grep, 107–109
displaying numbers for, 98–99
end-of-line characters, 262–263
extracting from input, 61
formatting long, 53–55
merging paired, 67
wrapping only long, 99–101
links
identifying broken internal, 217–220
symbolic vs. hard, 111
Linux
netstat command output format, 250
ps command output format, 151
systems, designed to run as servers, 159
loancalc script, 87–90
loan payments, calculating, 87–90
local permissions, FTP to retain, 231
.locatedb file, separate for each user, 127
locate script, 69
locating. See finding
locking files, 37–40
log-duckduckgo-search script, 203–206
log files
Apache access_log, 235–239
Apache error_log, 242–246
displaying in order, 99
netperf script for contents analysis, 255
ownership permission, 64–65
rotating, 162–166
splitting for web server, 243
logging
file removals, 62–65
web events, 203–206
logical AND (-a), 25–26
logrm script, 62–65
long-words.txt file, 275–277
lookup
area code, 183–185
ZIP code, 182–183
for displaying backed-up file, 169
file listings in chronological order, 58
lynx tool, 173–174
extracting URLs with, 177–180
identifying broken internal links with, 218–220
MacKenzie, David, 329
mailbox, remote, as archive, 247
MAILER environment variable, 11
Mailinator, 209
Microsoft
Windows, bash on, 341–344
mklocatedb script, 68–71
mkslocatedb script, 127–128
mogrify utility, 322
monthNumToName() function, 17–20
moon phase, finding by date, 351–353
moonphase script, 351–353
compressed files and, 109–112
moviedata script, 187–190
movies, info from IMDb (Internet Movie Database), 187–190
MS-DOS environment, emulating, 71–73
multiprocessor machines, bulk-running commands on, 348–351
mysftp script, 104–106
name
finding files by, 68–71
killing processes by, 150–154
renicing processes by, 255–259
ncal program, 330–331
NcFTP, 174
neqn shell script, 5–6
nested double quotes, 61
netstat log file, 253–254
network, monitoring status, 249–255
network filesystem (NFS), lockfile and, 40
Network Time Protocol (NTP), 150
newdf script, 123–125
newline
causing unexpected end of file, 47
tr command to replace carriage return with, 262
newquota script, 103–104
newrm script, 55–58
NFS (network filesystem), lockfile and, 40
nicenumber script, 20–23, 89, 236
nroff command, 53
NTP (Network Time Protocol), 150
number-guessing game, 45–46
numberlines script, 98–99
numbers. See also calculations; floating-point
integers, validating input, 23–26
large, presenting attractively, 20–23
number-guessing game, 45–46
prime, 285–287
scientific notation, 28–29
OneDrive (Microsoft), 299, 304
open2 script, 269–271
open application, 262
open command, fixing in OS X, 269–271
OpenOffice documents, page count for folder of, 7–8
operating systems. See also individual operating systems
MS-DOS, emulating, 71–73
number of commands available in, 52
OS X
automated screencapture, 263–266
command count, 52
date format, 149
fixing line endings, 262–263
killing all csmount processes on, 153
open command, 269–271
ps output, 151
summary listings of iTunes libraries, 267–269
Terminal app, 2
setting title dynamically, 266–267
user account database for, 131
voice synthesis system, 309–312
output device, redirecting, 221–222
PAGER environment variable, 11
paging, 81
paired lines, merging, 67
palindrome checker, 274
parsing HTML, with lynx tool, 177
password-protected account, FTPand,177
passwords
for Apache, 223–229
changing for user, 135
htpasswd program for encryption, 226
PATH environment variable, 3
checking for valid directories, 139
configuring, 4–5
count of commands in, 51–52
finding programs in, 11–15
pax command, 168
period (.)
escaping for grep command, 70
for hidden files, 12
to source script, 43
permissions
default, for newly created file, 57
FTP to retain local, 231
log file ownership, 64–65
phase of the moon, finding by date, 351–353
phone number, validating, 17
photos. See also images
creating slide show from cloud storage, 304–306
creating web-based album, 211–213
pipe, with sftp program, 231
portable shell scripts, 7
POSIX (Portable Operating System Interface), 10–11
Preview utility, 263
prime numbers, 285–287
priority of task
changing, 255–259
for time-critical programs, 258
processes
killing by name, 150–154
renicing by name, 255–259
running in parallel, 348–351
.profile (login script), 4–5, 12
prompt, for bash shell, 4
protocols, information on supported, 249–250
query from web client, 202
QUERY_STRING variable, 205
quota, emulating GNU-style flags with, 103–104
quota analysis, of disk usage, 119–120
$RANDOM environment variable, 47
random number, generating, 46–47, 287–290
randomquote script, 213–215, 276
random text, displaying, 213–215
read command, 81
realrm command, 56
records, 82
Red Hat Linux, ps output, 151
region, checking time in, 76
region highlighting, ANSI color sequences for, 107–109
case statement conditional tests as, 72
for variable slicing, 14
remember script, 80–82
remindme script, 80–82
remote archive, for backups, 246–249
remotebackup script, 246–249
remote host, prompting for, 104–106
removed file archive, 58–62
renaming files in bulk, 346–348
renicename script, 255–259
reset sequence, for ANSI color sequences, 41
restoring deleted files, 55
retransmission percentage, for network traffic, 254
return code, from awk, 243
right-rooting pattern, 153
rm command, 55
rolldice script, 287–290
root user, running script as, 69
rot13, 274
rotatelogs script, 162–166
running
commands, 3–4
commands in bulk, 348–351
script, as root user, 69
RVM (Ruby version manager), 4
Safari web browser, HTTP_USER_AGENT string for, 202, 203
sayit script, 310–312
say utility (OS X), 309–312
scale, in bc, 36
scheduling jobs, with cron, 154
scientific notation, 28–29
screencapture, automated, 263–266
screencapture script, 263
screencapture2 script, 264–266
scriptbc script, 34–36, 82–85, 194, 236
for converting GPS data, 327
scripts. See shell scripts
search engine traffic, 239–242
searchinfo script, 240–241
secure locate, implementing, 127–131
security
for apm script, 229
CGI scripts and, 199
for images, 316–318
risk in web form collection of email address, 200–201
script as root and, 69
substitution, 86
transform based on, 16
semaphore, 37
sendmail, 200–201
servers, Linux systems designed to run as, 159
server-side include (SSI) capability, of web servers, 213–215
setdate script, 148–150
setgid command, checking for improper protection, 146–148
setuid command, checking for improper protection, 146–148
setuid permission, 64
SFTP, syncing files with, 229–233
sftpsync script, 229–233
wrapper for, 232–233
sftp utility, 104–106
sh command, 6
shell alias, 71
shell scripts, 1–3
calling directly, 6–7
debugging, 45–49
determining line count, 118
file extension for, 5
library of, 42–45
programming environment, 9–10
reasons for using, 7–8
running, 5–6
shift command, 54
showCGIenv script, 202
showfile script, 101–103
.shtml file extension, 215
shuffleDeck function, 290–291, 295
shutdown of system, cron job and, 159
sieve algorithms, for prime numbers, 287
SIGEXIT (EXIT) signal, 109
SIGHUP (HUP) hang-up signal, 135
SIGKILL (KILL) signal, 135
SkyDrive, 304
slideshow script, 304–306
.slocatedb file, 130–131
slocate script, 128–129
SNMP (Simple Network Management Protocol), 131
Solaris, command count, 52
Soundex algorithm, 284
source command (bash), 43
sourcing a file, 42–43
ssh (Secure Shell), 104–106, 229
SSI (server-side include) capability, of web servers, 213–215
ssync script, 232–233
startdropbox script, 300–301
state capitals quiz, 282–284
stderr, 221–222
stdin, 81
stdout, 221–222
Stickies, 80
storage. See cloud storage; disk usage
subshell, 5
substitution cipher, 274
Sundaram, sieve of, 287
suspending user accounts, 133–136
suspenduser script, 133–136, 137
symbolic links, 111
syncdropbox script, 301–304
syncgdrive script, 307–309
syncing files
in Dropbox, 301–304
with Google Drive, 307–309
with SFTP, 229–233
syslog data stream, adding entry to, 64
system daemons, user IDs for, 119
system() function, 124
system maintenance, 145
backups, 166–171
ensuring cron jobs are run, 159–162
killing processes by name, 150–154
rotating log files, 162–166
setting system date, 148–150
tracking set user ID applications, 146–148
validating user crontab entries, 154–159
tail command, for monitoring website searches, 206
tarball, 232
TCP (Transmission Control Protocol)
netstat command for information, 253
script analysis of, 249–250
tcsh shell, 2
temperatures, converting, 85–87
Terminal app
opening window, 2
setting title dynamically, 266–267
text, displaying random, 213–215
TextEdit, 5
text editors, 5
thousands delimiter, international variations, 22–23
thumbnails script, 322–325
time
displaying in different time zones, 73–77
validating specifications, 32
timed(8), 150
timein script, 73–77
timestamp
adding to renamed files, 348
for backup, 168
changing, 58
pruning .deleted-files archive by, 62
time zones, 73–77
titleterm script, 266–267
toolong script, 99–101
tracking
calendar events, 90–95
web page changes, 194–197
trap command, 108–109
traverse.dat file, 220
traverse function, 218
to replace carriage return with newline, 262
trimmailbox script, 249
type effects, 41–42
TZ time zone variable, 73
Ubuntu
cgi-bin directory for, 201
command count, 52
default web root on, 212
umask value, 57
unable to read font error, 318
underlined type, 41–42
unexpected end of file (EOF), 47
Unix
case sensitivity, 72
early development, 10
netstat command output format, 250
philosophy, on commands, 52
tweaking, 97–115
file display with additional information, 101–103
file display with line numbers, 98–99
GNU-style flags, emulating with quota, 103–104
grep, customizing, 107–109
maximizing file compression, 113–115
sftp, customizing, 104–106
working with compressed files, 109–112
wrapping long code lines, 99–101
unrm script, 58–62
unscramble (word game), 275–277
uploading files to FTP server, 177
uppercase letters, requiring in input, 17
URLs
extracting from web page, 177–180
user visit prior to page request, 236
user accounts
adding, 131–133
deleting, 136–138
suspending, 133–136
user IDs for, 119
user commands, 51–53
user IDs, for system daemons and user accounts, 119
user input. See input
user management, 117–118. See also user accounts
available disk space, 125–127
cleaning up after guest user, 141–143
df output readability, 123–125
diskhogs script, 121–123
disk usage analysis, 119–120
getting GitHub information, 180–182
secure locate, 127–131
user environment validation, 139–141
/usr directory, 119
UTC (Coordinated Universal Time), 73
uuencode command, 248
validating
alphanumeric input, 15–17
date formats, 29–32
floating-point input, 26–29
integer input, 23–26
phone number input, 17
time specifications, 32
user environment, 139–141
validator script, 139–141
valid-date script, 29–32
van den Berg, Stephen, 37
variables
echo command to track, 45
names for search value, 241
naming scheme for, 254
variable-slicing syntax, 13, 14, 141
verifycron script, 154–159
Vim, 5
voice synthesis system, 309–312
watermarking images, 316–318
watermark script, 316–318
wc (word count) command, 100
weather, 185–186
Weather Underground, 185
webaccess script, 236–239
web-based photo album creation, 211–213
web client, query from, 202
weberrors script, 242–246
web events, logging, 203–206
web form, collecting email address, as security risk, 200–201
web pages
building on the fly, 207–208
as email message, 209–211
extracting URL from, 177–180
tracking changes, 194–197
user visit prior to page request, 236
web servers
managing, 235
server-side include (SSI) capability, 213–215
splitting logs, 243
websites
getlinks for site analysis, 180
tail command for monitoring searches, 206
which command, 3
whoami command, 69
Windows 10, bash shell, 341–344
word count (wc) command, 100
wrappers, 53
installing, 63
open2 script, 269–271
for rm command, 62–65
for sftpsync, 232–233
wrapping only long lines of text, 99–101
X11 (graphics library), 304
xargs command, 67
XML (Extensible Markup Language), 174
xmllint, calling, 186
XQuartz software package, 304
Yahtzee, 289
yum package manager, installing with, 201
zcat command, 109–112
zero-character quoted phrase, vs. blank phrase, 17
zgrep command, 109–112
ZIP code lookup, 182–183
zmore command, 109–112
zsh shell, 2