In this section, we will learn more essential Linux Bash commands that every Linux user should know. Use the cat command to quickly cut columns out of text files. This is like a light version of awk.
We'll be discussing the following commands:
- cat
- sort
- awk
- tee
- tar
- Other miscellaneous commands
First, let's create a smaller version of the passwd file to work with the cat command:
-d sets the field delimiter; by default it's the tab character. -f uses a single field number or comma-separated list of field numbers that you want to extract. If using comma-separated lists also, the split input delimiter will be output, which can be changed using -- output-delimiter.
Next, let's create a smaller version of the services file without comments and empty lines. Using the cat command is very limited to the special use case that a file separator is a single character, such as a colon or the tab character. For splitting text files as multiple consecutive whitespace characters, which are often used in Linux config files, for example, in the /etc/services file, the cat command does not work. Also, when using the cat command, the field order must be fixed in every line or you will run into problems.
In the following screenshot, you can see that the services file contains no tab separator, but multiple whitespace characters marked with the star character:
If you use cat on this file, it will produce nothing but garbage. To split files with multiple consecutive whitespaces, use awk instead. The tr command is like a lightweight version or subset of the set substitute mode. It translates a character set one into a character set two, reading from stdin and outputting to stdout. The syntax is self-explanatory. You can translate both single characters and ranges of characters. The character sets are similar to POSIX regular expression classes; read the manual to find out more.
Let's discuss the sort command. The sort command sorts the text file line by line. By default, it takes the whole line into account for sorting. The -u flag only prints out unique fields. If we take a file that has numbers instead of alphanumeric values, by default, sort expects alphanumeric values, so the sorting of numbers is wrong or unnatural. To fix this, use the -n option, which sorts using numbers. To sort values from bottom to top, use the -r flag. You can also influence the sort column if you need. sort always takes the whole line into account. To fix this, use the -k 2.2 option to sort by the second column. There are many more sort options. Refer to the manual to find out more.
Now, in order to combine the power of cat or awk, sort and unique, let's use these tools together to print the 10 most recurring service names from the /etc/services file while ignoring comments and empty lines:
The second command should now be pretty self-explanatory. As you can see, discard, exp1, and exp2 are the most recurring service names in the /etc/services file with four occurrences. To count all lines in the file, use the wc for the word count command. To extract the pure filename from a path, use the basename command, which is often used in scripts, as we will see later. If you know the extension of a file, you can also use the basename command to extract the filename from the extension. Similarly, to extract the path name, use the dirname command. To measure the time a command needs, execute the prefix of your command with the time command. To compare two files, use the diff command, which will print an empty output. If these files are identical, there will be no output. Otherwise, the changes between the files will be shown. The diff command can also be used for comparing two directories, file by file, using the recursive flag, which will go through all the files from A and compare them to the corresponding size files from B with the same name in folder B. The command that can be used to print out where a specific command is located in the filesystem is based on the /path variable, which we will see later.
tee is a useful command, which can be used to store an stdout command in a file, as well as print it on the command line. It is useful for keeping a record of an output while also seeing what's going on at the same time. Just give the tee command the filename you want to write to as an argument. To compress a single file, which means reduce the file size, use gzip. To uncompress, use the gunzip command.
To compress a complete subdirectory, recursively use the tar command. Note that the f option must be the last option followed by the archive name you want to create as the first argument and then the directory you want to archive and compress as the second argument. To extract an archive to any following directory, use the tar command with the following flag, -C is the output directory. hostname prints out the hostname; uptime prints out how long the server computer is powered on, and uname prints system information such as the kernel version.
In the /etc/redhat-release file, you will find the version of Red Hat Enterprise that this CentOS 7 is based on. In the /prog/meminfo file, you will find memory information, for example, how much RAM you have. In /proc/cpuinfo, you will find information about your CPUs and cores. free -m prints out useful memory information, for example, how much free RAM you've got. df prints out information about the available disk space. du -page prints out how much space the files in the current directory take. If you use it with the max-depth=1 option, you will also get a summary of the folder content. users print out all the users currently logged in to the system. The whoami command prints the name of the user who is currently using this Terminal.
Now, we'll see some very useful commands. To print the current date and time, use the date command. Use +%s to generate a unique timestamp. To print out a calendar, use the cal command. To pause, interrupt the shell execution using the sleep command. The dd program, or disk dump, is a very essential tool every Linux user needs to know. It is used to copy data from an input file or device to an output file or device. We have used dd before, in the first section, to override the filesystem's free space with zeros so we can shrink our VM images, but there are many more use cases for the dd command. dd basic syntax uses if for input file and of for output file as arguments. Also, two options are very important, the block size and the count.
You will see that the block size, which means the amount of data read at once, is 1 MB, and the count is the amount of repetitions of the block size, so that, in our example, 1 MB multiplied by 1,024 equals exactly 1 GB. dd also supports reading from stdin and writing to stdout so that the command we just used can be rewritten as dd if=/dev/zero of=/tmp/1gig_file.empty bs=1M count=1024. You can use dd not only with device files, but also to copy normal files. Also, you can use it for creating images of whole partitions, for example, for backups. To access partitions, the root account is needed.