Docs/Drafts/CommandLineSurvivalGuide/Concepts

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Concepts

This section presents a quick overview of the background concepts and necessary theory before delving into the details of the command line interface.

Shells

Before graphical user environments such as Gnome and KDE desktops existed, users interacted with their computers using the shell as an interface. UNIX/Linux operating systems are multi-tasking/multi-user system. Applications, programs, and commands were run on the computer using a shell. When a command is specified at the shell, the shell tries to locate it and execute it (there are more internal activities that occur but the details are elaborated in the later parts of this guide).

Typical Fedora installations are booted into the the default graphical environment mode. Use the Terminal program to perform the command line tasks. It is accessed through Applications > Accessories > Terminal, which launches the gnome-terminal program. When the terminal opens, it shows a command shell prompt, such as:

[username@hostname ~] $

The dollar sign symbol $ represents that the user is logged in to the shell as an ordinary-level user. If the user is logged in as the root user, the prompt shows a # symbol, also called the hash symbol. In this guide, these prompts are not used when demonstrating commands.

Interaction with the shell is performed by specifying commands at the command prompt. Entering Iwantcheese at the prompt and pressing the [Enter] key on renders the following output:

[username@hostname ~] $Iwantcheese
Iwantcheese: Command not found.

Case-sensitivity is preserved in Linux
While whoami is a command, WhoAmI or WHOAMI or other similar variations are not the same command.

Some of the common shell environments are:

• Thompson shell [sh: /bin/sh] - The first original shell implemented by Ken Thompson for the Unix operating system
• Bourne shell [sh: /bin/sh] - Revised Thompson shell developed by Stephen Bourne, still popular default in many Unix operating systems
• C shell [csh: /bin/sh] - First developed by Bill Joy for the BSD Unix system
• Tenex C shell [tcsh: /bin/tcsh] - an improved C shell by Ken Greer later improved by Paul Placeaway and Wilfred Sanchez. This replaced C shell in most popular Unix based OSes.
• Korn Shell [ksh: /bin/ksh] - Developed by David Korn, combining powerful features of the C and Bourne shell
• Bourne again shell [ bash: /bin/bash] - an improvised Bourne shell was developed for the GNU project by Brian Fox and later maintained by Chet Ramey.

While there are different types of shells, the Bourne again shell (bash) and Tcsh (tcsh) are popularly used in Linux. This guide focuses on the default bash shell.

Bourne again Shell

The bash shell is the default on most Linux distros. Its rich feature set supersedes the Bourne shells capabilites by effectively combining the potentials of both the ksh and csh. Chet Ramey also maintains an interesting FAQ at ftp://ftp.cwru.edu/pub/bash/FAQ .

The Environment

Describes that the environment plays an important part of a Linux system. Describes how environment variables are used to supply options, paths, etc., providing flexibility in the system's setup.

Directories, Files, and Executables

Point to resources on Linux file systems

Scripts

Describes that commands may be grouped in various ways to provide a "macro" series of processing.

stdout, stderr, stdin

Point to references on the 3 standard streams and their normal operation. This is a setup for the redirection and pipes topic.

Redirection and Pipes

Follows discussion of the standard streams to show that they may be piped and redirected.

Permissions

Point to resources on Linux file permissions

Bash prompt customization

Very brief overview that the prompt can be customized. This will refer to other documents for more information.

Linux File Permission

Linux files are setup so access to them is controlled. There are three types of access: 1.read 2.write 3.execute Each file belongs to a specific user and group. Access to the files is controlled by user, group, and what is called other (term used to refer to someone who is not the user (owner) of the file, nor a member of the group the file belongs to. When talking about setting permissions for "other" users to use, it is commonly referred to as setting the world execute, read, or write bit since anyone in the world will be able to perform the operation if the permission is set in the other category.

File names and permission characters

File names can be up to 256 characters long with "-", "_", and "." characters along with letters and numbers. When a long file listing is done, there are 10 characters that are shown on the left that indicate type and permissions of the file. File permissions are shown according to the following syntax example: -rwerwerwe There are a total of 10 characters in this example, as in all Linux files. The first character indicates the type of file, and the next three indicate read, write, and execute permission for user, then group and then other. Since there are three types of permission for three users, there are a total of nine permission bits. The syntax is as follows: 1 -- file type 2,5,8 -- read (r) 3,6,9 -- write (w) 4,7,10 -- execute (x) 2,3,4 -- user permissions 5,6,7 -- group permissions 8,9,10 -- other permissions

Character 1 is the type of file: - is ordinary, d is directory, l is link, c is character mode files, b is block mode files, s is socket. Characters 2-4 show owner permissions. Character 2 indicates read permission, character 3 indicates write permission, and character 4 indicates execute permission. Characters 5-7 show group permissions. Character 5=read, 6=write, 7=execute Characters 8-10 show permissions for all other users. Character 8=read, 9=write, 10=execute There are 5 possible characters in the permission fields. They are: r = read - This is only found in the read field. w = write - This is only found in the write field. x = execute - This is only found in the execute field. s = setuid - This is only found in the execute field. If there is a "-" in a particular location, there is no permission. This may be found in any field whether read, write, or execute field. Examples Type "ls -l" and a listing like the following is displayed: total 10

drwxrwxrwx 4 kristina students 122 Dec 12 18:02 Projects -rw-rw-rw- 1 kristina students 1873 Aug 23 08:34 test -rw-rw-rw- 1 kristina students 1234 Sep 12 11:13 datafile

Which means the following: drwxrwxrwx -- Type and Permission field 4 -- Number of Links kristina -- File's Owner students -- File's Group 122 -- Size in Bytes Dec 12 18:02 -- Date of last modification Projects -- Filename

Commands

Chown - CHange the OWNer

This command changes the user and group ownership of each given file to a new owner and it can also change the ownership of a file to match the user or group of an existing reference line.

The syntax is:

Chown [options ] <user[:group]> <file|directory>[file|directory…]

The options include $ -R (recursively)change the permission on files that are in the subdirectories of the directory that you are currently in. $ -v (verbose mode) shows all actions performed for every file $ -c like –r (verbose) but shows only when a change is made

EXAMPLE

$ chown me test.txt The new owner of the file test.txt is me. $ chown me test1 test2

 The owner of the file test1 and test2 is now me. 

$chown me *.doc test/ Changes the ownership of all directories whose name ends with .doc and all files and subdirectories in the test/ directory to the user me and reports only files affected by the command.

Chgrp-CHange GRouP

The command chgrp is used to change the group owner of the file or directory. This is a root command and only root user can change the group owner of the file or directory.

The syntax is:

Chgrp [options ] <group> <file|directory>[file|directory…]

The options include $-f Force. Do not report errors. $-h If the file is a symbolic link, change the group of the symbolic link. Without this option, the group of the file referenced by the symbolic link is changed. $-R Recursive. chgrp descends through the directory, and any subdirectories, setting the specified group ID as it proceeds. When a symbolic link is encountered, the group of the target file is changed (unless the -h option is specified), but no recursion takes place.

EXAMPLE

$ chgrp me test.txt The new group owner of the file test.txt is me

$ chgrp –R me test/ The files and subdirectories also change the group ownership to me.

Chmod: CHange MODe

The command chmod changes permission on file and directories.

The syntax is

Chmod [options ] <change mode> <file|directory>[file|directory…]

The change mode can be specified in two ways : 1. Numeric permission

400 read by owner 040 read by group 004 read by anybody (other) 200 write by owner 020 write by group 002 write by anybody 100 execute by owner 010 execute by group 001 execute by anybody

EXAMPLE $chmod 755 test.cgi This would be the following 400+040+004+200+100+010+001 = 755 where you are giving all the rights except the capability for anyone to write to the test.cgi file(-rwxr-xr-x). 2.Expresion Permissions are expressed by a sequence of expressions separated by comas. The expression takes the following form [category] <+|-|=> [permission]

The category may be -u (user) permission for owner -g (group) permission for owner group -o (other) permission for ‘others’ If the category isn’t specified it will apple to all the categories.

A ‘+’ sets a permission but a ‘–‘removes it and a ‘=’ sets the permissions.

The permissions are -r for read -w for write -x for execute

EXAMPLE

$ chmod o-wx test

Removes the permission to write and execute for others

$chmod og-w test.txt Removes the permission to write for the groups and others

I/O Redirection

Standard Output

Most command line programs that display their results do so by sending their results to a facility called standard output. By default, standard output directs its contents to the display. To redirect standard output to a file, the ">" character is used like this: $ command > filename (redirect command output to a file)

EXAMPLE -ls > file_list.txt In this example, the ls command is executed and the results are written in a file named test.txt. Since the output of ls was redirected to the file, no results appear on the display. Each time the command above is repeated, test.txt is overwritten (from the beginning) with the output of the command ls. If you want the new results to be appended to the file instead, use ">>" like this: $ command >> filename APPEND into a file

EXAMPLE $ ls >> test.txt When the results are appended, the new results are added to the end of the file, thus making the file longer each time the command is repeated. If the file does not exist when you attempt to append the redirected output, the file will be created.

Standard Input

Many commands can accept input from a facility called standard input. By default, standard input gets its contents from the keyboard, but like standard output, it can be redirected. To redirect standard input from a file instead of the keyboard, the "<" character is used like this: $ command < filename Type a text file and pass the text to command

EXAMPLE

$ sort < test.txt

In the above example we used the sort command to process the contents of test.txt. The results are output on the display since the standard output is not redirected in this example. We could redirect standard output to another file like this:

$sort < test1.txt > test2.txt

As you can see, a command can have both its input and output redirected. Be aware that the order of the redirection does not matter. The only requirement is that the redirection operators (the "<" and ">") must appear after the other options and arguments in the command.

Standard Error

Command line programs send error messages to the user via standard error, abbreviated stderr. As is the case with standard output, its destination is the display screen by default, and it can likewise be redirected . Standard error is a separate data stream from standard output in order to allow the two streams to be redirected separately and thus prevent them from becoming intermingled.

Standard error can be redirected with the basic standard error redirection operator, which consists of the numeral 2 followed without an intervening space by a rightward facing angular bracket. In this case, it will create the file to which it is redirected if it does not yet exist, or it will overwrite the contents of the file if a file with the same name already exists. In the following example, the error message will be sent to a file named test and will not appear on the display screen: cat nofile 2> test An alternative is to use the standard error appending operator, which appends any error messages to the end of the text in the file rather than overwriting it. This operator, which consists of the numeral 2 followed with no intervening spaces by two rightward facing angular brackets, is useful for error log files. Redirect to NUL (hide errors)

 command  2> nul            Redirect error messages to NUL
 command  >nul 2>&1         Redirect error and information messages to NUL
 command  >filename 2> nul  Redirect info to file but suppress error
(command) >filename 2> nul  Redirect info to file but suppress CMD.exe errors

EXAMPLE $ ls -l myfile.txt 2> test.txt If the ls command doesn’t find the myfile.txt it will redirect the error in the test.txt file.

Pipes

By far, the most useful and powerful thing you can do with I/O redirection is to connect multiple commands together with what are called pipes. With pipes, the standard output of one command is fed into the standard input of another.

commandA | commandB Pipe the output from commandA into commandB

For example $ls -l | less In this example, the output of the ls command is fed into less. By using "| less" you can make any command have scrolling output. $ ls –lt | head Displays the 10 newest files in the current directory

Filters

One class of programs you can use with pipes is called filters. Filters take standard input and perform an operation upon it and send the results to standard output. In this way, they can be used to process information in powerful ways. Here are some of the common programs that can act as filters: Common filter commands are:

sort Sorts standard input then outputs the sorted result on standard output. uniq Given a sorted stream of data from standard input, it removes duplicate lines of data (i.e., it makes sure that every line is unique). grep Examines each line of data it receives from standard input and outputs every line that contains a specified pattern of characters. fmt Reads text from standard input, then outputs formatted text on standard output. pr Takes text input from standard input and splits the data into pages with page breaks, headers and footers in preparation for printing. head Outputs the first few lines of its input. Useful for getting the header of a file. tail Outputs the last few lines of its input. Useful for things like getting the most recent entries from a log file. tr Translates characters. Can be used to perform tasks such as upper/lowercase conversions or changing line termination characters from one type to another (for example, converting DOS text files into Unix style text files). sed Stream editor. Can perform more sophisticated text translations than tr. awk An entire programming language designed for constructing filters. Extremely powerful.

EXAMPLE $ ls | grep x

The ls command produces a list of files in the current directory, while the grep command reads the output of ls and prints only those lines containing the letter x.

$ ls | wc -l > count.txt

The ls command list all the files in the current directory and this list it is piped to wc, which, when used with its -l option, counts the number of lines and it is redirected to the file count.txt

$ cat * | grep "Linux" | grep -v "UNIX" | wc –l

Search the contents of all of the files in current directory and display the total number of lines in them that contain the string Linux but not the string UNIX.

THIS DOCUMENTATION IS IN DRAFT STAGE AND NEEDS TO BE DEVELOPED.