Docs/Drafts/CommandLineSurvivalGuide/Concepts

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Concepts

This section presents a quick overview of the background concepts and necessary theory before delving into the details of the command line interface.

Shells

Before graphical user environments such as Gnome and KDE desktops existed, users interacted with their computers using the shell as an interface. UNIX/Linux operating systems are multi-tasking/multi-user system. Applications, programs, and commands were run on the computer using a shell. When a command is specified at the shell, the shell tries to locate it and execute it (there are more internal activities that occur but the details are elaborated in the later parts of this guide).

Typical Fedora installations are booted into the the default graphical environment mode. Use the Terminal program to perform the command line tasks. It is accessed through Applications > Accessories > Terminal, which launches the gnome-terminal program. When the terminal opens, it shows a command shell prompt, such as:

[username@hostname ~] $

The dollar sign symbol $ represents that the user is logged in to the shell as an ordinary-level user. If the user is logged in as the root user, the prompt shows a # symbol, also called the hash symbol. In this guide, these prompts are not used when demonstrating commands.

Interaction with the shell is performed by specifying commands at the command prompt. Entering Iwantcheese at the prompt and pressing the [Enter] key on renders the following output:

[username@hostname ~] $Iwantcheese
Iwantcheese: Command not found.

Case-sensitivity is preserved in Linux
While whoami is a command, WhoAmI or WHOAMI or other similar variations are not the same command.

Some of the common shell environments are:

• Thompson shell [sh: /bin/sh] - The first original shell implemented by Ken Thompson for the Unix operating system
• Bourne shell [sh: /bin/sh] - Revised Thompson shell developed by Stephen Bourne, still popular default in many Unix operating systems
• C shell [csh: /bin/sh] - First developed by Bill Joy for the BSD Unix system
• Tenex C shell [tcsh: /bin/tcsh] - an improved C shell by Ken Greer later improved by Paul Placeaway and Wilfred Sanchez. This replaced C shell in most popular Unix based OSes.
• Korn Shell [ksh: /bin/ksh] - Developed by David Korn, combining powerful features of the C and Bourne shell
• Bourne again shell [ bash: /bin/bash] - an improvised Bourne shell was developed for the GNU project by Brian Fox and later maintained by Chet Ramey.

While there are different types of shells, the Bourne again shell (bash) and Tcsh (tcsh) are popularly used in Linux. This guide focuses on the default bash shell.

Bourne again Shell

The bash shell is the default on most Linux distros. Its rich feature set supersedes the Bourne shells capabilites by effectively combining the potentials of both the ksh and csh. Chet Ramey also maintains an interesting FAQ at ftp://ftp.cwru.edu/pub/bash/FAQ .

The Environment

Describes that the environment plays an important part of a Linux system. Describes how environment variables are used to supply options, paths, etc., providing flexibility in the system's setup.

Directories, Files, and Executables

Point to resources on Linux file systems

Scripts

Describes that commands may be grouped in various ways to provide a "macro" series of processing.

stdout, stderr, stdin

Point to references on the 3 standard streams and their normal operation. This is a setup for the redirection and pipes topic.

Redirection and Pipes

Follows discussion of the standard streams to show that they may be piped and redirected.

Permissions

Point to resources on Linux file permissions

Bash prompt customization

Very brief overview that the prompt can be customized. This will refer to other documents for more information.

Linux File Permission

Written by Kristina Ban.

Linux files are setup so access to them is controlled. There are three types of access: 1.read 2.write 3.execute Each file belongs to a specific user and group. Access to the files is controlled by user, group, and what is called other (term used to refer to someone who is not the user (owner) of the file, nor a member of the group the file belongs to. When talking about setting permissions for "other" users to use, it is commonly referred to as setting the world execute, read, or write bit since anyone in the world will be able to perform the operation if the permission is set in the other category.

File names and permission characters

File names can be up to 256 characters long with "-", "_", and "." characters along with letters and numbers. When a long file listing is done, there are 10 characters that are shown on the left that indicate type and permissions of the file. File permissions are shown according to the following syntax example: -rwerwerwe There are a total of 10 characters in this example, as in all Linux files. The first character indicates the type of file, and the next three indicate read, write, and execute permission for user, then group and then other. Since there are three types of permission for three users, there are a total of nine permission bits. The syntax is as follows: 1 -- file type 2,5,8 -- read (r) 3,6,9 -- write (w) 4,7,10 -- execute (x) 2,3,4 -- user permissions 5,6,7 -- group permissions 8,9,10 -- other permissions

Character 1 is the type of file: - is ordinary, d is directory, l is link, c is character mode files, b is block mode files, s is socket. Characters 2-4 show owner permissions. Character 2 indicates read permission, character 3 indicates write permission, and character 4 indicates execute permission. Characters 5-7 show group permissions. Character 5=read, 6=write, 7=execute Characters 8-10 show permissions for all other users. Character 8=read, 9=write, 10=execute There are 5 possible characters in the permission fields. They are: r = read - This is only found in the read field. w = write - This is only found in the write field. x = execute - This is only found in the execute field. s = setuid - This is only found in the execute field. If there is a "-" in a particular location, there is no permission. This may be found in any field whether read, write, or execute field. Examples Type "ls -l" and a listing like the following is displayed: total 10

drwxrwxrwx 4 kristina students 122 Dec 12 18:02 Projects -rw-rw-rw- 1 kristina students 1873 Aug 23 08:34 test -rw-rw-rw- 1 kristina students 1234 Sep 12 11:13 datafile

Which means the following: drwxrwxrwx -- Type and Permission field 4 -- Number of Links kristina -- File's Owner students -- File's Group 122 -- Size in Bytes Dec 12 18:02 -- Date of last modification Projects -- Filename

Commands

Chown - CHange the OWNer

This command changes the user and group ownership of each given file to a new owner and it can also change the ownership of a file to match the user or group of an existing reference line.

The syntax is:

Chown [options ] <user[:group]> <file|directory>[file|directory…]

The options include $ -R (recursively)change the permission on files that are in the subdirectories of the directory that you are currently in. $ -v (verbose mode) shows all actions performed for every file $ -c like –r (verbose) but shows only when a change is made

EXAMPLE

$ chown me test.txt The new owner of the file test.txt is me. $ chown me test1 test2

 The owner of the file test1 and test2 is now me. 

$chown me *.doc test/ Changes the ownership of all directories whose name ends with .doc and all files and subdirectories in the test/ directory to the user me and reports only files affected by the command.

Chgrp-CHange GRouP

The command chgrp is used to change the group owner of the file or directory. This is a root command and only root user can change the group owner of the file or directory.

The syntax is:

Chgrp [options ] <group> <file|directory>[file|directory…]

The options include $-f Force. Do not report errors. $-h If the file is a symbolic link, change the group of the symbolic link. Without this option, the group of the file referenced by the symbolic link is changed. $-R Recursive. chgrp descends through the directory, and any subdirectories, setting the specified group ID as it proceeds. When a symbolic link is encountered, the group of the target file is changed (unless the -h option is specified), but no recursion takes place.

EXAMPLE

$ chgrp me test.txt The new group owner of the file test.txt is me

$ chgrp –R me test/ The files and subdirectories also change the group ownership to me.

Chmod: CHange MODe

The command chmod changes permission on file and directories.

The syntax is

Chmod [options ] <change mode> <file|directory>[file|directory…]

The change mode can be specified in two ways : 1. Numeric permission

400 read by owner 040 read by group 004 read by anybody (other) 200 write by owner 020 write by group 002 write by anybody 100 execute by owner 010 execute by group 001 execute by anybody

EXAMPLE $chmod 755 test.cgi This would be the following 400+040+004+200+100+010+001 = 755 where you are giving all the rights except the capability for anyone to write to the test.cgi file(-rwxr-xr-x). 2.Expresion Permissions are expressed by a sequence of expressions separated by comas. The expression takes the following form [category] <+|-|=> [permission]

The category may be -u (user) permission for owner -g (group) permission for owner group -o (other) permission for ‘others’ If the category isn’t specified it will apple to all the categories.

A ‘+’ sets a permission but a ‘–‘removes it and a ‘=’ sets the permissions.

The permissions are -r for read -w for write -x for execute

EXAMPLE

$ chmod o-wx test

Removes the permission to write and execute for others

$chmod og-w test.txt Removes the permission to write for the groups and others

I/O Redirection

Standard Output

Most command line programs that display their results do so by sending their results to a facility called standard output. By default, standard output directs its contents to the display. To redirect standard output to a file, the ">" character is used like this: $ command > filename (redirect command output to a file)

EXAMPLE -ls > file_list.txt In this example, the ls command is executed and the results are written in a file named test.txt. Since the output of ls was redirected to the file, no results appear on the display. Each time the command above is repeated, test.txt is overwritten (from the beginning) with the output of the command ls. If you want the new results to be appended to the file instead, use ">>" like this: $ command >> filename APPEND into a file

EXAMPLE $ ls >> test.txt When the results are appended, the new results are added to the end of the file, thus making the file longer each time the command is repeated. If the file does not exist when you attempt to append the redirected output, the file will be created.

Standard Input

Many commands can accept input from a facility called standard input. By default, standard input gets its contents from the keyboard, but like standard output, it can be redirected. To redirect standard input from a file instead of the keyboard, the "<" character is used like this: $ command < filename Type a text file and pass the text to command

EXAMPLE

$ sort < test.txt

In the above example we used the sort command to process the contents of test.txt. The results are output on the display since the standard output is not redirected in this example. We could redirect standard output to another file like this:

$sort < test1.txt > test2.txt

As you can see, a command can have both its input and output redirected. Be aware that the order of the redirection does not matter. The only requirement is that the redirection operators (the "<" and ">") must appear after the other options and arguments in the command.

Standard Error

Command line programs send error messages to the user via standard error, abbreviated stderr. As is the case with standard output, its destination is the display screen by default, and it can likewise be redirected . Standard error is a separate data stream from standard output in order to allow the two streams to be redirected separately and thus prevent them from becoming intermingled.

Standard error can be redirected with the basic standard error redirection operator, which consists of the numeral 2 followed without an intervening space by a rightward facing angular bracket. In this case, it will create the file to which it is redirected if it does not yet exist, or it will overwrite the contents of the file if a file with the same name already exists. In the following example, the error message will be sent to a file named test and will not appear on the display screen: cat nofile 2> test An alternative is to use the standard error appending operator, which appends any error messages to the end of the text in the file rather than overwriting it. This operator, which consists of the numeral 2 followed with no intervening spaces by two rightward facing angular brackets, is useful for error log files. Redirect to NUL (hide errors)

 command  2> nul            Redirect error messages to NUL
 command  >nul 2>&1         Redirect error and information messages to NUL
 command  >filename 2> nul  Redirect info to file but suppress error
(command) >filename 2> nul  Redirect info to file but suppress CMD.exe errors

EXAMPLE $ ls -l myfile.txt 2> test.txt If the ls command doesn’t find the myfile.txt it will redirect the error in the test.txt file.

Pipes

By far, the most useful and powerful thing you can do with I/O redirection is to connect multiple commands together with what are called pipes. With pipes, the standard output of one command is fed into the standard input of another.

commandA | commandB Pipe the output from commandA into commandB

For example $ls -l | less In this example, the output of the ls command is fed into less. By using "| less" you can make any command have scrolling output. $ ls –lt | head Displays the 10 newest files in the current directory

Filters

One class of programs you can use with pipes is called filters. Filters take standard input and perform an operation upon it and send the results to standard output. In this way, they can be used to process information in powerful ways. Here are some of the common programs that can act as filters: Common filter commands are:

sort Sorts standard input then outputs the sorted result on standard output. uniq Given a sorted stream of data from standard input, it removes duplicate lines of data (i.e., it makes sure that every line is unique). grep Examines each line of data it receives from standard input and outputs every line that contains a specified pattern of characters. fmt Reads text from standard input, then outputs formatted text on standard output. pr Takes text input from standard input and splits the data into pages with page breaks, headers and footers in preparation for printing. head Outputs the first few lines of its input. Useful for getting the header of a file. tail Outputs the last few lines of its input. Useful for things like getting the most recent entries from a log file. tr Translates characters. Can be used to perform tasks such as upper/lowercase conversions or changing line termination characters from one type to another (for example, converting DOS text files into Unix style text files). sed Stream editor. Can perform more sophisticated text translations than tr. awk An entire programming language designed for constructing filters. Extremely powerful.

EXAMPLE $ ls | grep x

The ls command produces a list of files in the current directory, while the grep command reads the output of ls and prints only those lines containing the letter x.

$ ls | wc -l > count.txt

The ls command list all the files in the current directory and this list it is piped to wc, which, when used with its -l option, counts the number of lines and it is redirected to the file count.txt

$ cat * | grep "Linux" | grep -v "UNIX" | wc –l

Search the contents of all of the files in current directory and display the total number of lines in them that contain the string Linux but not the string UNIX.

Discovering Commands and Shell Completion

Written by Tea Dasović.

DISCOVERING COMMANDS IS MISSING.

Most powerful and underused features of many shells, like ksh, bash or zsh, is filename completion facility, just like features in C shell and old operating systems like DEC TOPS-20.

Idea behing filename completion is when you need to type a filename, you shouldn't have to type more then necessary to identify the file. This is a very good idea so it can save you quite a bit of typing.

The most primitive completion is when instead of typing in the arguments/options following a command, the user presses TAB button and shell automatically gives you the option of inserting new text based on the command you are typing/editing.

Example: Let's say we want to extract files from UNIX archive (something like .tar or .tar.gz) which is downloaded to the /tmp directory

We should type: $ tar -zxvf (then we press TAB button on the keyboard) Assuming archive.tar.gz is the only .tar.gz archive in a directory: $ tar -zxvf archive.tar.gz

If we had chosen -xvf instead, it would complete on files ending in .tar. $ tar -xvf (then we press TAB button on the keyboard) Again, assuming archive.tar.gz is the only .tar archive in a directory, command results like this: $ tar -xvf archive.tar

***THIS IS TAKEN FROM SOMEWHERE ON THE NET; SHOULD BE REPLACED.***

This is where things get really nifty. Say you only want to extract the file zsh-3.1.6-pws-11/Completion/User/_perl_builtin_funcs from zsh-3.1.6-pws-11.tar.gz. No need to type in all of that massive filename. Instead, going through the process step by step:

/tmp % tar zxf z (then we press TAB button on the keyboard) /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz (then we press TAB button on the keyboard again) /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/ /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/C(TAB) ChangeLog ChangeLog.3.0 Completion/ Config/ /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Com(TAB) /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Completion/ /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Completion/U(TAB) /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Completion/User/ /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Completion/User/_pe(TAB) _perl_basepods _perl_builtin_funcs _perl_modules _perldoc /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Completion/User/_perl(TAB) /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Completion/User/_perl_basepods(TAB again) /tmp % tar zxf zsh-3.1.6-pws-11.tar.gz zsh-3.1.6-pws-11/Completion/User/_perl_builtin_funcs

--.

Using completion in this way, you can use the completion system as a way of browsing around directory structures. Note that this applies equally outside compressed archives, if you were intending to view or edit a file but didn't know exactly where it was, for example.

BASH COMPLETION MODES

Bash can support few completion modes with practical importance and return on investment for other modes can be negative in general:

complete (TAB) -- This attempts to perform completion on the text before point. Bash attempts completion treating the text as a variable if the text in command begins with ‘$’ (dollar sign), username if the text begins with ‘-‘, hostname if the text begins with ‘@’ and commands in turn. If there is no match with any of this, filename completion is attempted. possible-match (M-?) -- List the possible completions of the text before point. insert-completion (M-*) -- Insert all completions of the text before point that would have been generated by possible-completions. menu-complete () -- Similar to complete, but replaces the word to be completed with a single match from the list of possible completions. Repeated execution of menu-complete steps through the list of possible completions, inserting each match in turn. At the end of the list of completions, the bell is rung (subject to the setting of bell-style) and the original text is restored. An argument of n moves n positions forward in the list of matches; a negative argument may be used to move backward through the list. This command is intended to be bound to TAB, but is unbound by default. delete-char-or-list () -- Deletes the character under the cursor if not at the beginning or end of the line (like delete-char). If at the end of the line, behaves identically to possible-completions. This command is unbound by default. complete-filename (M-/) -- Attempt filename completion on the text before point. possible-filename-completions (C-x /) -- List the possible completions of the text before point, treating it as a filename. complete-username (M-~) -- Attempt completion on the text before point, treating it as a username. possible-username-completions (C-x ~) -- List the possible completions of the text before point, treating it as a username. complete-variable (M-$) -- Attempt completion on the text before point, treating it as a shell variable. possible-variable-completions (C-x $) -- List the possible completions of the text before point, treating it as a shell variable. complete-hostname (M-@) -- Attempt completion on the text before point, treating it as a hostname. possible-hostname-completions (C-x @) -- List the possible completions of the text before point, treating it as a hostname. complete-command (M-!) -- Attempt completion on the text before point, treating it as a command name. Command completion attempts to match the text against aliases, reserved words, shell functions, shell builtins, and finally executable filenames, in that order. possible-command-completions (C-x !) -- List the possible completions of the text before point, treating it as a command name. dynamic-complete-history (M-TAB) -- Attempt completion on the text before point, comparing the text against lines from the history list for possible completion matches. complete-into-braces (M-{) -- Perform filename completion and insert the list of possible completions enclosed within braces so the list is available to the shell

Customizing Command Line Environment

Alias

In computing, alias is a command in various shells which enables a replacement of a word with another string. It is used for abbreviating a system command, or for adding default arguments to a regularly used command. Typically, an alias will last for the life of the shell session but regularly used aliases can be placed in the shell’s configuration file (~/.cshrc or the systemwide /etc/csh.cshrc for csh, or ~/.bashrc or the systemwide /etc/bashrc or /etc/bash.bashrc for bash) so that they will be available for all shell sessions.

Creating aliases Example of the bash shell syntax is: alias copy=”cp”

Overriding aliases Example: alias ls=’ls –la’

To override the alias and execute the ls command as it was originally defined, the following syntax can be used: ‘ls’

or

\ls

Removing aliases unalias copy – removes the copy alias unalias –a – removes all aliases

etc/profile

The /etc/profile file contains system wide environment stuff and startup programs. All customizations that you put in this file will apply for the entire environment variable on your system.

PS1

In bash, you can set your prompt by changing the value of the PS1 environment variable, as follows:

$ export PS1="\u@\H > " will result: tea@dasovic >

We can use PS1 and combine it with colors:

***WE SHOULD INSERT A COLOR CHART HERE***

To use this chart, find the color you'd like to use, and find the corresponding foreground (30-37) and background (40-47) numbers. For example, if you like green on a normal black background, the numbers are 32 and 40. Then, take your prompt definition and add the appropriate color codes. Here is the example:

export PS1="\[\e[36;1m\]\u@\[\e[32;1m\]\H> \[\e[0m\]

THIS DOCUMENTATION IS IN DRAFT STAGE AND NEEDS TO BE DEVELOPED.