The ``twelve'' most important Unix commands
Carl Mason
carlm@demog.berkeley.edu
rev 1.32 Fall 2013
Although Unix has a point and click graphic user interface, called
X11, which works just like those other operating systems, Unix is at heart a
command line operating system. So while it is possible in many cases
to do what you want via pointing and clicking, using the command line
and other text based tools will make you happier and
much more
efficient... eventually.
To operate with the command line, you will need to know the 12 most
important Unix commands described in Section 5. To
enjoy it you will also need to know a few tricks that are also covered
in this document.
You don't need to know much about Unix in order to start
doing Science, but it would not hurt to learn more. In your copious
free time, check out some of the Unix primers on the web. Ask google
something like ``Unix beginner'' to find more resources than you could
possibly want.
Note that since the Mac OS is simply a Unix application, nearly
everything in this document works the same way in a mac. On a mac,
the terminal window application is under Applications/Utilities.
Terminal windows
In order to use the command line or
shell, you must open a
terminal window (also known an
xterm window). A
terminal window can be launched from:
[Application][Accessories][Terminal].
It should look something like Figure 1. Notice
that the window features a menu bar, unfortunately the menu bar it is
both useless and misleading. Make the menu bar disappear by pressing
the
and clearing the ``show Menubar'' check-box.
Now aside from the title bar at the top, the only words in the
terminal window should be the Unix prompt. The purpose of the
Unix prompt is to indicate that the shell is ready to accept
commands. It also contains useful information.
In Figure 1, the prompt is is [carlm@twins ~]$
,
indicating the user, carlm, the machine, twins and the
current directory which is indicated by the ~
. In this and
other documents, the Unix prompt will look like this: @: . In the real
Unix prompt, the symbol
~
is a special character whose
meaning is "home directory".
~/Dissertation
means a file or
directory called "Dissertation" which is located within your home
directory. In my case this would be
/hdir/0/carlm/Dissertation. ~wachter/Brilliant/insight
translates to a file (or possibly a directory) called insight
in a directory1 called Brilliant in Ken Wachter's home
directory, or /hdir/0/wachter/Brilliant/insight. More about
home directories can be found in Section 2.
If you are in no particular hurry to finish your dissertation, you can
modify a large number of colors and beeps and other important features
of the terminal window.
[Edit Current Profile] is the place
to start wasting time.
If you have already wasted time on this sort of thing and are thus old
enough to find the default font a bit small, a useful trick with
terminal windows (and browser windows too) is the
+
+
to increase and
+
to
decrease the size of the typeface.
Figure 1:
terminal window
|
The Filesystem
Whenever you login to a machine on the Demography network, your
initial present working directory - the location within the
filesystem in which applications will begin looking for the files
that you specify - is your home directory. Every user has exactly one
home directory.
In a multiuser system such as the Demography Lab, your home directory
is one of a huge number of interconnected directories that form a
single unified filesystem. The magic of the filesystem is that
even though
the various files and directories of which it is composed are
``physically''/footnoteor electromagnetically present on various different machines all over the network, to us
users, the whole thing appears to be one single thing and that thing
looks and feels the same no matter which Demography Lab machine we
happen to be using at the moment.
An upside down tree makes a pretty good metaphor for the filesystem.
Such a ``tree'' is shown in Figure 2. At the top of
the figure is a directory called ``/'' which is the ``root'' of the
filesystem. Every file and directory in the filesystem can be uniquely
specified by a filepath that begins with root. For example, the
file that holds my correspondence with my mother is
/hdir/0/carlm/mail/mom.
As you can see in Figure 2 home directories all live in
a directory called /hdir/0. Although it is just one of many
directories within this giant upside down tree of a filesystem, your
home directory is a special place that you will come to know and love
and where you will do your very best work. It is the part of the
filesystem that you own and the ``place'' where you will find yourself
when you first login.
Because the entire filesystem looks the same to all users all the
time, it is easy to share data with your colleagues. This is good
thing because humanity benefits when scientists collaborate. But
unfortunately scientists can occasionally turn out to be creeps so
sharing a filesystem is a little scary as well.
The ``solution'' to the creep problem is to not keep sensitive
information on Demography computers. You have already promised not to
keep data covered by SB 13862. It goes without
saying that files that can tie you to illegal activities are also a
no-no. There are however, a few files that belong on the network and
yet where privacy is an issue (e.g. email). For those files, managing
who may read and/or change them requires understanding the mode and
ownership of files. Each file and directory has an owner and the
owner can determine who is allowed to read, write and/or execute each
file. See the chmod command below for how to change the
various file modes or permissions. The chmod command
is described in 6.
Figure 2:
The Demography Lab filesystem
|
The command interpreter
The command interpreter, or
shell is the program that runs in
each terminal window. It waits
for you to type something at the
Unix prompt,
@: , and then does
what
it thinks you meant. The shell we use here is called
bash (pronounced ``bash''). Bash is one of several modified
versions of the original sh (pronounced ``s-h'') shell.
The most important thing that the shell does for you is to let you give
commands to the computer. These include the 12 most important Unix
commands (Section 5) as well as commands to launch
applications like R, Stata, word processors or spreadsheets3. The shell does
several other things for you some are essential, some enhance
efficiency and others are just cool.
Among the essential features of the shell is a mechanism to
communicate with running programs that are not expecting user input
or have run amok. This is not all that common, but when it happens
you need to be able to get the program's attention and tell it -
generally to drop dead.
+
does this.
Below is a list of some handy/essential key combinations that the
shell recognizes:
To make you more efficient, the shell offers three particularly
nice features: ``history'', ``
completion'' and ``scripting''.
The history feature allows you to recall
and edit any command that you have previously issued. To make
the
previous command appear at the
@: hit
+
or equivalently
the up arrow key. To see
even more previous stuff type
+
more times.
+
or equivalenlty
the down arrow will make the
next command appear -
obviously, this makes no sense unless you have typed
+
at
least once.
You can operate on a recalled command using several standard emacs
editing keys:
-
+
To go to the beginning of the current line
-
+
To go to the end of the current line
-
+
To go forward one character
-
+
To go backward one character
You can also use the left arrow and right arrow to
move about within a recalled line. The
and
keys do what you would expect.
If you hit the
key anytime
while constructing a command, the shell will do it's best to figure out
what you
are planning to type next. If you are typing a
command it will try to find a command that starts out with what you
have already typed. If you are typing the name of a file the shell will try
to complete if for you. If what you have typed does not uniquely
determine a command or filename, the shell will beep at you and provide a
list of possible completions. You can then
type a few more characters and hit
again.
Whenever you find yourself typing the same
command several times, it's time to consider scripting. A shell script
is just a file of commands that you
could have entered at the
keyboard, but typed into a file instead. You can then set the file's
execute bit (See Most Important Command number
6) and
execute that file - perhaps now, perhaps later. You will need to use
an editor such as emacs to create that shell script. Knowing how to
use emacs can save you lots of time and hair loss - particularly if
many of the commands you are typing are quite similar.
Scripts are also very useful for people who like the idea of being
able to reproduce results.
The shell is also responsible for displaying the results of the
ls
command (See
1) in lots of colors.
Regular expressions are combinations of symbols that the shell
interprets in clever ways. Generally we use regular expressions to
specify lists of files or directories on which a command should
operate, but they have many other uses as well. A typical use would be
to
delete from your current working directory all of the .pdf files
whose name begins with a vowel:
@: rm [AEIOUaeiou].pdf
regular expressions come up in several of the ``12'' important
commands.
Below is a list of the 12 most important Unix commands. They are
simple enough to be easily memorized by anyone who can keep
the names of all twelve months in his head.
For the most part, these commands are for logging on and off; for
printing; and for moving files and directories around. Many of these
commands functions can be done using a file manager or under emacs, but
knowing how to do them from the command line, makes you more
efficient, reduces errors and opens the possibility of
automating tasks with shell scripts.
- ls -ltr regular expression The ``list
file'' command, ls is used -not surprisingly- to list the names and
pertinent information about some or all of the files in a particular
directory. The most common and useful option is -l that's a
lower case L not a one. It reveals the most interesting properties of
your files. Adding tr causes ls to present it results sorted by
time in reverse order.
@: ls -ltr directory1
produces a list of files sorted so that the most recent ones are at
the bottom of the list.
A regular expression is a set of special characters (or
``meta-characters'') that can be used to represent a list of files or
more generally, the set of all character strings with certain
characteristics. Regular expressions come up in a lot of places and
can be very useful when working with datasets that are full text
and/or errors. Regular expressions are used with ls
to limit the number of files displayed. For example ls X*Y
would list only the filenames that start with an X and end with a Y;
ls [aeiou]* would show you all the files that begin with a lower case
vowel (In the previous example the [ ] are part of the command)
- mkdir -p new-directory-name The ``make directory''
command is used to create a new sub-directory of the current
working directory. The -p argument causes mdir to create ``parent'' directories as needed. In other words,
@: mdir -p first/second/third
would create a a directory called third which would be a
subdirectory of second which in turn would be a subdirectory of
first. The -p argument instructs mkdir to create
second and/or first if they do not already exist.
- cd directory-name The ``change directory'' command
makes another directory your present working directory. With no
argument, it "moves you" to your home directory. To move one
directory "higher" use ".." (two dots) in place of the directory's
name. The one and only parent directory of the current directory is
always addressable as "..".
- cp -R source target
The copy command, cp is used to copy files or a directory full of files.
- To copy a file you specify the name of an existing file as the
``source'' and you specify a legal filename as the ``target''.
- If you want the new copy of the file to be in a different
directory, then you can specify a path_to_an_existing directory
as the ``target''.
- To copy a directory full of files you use the -R argument. As
is often the case, the ``-R'' stands for ``recursive''.
@: cp -R directory1 directory2
The above command will copy directory1 and all the files and
subdirectories contained therein into directory2. If directory2
doesn't exist, it will be created, if it does exist, then this command
will create a subdirectory of directory2 with the same name as
directory1 and containing copies of all of the files and
directories in directory1.
- rm file-name or regular expression The
rm command is used to remove, or erase files. Here again,
regular expressions can be very useful - and quite dangerous.
rm is often aliased4 to rm -i, so that it asks you to
verify that you really want to remove a file. If you get tired
of this safety feature, use \rm
instead.
Note: rm will accept a regular expression as an
argument. The simplest regular expression is ``*'' which stands
for everything in the present working directory. So be careful.
To remove an entire directory and and all the files and
subdirectories in it, you use the -r argument and the
leading back slash, ``:
@: -rf directory1
the above command will remove directory1 and all of the files and
subdirectories within it, the -f argument ensures that rm will
not ask for permission with each file. -rf * is a
VERY dangerous command. If you find yourself typing it make
sure you are not drunk.
- chmod aogu +/- rwx filename-or-directory The
``change mode'' command is used to modify the permissions
(or mode) of a file or directory. Permissions are the
characteristic of a file or directory which determine who has
what type of access to it. All files and all directories have
permissions, only the owner of the file/ directory is permitted
to change modes.
The first argument is a string of characters that grant (+) or
revoke (-) permission to read(r) write(w) or execute (x) the file
or directory. The letters aogu indicate who is to receive or lose
the given permission. u=user, g=group, o=other, and a=all. Thus
to revoke write permission to all users you would type:
chmod a-w filename
To grant permission to yourself and the group to write and
execute a file you would type:
chmod ug+wx filename
- ln -s real-file-name artificial-file-name
The ``link'' command creates an alternative name for an
existing file or directory. This is particularly useful when
using data sets that you keep in /data/commons (as you
should). Rather than typing /data/commons/userid/datafile to
reference your data, a symlink would allow you to type something
much shorter.
ln -s /data/commons/userid DATA
would create a link in your current directory called
``DATA''. But DATA is really just a secret back way to
/data/commons/userid. typing ls DATA for example is the
equivalent of typing ls /data/commons/userid.
It would be a good idea to create the above link right now. Use
the mkdir to create a new directory in /data/commons
called your userid. Then create a link in your home directory so
that you can start storing and accessing huge data sets right
away.
- mv file-name new-file-name The ``move''
command changes the name or location within the filesystem of a file
or a directory.
- less file-name Variant of the more command -
less is used to scroll through a file on the screen. While
displaying a file,
scrolls one additional line;
scrolls one additional screen full;
scrolls
backwards,
quits,
searches forward for
``word'',
searches backward for ``word''.
- gtklp filename launches a gui application which
allows you to print filename to any Demography of
Sociology printer command prints a file to the named
printer. Most of the time we print from with applications so this
command is not so frequently used anymore. Also since it launches
a gui application, it is not strictly speaking a Linux
command. For purists and dinosaurs the Linux command in
lpr.
Demography and Sociology Department Printers
Printer |
Location |
Type |
age |
Basement Lab |
HP Laserjet 4015 postscript monochrom duplex 1200dpi |
parity |
Room 101b |
HP 4050n postscript monochrom 600dpi |
class |
2224 2nd floor |
HP 4100n postscript 600dpi |
cohort |
2232 2nd floor |
HP 4200n postscript 600dpi |
reproduction |
2232 1st floor |
Canon Image Runner Advanced C5235 |
status |
Library |
HP 4200n postscript monochrome 600dpi |
ses |
Room 101B |
Ricoh postscript color duplex 1200dpi (administrative use ...mostly) |
Barrows477 |
Rm 477 Barrows |
Xerox phaser |
Barrows483 |
Rm 483 Barrows |
Xerox phaser |
- pwd ``present working directory'' tells you where you
are, that is, it tells you which directory the shell thinks is the
current directory.
- du directory The ``disk use''
command is designed to tell you how much disk space each
directory is consuming. It's main use, however, is simply to
display the directory structure.
- man -k key-word | command-name
The "manual" command is used to display manual pages on your
screen. To say that man pages are not particularly easy to read
is is an understatement of almost biblical magnitude. But they
are very handy for refreshing your memory or searching for
something very specialized.
The man program puts the contents of the man page in a ``less''
process, see item 9 for a description of how to navigate
in less.
In this century a very good source of information on Unix is
google. The web knows all about Unix and while there are lots of
different distributions, command line tools in particular are
nearly identical in all distributions including Solaris, Hpux,
every flavor Linux, BSD, and even Apple's OS X (which by the way
is Unix too).
- ssh -l userid hostname ``ssh'' stands for
``secure shell'' it is really a separate application
but it behaves like a shell command
and is really useful so it is included here. If you type
ssh galton at the unix prompt, (and then your password when
prompted) a remote shell will open on an entirely different
machine from the one you are sitting in front of.
The new remote shell on galton will have a prompt like:
[userid@galton ~]$
indicating that the commands that
you type will be executed the machine galton which
happens to live in the basement of 2224 Piedmont. Happily the
new shell will see the same filesystem and understand the same
Unix commands.
The reason for ssh'ing to galton is that it is much more powerful
than any of the workstations.
NOTE: when using ssh or ssh-like programs on machines outside of
the Demography Lab, you will need to specify both your userid
(with the -l flag) and the fully qualified hostname e.g.
galton.demog.berkeley.edu5.
NOTE even more urgently: Since we started running freeNX
servers, ssh'ing from outside of the department instantly became
anachronistic. FreeNX provides a much better way of connecting to
the Demography network if your goal is to do
science. http://lab.demog.berkeley.edu/LabWiki is the place
to go to find out about freeNX.
- exit or logout closes the current Unix window,
and logs you off - if the current window is the console window.
In addition to the key combinations and commands discussed, Unix also
supports several characters with special meanings to the shell. Below
is a list some of the more common ones:
- *
- The asterisk or ``star'' character is used in regular
expressions (See item 1). When the shell sees a * by
itself as in @: ls * it replaces * with a list of all the
files and subdirectories in the current directory. @: ls *
tells the shell to run the ls command on each and every file and
subdirectory in the current directory. So where @: ls will
show files and subdirectories @: ls * will list the files
that live in subdirectories of the current directory as
well.
- &
- The ampersand tells the shell to run the process in the
``background''. When a process is launched in the background, the
xterm (See 2) immediately returns with a prompt. When you
run a process in the foreground (the usual case) the prompt comes
back only when the process exits.
NOTE it only makes sense to run programs in the background
if the program spawns a new window. So emacs, Stata,
userfirefox, or oowriter are all fine running in the
background. The 12 most important Unix commands are not. They all
write their responses to the terminal window. If you put them in the
background they cannot do this.
REALLY important: R should not be run with the & for
the same reason: it runs in the window from which it was launched.
This will all make sense after the first week or two of 213.
To bring a backgrounded program to the foreground, type
@: fg %n
where %n is the percent sign followed by a
number indicating which backgrounded process you want to
foreground. You only need to enter the %n if you have more than one
process running in the background. Type
@: jobs
to get a list
of backgrounded processes associated with the current xterm.
- the tilde character is interpreted by the shell to mean
``home directory'' by itself, it means your home directory,
if it is followed by a username as in carlm it refers to that
user's home directory. The can be used in complicated
pathnames such as
carlm/public_html/213F97/welcome.html.
For it to make sense, the must be the first character (and
perhaps the only character) of a pathname.
- The ``pipe'' is used to send the standard output of one
process into the standard input of another. For example, if you
wanted to know the number of lines in every data file in the current
directory you might type: @: ls *.data | wc -l . The ls
*.data produces a list of files in the current directory that end
in ``.data'', the then feeds this list to the word
count command ``wc''. The -l argument tells
wc to only report the number of lines. This example assumes
that you have named all of your data files
somethingorother.data.
- The right angle bracket (or greater than sign) is used to
send the output of a process into a file. @: ls > file.list
would produce a file called file.list containing (surprise) a list
of files. Use double angle brackets to append a process's output to
an existing file.
@: ls /public_html » file.list
would add
the names of the file's in your public_html directory.
The ``twelve'' most important Unix commands
This document was generated using the
LaTeX2HTML translator Version 2008 (1.71)
Copyright © 1993, 1994, 1995, 1996,
Nikos Drakos,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999,
Ross Moore,
Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -split 0 -local_icons 12important
The translation was initiated by Carl Mason on 2015-08-31
Footnotes
- ... directory1
- directories are also called
"folders"
- ... 13862
- See the statement of compliance
that you signed before we gave you an account
- ... spreadsheets3
- It is of
course possible to launch most applications via the menu system or
by clicking on corresponding data files in the file manager, but the
command line is often faster
- ... CLASS="textbf">aliased4
- as you might expect, ``alias'' is a shell feature that allows you to create new names for commands. It is possible and common to use this feature so that when you type rm the shell substitutes rm -i. The -i argument causes rm to ask for verification before it removes a file
- ...galton.demog.berkeley.edu5
- surprise - from outside
the department you will
probably end up on refugee rather than galton if you type
this. The reason is that from non Demography Lab machines, all
.demog.berkeley.edu hostnames ``resolve'' to the outside interface of
our firewall. You can ssh to galton from whichever host you wind
up on
Carl Mason
2015-08-31