much more efficient... eventually.
To operate with the command line, you will need to know the 12 most important Unix commands described in Section 5. To enjoy it you will also need to know a few tricks that are also covered in this document.
You don't need to know much about Unix in order to start doing Science, but it would not hurt to learn more. In your copious free time, check out some of the Unix primers on the web. Ask google something like ``Unix beginner'' to find more resources than you could possibly want.
Note that since the Mac OS is simply a Unix application, nearly everything in this document works the same way in a mac. On a mac, the terminal window application is under Applications/Utilities.
shell, you must open a terminal window (also known an xterm window). A terminal window can be launched from: [Application][Accessories][Terminal].
It should look something like Figure 1. Notice that the window features a menu bar, unfortunately the menu bar it is both useless and misleading. Make the menu bar disappear by pressing the and clearing the ``show Menubar'' check-box.
Now aside from the title bar at the top, the only words in the
terminal window should be the Unix prompt. The purpose of the
Unix prompt is to indicate that the shell is ready to accept
commands. It also contains useful information.
In Figure 1, the prompt is is
indicating the user, carlm, the machine, twins and the
current directory which is indicated by the
~. In this and
other documents, the Unix prompt will look like this: @: . In the real
Unix prompt, the symbol
~ is a special character whose
meaning is "home directory".
~/Dissertation means a file or
directory called "Dissertation" which is located within your home
directory. In my case this would be
translates to a file (or possibly a directory) called insight
in a directory1 called Brilliant in Ken Wachter's home
directory, or /hdir/0/wachter/Brilliant/insight. More about
home directories can be found in Section 2.
If you are in no particular hurry to finish your dissertation, you can modify a large number of colors and beeps and other important features of the terminal window. [Edit Current Profile] is the place to start wasting time.
If you have already wasted time on this sort of thing and are thus old enough to find the default font a bit small, a useful trick with terminal windows (and browser windows too) is the + + to increase and + to decrease the size of the typeface.
Whenever you login to a machine on the Demography network, your initial present working directory - the location within the filesystem in which applications will begin looking for the files that you specify - is your home directory. Every user has exactly one home directory.
In a multiuser system such as the Demography Lab, your home directory is one of a huge number of interconnected directories that form a single unified filesystem. The magic of the filesystem is that even though the various files and directories of which it is composed are ``physically''/footnoteor electromagnetically present on various different machines all over the network, to us users, the whole thing appears to be one single thing and that thing looks and feels the same no matter which Demography Lab machine we happen to be using at the moment.
An upside down tree makes a pretty good metaphor for the filesystem. Such a ``tree'' is shown in Figure 2. At the top of the figure is a directory called ``/'' which is the ``root'' of the filesystem. Every file and directory in the filesystem can be uniquely specified by a filepath that begins with root. For example, the file that holds my correspondence with my mother is /hdir/0/carlm/mail/mom.
As you can see in Figure 2 home directories all live in a directory called /hdir/0. Although it is just one of many directories within this giant upside down tree of a filesystem, your home directory is a special place that you will come to know and love and where you will do your very best work. It is the part of the filesystem that you own and the ``place'' where you will find yourself when you first login.
Because the entire filesystem looks the same to all users all the time, it is easy to share data with your colleagues. This is good thing because humanity benefits when scientists collaborate. But unfortunately scientists can occasionally turn out to be creeps so sharing a filesystem is a little scary as well.
The ``solution'' to the creep problem is to not keep sensitive information on Demography computers. You have already promised not to keep data covered by SB 13862. It goes without saying that files that can tie you to illegal activities are also a no-no. There are however, a few files that belong on the network and yet where privacy is an issue (e.g. email). For those files, managing who may read and/or change them requires understanding the mode and ownership of files. Each file and directory has an owner and the owner can determine who is allowed to read, write and/or execute each file. See the chmod command below for how to change the various file modes or permissions. The chmod command is described in 6.
shell is the program that runs in each terminal window. It waits for you to type something at the Unix prompt, @: , and then does what it thinks you meant. The shell we use here is called bash (pronounced ``bash''). Bash is one of several modified versions of the original sh (pronounced ``s-h'') shell.
The most important thing that the shell does for you is to let you give commands to the computer. These include the 12 most important Unix commands (Section 5) as well as commands to launch applications like R, Stata, word processors or spreadsheets3. The shell does several other things for you some are essential, some enhance efficiency and others are just cool.
+ does this.
Below is a list of some handy/essential key combinations that the shell recognizes:
completion'' and ``scripting''.
and edit any command that you have previously issued. To make the previous command appear at the @: hit + or equivalently the up arrow key. To see even more previous stuff type + more times. + or equivalenlty the down arrow will make the next command appear - obviously, this makes no sense unless you have typed + at least once.
You can operate on a recalled command using several standard emacs editing keys:
You can also use the left arrow and right arrow to move about within a recalled line. The and keys do what you would expect.
key anytime while constructing a command, the shell will do it's best to figure out what you are planning to type next. If you are typing a command it will try to find a command that starts out with what you have already typed. If you are typing the name of a file the shell will try to complete if for you. If what you have typed does not uniquely determine a command or filename, the shell will beep at you and provide a list of possible completions. You can then type a few more characters and hit again.
could have entered at the keyboard, but typed into a file instead. You can then set the file's execute bit (See Most Important Command number 6) and execute that file - perhaps now, perhaps later. You will need to use an editor such as emacs to create that shell script. Knowing how to use emacs can save you lots of time and hair loss - particularly if many of the commands you are typing are quite similar.
Scripts are also very useful for people who like the idea of being able to reproduce results.
1) in lots of colors.
Regular expressions are combinations of symbols that the shell interprets in clever ways. Generally we use regular expressions to specify lists of files or directories on which a command should operate, but they have many other uses as well. A typical use would be to delete from your current working directory all of the .pdf files whose name begins with a vowel:
@: rm [AEIOUaeiou].pdf
regular expressions come up in several of the ``12'' important commands.
For the most part, these commands are for logging on and off; for printing; and for moving files and directories around. Many of these commands functions can be done using a file manager or under emacs, but knowing how to do them from the command line, makes you more efficient, reduces errors and opens the possibility of automating tasks with shell scripts.
@: ls -ltr directory1
produces a list of files sorted so that the most recent ones are at the bottom of the list.
A regular expression is a set of special characters (or ``meta-characters'') that can be used to represent a list of files or more generally, the set of all character strings with certain characteristics. Regular expressions come up in a lot of places and can be very useful when working with datasets that are full text and/or errors. Regular expressions are used with ls to limit the number of files displayed. For example ls X*Y would list only the filenames that start with an X and end with a Y; ls [aeiou]* would show you all the files that begin with a lower case vowel (In the previous example the [ ] are part of the command)
@: mdir -p first/second/third
would create a a directory called third which would be a subdirectory of second which in turn would be a subdirectory of first. The -p argument instructs mkdir to create second and/or first if they do not already exist.
The copy command, cp is used to copy files or a directory full of files.
@: cp -R directory1 directory2
The above command will copy directory1 and all the files and subdirectories contained therein into directory2. If directory2 doesn't exist, it will be created, if it does exist, then this command will create a subdirectory of directory2 with the same name as directory1 and containing copies of all of the files and directories in directory1.
rm is often aliased4 to rm -i, so that it asks you to
verify that you really want to remove a file. If you get tired
of this safety feature, use
Note: rm will accept a regular expression as an argument. The simplest regular expression is ``*'' which stands for everything in the present working directory. So be careful.
To remove an entire directory and and all the files and subdirectories in it, you use the -r argument and the leading back slash, ``:
@: -rf directory1
the above command will remove directory1 and all of the files and subdirectories within it, the -f argument ensures that rm will not ask for permission with each file. -rf * is a VERY dangerous command. If you find yourself typing it make sure you are not drunk.
The first argument is a string of characters that grant (+) or revoke (-) permission to read(r) write(w) or execute (x) the file or directory. The letters aogu indicate who is to receive or lose the given permission. u=user, g=group, o=other, and a=all. Thus to revoke write permission to all users you would type:
chmod a-w filename
To grant permission to yourself and the group to write and execute a file you would type:
chmod ug+wx filename
ln -s /data/commons/userid DATA
would create a link in your current directory called ``DATA''. But DATA is really just a secret back way to /data/commons/userid. typing ls DATA for example is the equivalent of typing ls /data/commons/userid.
It would be a good idea to create the above link right now. Use the mkdir to create a new directory in /data/commons called your userid. Then create a link in your home directory so that you can start storing and accessing huge data sets right away.
Demography and Sociology Department Printers
|age||Basement Lab||HP Laserjet 4015 postscript monochrom duplex 1200dpi|
|parity||Room 101b||HP 4050n postscript monochrom 600dpi|
|class||2224 2nd floor||HP 4100n postscript 600dpi|
|cohort||2232 2nd floor||HP 4200n postscript 600dpi|
|reproduction||2232 1st floor||Canon Image Runner Advanced C5235|
|status||Library||HP 4200n postscript monochrome 600dpi|
|ses||Room 101B||Ricoh postscript color duplex 1200dpi (administrative use ...mostly)|
|Barrows477||Rm 477 Barrows||Xerox phaser|
|Barrows483||Rm 483 Barrows||Xerox phaser|
The man program puts the contents of the man page in a ``less'' process, see item 9 for a description of how to navigate in less.
In this century a very good source of information on Unix is google. The web knows all about Unix and while there are lots of different distributions, command line tools in particular are nearly identical in all distributions including Solaris, Hpux, every flavor Linux, BSD, and even Apple's OS X (which by the way is Unix too).
The new remote shell on galton will have a prompt like:
[userid@galton ~]$ indicating that the commands that
you type will be executed the machine galton which
happens to live in the basement of 2224 Piedmont. Happily the
new shell will see the same filesystem and understand the same
The reason for ssh'ing to galton is that it is much more powerful than any of the workstations.
NOTE: when using ssh or ssh-like programs on machines outside of the Demography Lab, you will need to specify both your userid (with the -l flag) and the fully qualified hostname e.g. galton.demog.berkeley.edu5.
NOTE even more urgently: Since we started running freeNX servers, ssh'ing from outside of the department instantly became anachronistic. FreeNX provides a much better way of connecting to the Demography network if your goal is to do science. http://lab.demog.berkeley.edu/LabWiki is the place to go to find out about freeNX.
NOTE it only makes sense to run programs in the background if the program spawns a new window. So emacs, Stata, userfirefox, or oowriter are all fine running in the background. The 12 most important Unix commands are not. They all write their responses to the terminal window. If you put them in the background they cannot do this.
REALLY important: R should not be run with the & for the same reason: it runs in the window from which it was launched. This will all make sense after the first week or two of 213.
To bring a backgrounded program to the foreground, type
@: fg %n
where %n is the percent sign followed by a number indicating which backgrounded process you want to foreground. You only need to enter the %n if you have more than one process running in the background. Type @: jobs
to get a list of backgrounded processes associated with the current xterm.
would add the names of the file's in your public_html directory.
This document was generated using the LaTeX2HTML translator Version 2008 (1.71)
Copyright © 1993, 1994, 1995, 1996,
Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -split 0 -local_icons 12important
The translation was initiated by Carl Mason on 2015-08-31