Optimizing your mac environment for computing at the Demography Lab

Abstract:

A HowTo on setting up a great R-centric computing environment. The key components of which are: R, LaTeX, Emacs. All the tools necessary for moving projects back and forth between your personal computer and the Demography Lab servers are installed by default. With this setup you can run jobs on your personal computer and on the DemographyLab servers with nearly identical tools. You can move projects back and forth between your PC and the Lab servers and you can run jobs on the Lab servers as if they were running on your local PC.

STOP you might not need to do any of the things listed below. You can instead use freeNX to connect to the server from your personal computer in a way that is far simpler to setup and, for most people, much easier to use. See http://lab.demog.berkeley.edu/LabWiki for instructions on using FreeNX. The instructions below may be of slightly more than historical interest to those who like to install things and who want to maintain an independent ability to do science...from their caves..after the ApoCALypse.

What you need

The Mac OS is essentially a Unix application, as a consequence, inter operating with Unix or Linux servers is quite straight forward. Most of what you need is already installed, the few bits that are not are free and open-source.

You will need the following parts:

R: The best statistical computing environment for Demographers and other thoughtful people.
Emacs: The editor that you will soon learn to love. (There is a version already on the mac, but we'll download a fancier one)
ESS: An add-in for Emacs that understands R and your innermost thoughts. (This will come with the Emacs version that we are going to install)
Texlive (LaTeX): A document preparation system that works well for scientists. (We won't use this until November so you can safely wait to install it until later)
rsync: A program for synchronizing directory trees across machines. (this is already on your mac)
ssh: A secure shell program which allows you to run a terminal window on remote Unix machine or mac machine. (this is already on your mac)
X11 server: A windowing system that allows you to control remotely running applications as though they were running locally. (this is already on your mac) NOTE: A FreeNX client offers a terrific (faster) alternative to using X11. See http://lab.demog.berkeley.edu/LabWiki for more on FreeNX.

Emacs

Emacs is both and editor and a religion. You will get to know it in 213. There are many versions available for the mac, the most convenient one is Aqua Emacs. Download it from http://aquamacs.org/. Then drag it into Applications in the usual way.

R

The most important application is of course R. The latest version can be found and downloaded at http://cran.r-project.org/bin/macosx. Choose the latest version (currently 2.13.1); download it and run the installation program.

LaTeX

LaTeX is a document preparation system that is particularly well suited for scientists and for writers of dissertations. You'll learn about it in 213. We won't look at LaTeX until November, so you don't need to download this right now. If you're on a roll (compared to your windows encumbered colleagues) you might go for it. It is a large (1.8GB) download.

You can download the mac version from http://www.tug.org/mactex/2011/

Test everything

Once everything is installed, let's make sure that it all works as I promised it would.

synchronizing directory trees (1).

Let's synchronize your 213 directory so that it is the same in your Demography Lab account as it is on your personal computer. Launch a terminal window. The terminal window application live in Applications $\rightarrow$ Utilities. You probably want to drag it to the doc so you can launch it more easily.

The terminal window application in the mac is very similar to the one that you run on the Linux workstations. Once you have launched a terminal window notice that you are in your home directory, and then run this command OBVIOUSLY changing ``userid'' into your own userid on the Demography Lab system.

@: $\textgreater$ rsync -uva -e ssh userid@coale.demog.berkeley.edu:213 .

NOTE the significant dot at the end of the above command.

Run a moderate (actually tiny) R session on your personal computer

Once you have downloaded the 213 directory you should be able to do the following:

Launch Emacs; find and open the 213/Week1/213ex1.r file.
Launch R in the usual way (from Emacs) and run the commands.
Make a trivial change to the 213ex1.r buffer; save it; quit R and exit Emacs.

synchronizing directory trees (2)

The procedure for synchronizing your Demography Lab copy of your 213 directory tree with what's on your personal computer is very similar BUT NOT identical to the command you ran to bring a copy of the 213 directory from the server to your personal computer. Here is the command to move the changes that you made on your PC to the 213 directory on the server (in other words to synchronize the two copies). Do this in a terminal window application from your home directory.

@: $\textgreater$ rsync -uva -delete -e ssh ./213 userid@coale.demog.berkeley.edu:

IMPORTANT: rsync copies the contents of or the changes in the source directory onto the target directory. Which is the source and which the target is determined by order in which they appear in the command. In the above command, ./213 is the source and userid@coale.demog.berkeley.edu: (which translates to userid's home directory on coale.demog.berkeley.edu) is the target.

rsync will never change the source directory so as long as you get the source and target business right, you cannot do irreparable harm. BUT be careful with rsync, it is like a circular saw with the safety equipment removed: powerful but also dangerous.

@: $\textgreater$ rsync -uva -delete -e ssh ./213/ userid@coale.demog.edu

NOTE also the -delete argument. In the present case this will not matter, but it tells rsync to delete files that exist in the target directory but not in the source. It is wise to use this flag because otherwise your life will fill up with useless files that you thought you had deleted already.

Although the rsync man page is very difficult to understand, it is good idea to take a look at it. (man rsync on your linux or mac machine). Since rsync can cause a huge amount of damage when used incorrectly, it is worth taking some time to figure it out. Here is one of many short tutorials: http://www.unixtutorial.org/2008/09/how-to-synchronize-directories-with-rsync/

Running R on the server

Now let's connect to the server in such a way that you can run R inside Emacs on the server but have it behave as though you were running it locally on your PC.

Launch your X11. On most modern macs this is not necessary, X11 will launch itself when it needs to. So don't worry about it until but keep this in mind in case something goes wrong.

In a terminal window application type:

@: $\textgreater$ ssh -X userid@coale.demog.berkeley.edu

OBVIOUSLY replace ``userid'' with your userid on the demography lab network. Answer yes to the obscure warning about not being able to establish the ``authenticity of host'' bla bla bla.

then type your password when prompted.

The ``-X'' argument in the command above tells ssh to ``tunnel X11''. To make sure that this worked type:

@: $\textgreater$ echo $DISPLAY

If that results in localhost:10.0 (the number is not important). Then continue, otherwise, get help.

Assuming that you have a terminal window with a shell running on coale, and which is tunneling X11, then do the following:

@: $\textgreater$ cd 213/Week1

@: $\textgreater$ emacs 213ex1.r &

In the resulting emacs window, verify that the trivial change that you made to this file is indeed present.

If you want to, you can run all the commands in the file, but that's more boring than it's worth. To test the important feature of the X11 server, create graph in R