Help:Disk Usage

From LabWiki
Jump to: navigation, search

Places that you can put stuff

There are 3 different kinds of places where you can store your stuff:

  1. Your Home Directory is where you land when you login. The files you see on your Desktop are in subdirectory of your home directory. Home directories are appropriate for storing high value files such as programs and documents that you create. You can put a GB of stuff in your home directory before the system starts to threaten you. The stuff in your home directory is backed up daily and retained for a long time. See Help:Backups for details.
  2. The COMMONS directory (which is a symbolic link to /data/commons/Your-name) is also backed daily but backups are not retained for quite so long. It's a good place for larger less valuable files such as those datasets that you downloaded. Big files tend to be easy to reproduce and expensive to backup, so please put those in COMMONS.
  3. /72hours is a huge partition that is local to each machine. As the name implies, things that are not used, disappear from /72hours in 72 hours 60 days at leaste. Since the partitions are local to each machine, they are fast. This is the place for intermediate files of downloads that you really don't need to keep. The more you can use /72hours, the better. It's for stuff that doesn't need to be backed up... 'cause /72hours is not backed up.

The Quota Police

Home Directories and COMMONS each have impersonally enforced quotas or limits on how much stuff you can put there. /72hours is only limited by the size of the disk. When you exceed your disk "quota" you will start getting vaguely threatening, barely comprehensible email from the "Demog Quota Police". What these messages are trying to say is:

"Please rearrange, backup and/or delete stuff so that other scientists can use the space. Home directories have a 7 day grace period, COMMONS has a 30 day grace period. When these generous grace periods are exhausted, you will not be able to write to the respective disk -- until you clear out enough stuff to be under quota once again."

Bring out you dead

Cromagnon file management, wherein you throw the "bones" to the back of the cave and when the cave fills up...you get a new cave is practiced by many good scientists. You can do it here too if you like -- except the part about getting a new "cave". While disk space is cheap, backing it all up is expensive. consequently, when you run out of quota space, you will need to dispose of those bones.

Compressing files

You can compress individual using the gzip command

gzip filename

You can compress entire directories with the tar command:

tar -czvf tarfilename.tgz ./path/to/directory

Of course this only saves space if you then delete the original directory

/bin/rm -rf ./path/to/directory
Be more than a little bit careful with this as there is no "undelete" other than restoring from backups

Finding and deleting useless stuff

Heads UP: moving stuff to Trash does not change your quota situation until you empty the trash

A helpful local tool for finding files and directories that have outlived their usefulness is findtrash

After cd'ing to the top level directory of interest (generally this is your home directory but it could be COMMONS or somewhere else)...

findtrash

The result will be a list of your ten biggest directories and the ten biggest files along with the date of last change.