The hpc is a Red Hat LSF system, and its LSF-related commands include bsub
, bhist
, bjobs
, bqueues
, bpeek
, and bkill
. The Red Hat version is Enterprise AS release 4 (Nahant Update 5), and the shell is tcsh. More info about the cluster is here: http://www.ncsu.edu/itd/hpc/Documents/BladeCenter/GettingStartedbc.php
Basic LSF commands (this webpage is for a different cluster): http://kipac-prod.stanford.edu/collab/computing/docs/lsfbasics
To change from tcsh to bash:
chsh -s /bin/bash
Note that this could cause problems:
- The
add
scripts only work with tcsh. - The tcsh shell is already configured with a bunch of paths and environment variables required for LSF.
The shell change does not take effect until you log out and back in again. Furthermore, if you log in to the generic login address and it sends you to a different login node, then you will need to chsh again and re-login.
This web page has some information about which startup file is sourced at which time for each shell. http://hayne.net/MacDev/Notes/unixFAQ.html#loginShell
I think that the same startup files are used regardless of the specific login node, unlike the node-specific conditions encountered for chsh.
If you want to leave a terminal open while you work on something else, or if you want to leave a terminal open so that you can work on it from a different login, then gnu screen is useful. It has been enhanced for Ubuntu as byobu and can be installed on the hpc by extracting http://people.ubuntu.com/~kirkland/byobu/byobu.tar.gz to your home directory.
The program called bsub is the primary way to access the resources of an LSF cluster. The example bsub script can be run as follows:
$ bsub < hello.bsub
Because I only have enough disk quota in /home
for a few dotfiles, installed programs such as git and python are prefixed to /brc_share/brc/username/install
.
Download virtualenv and create a virtual environment:
$ python virtualenv.py /brc_share/brc/username/myenv
This also installs setuptools to the virtual environment, including the easy_install script which can be used to install the better package manager pip. Note that the virtualenv.py
script can be used without actually installing virtualenv.
The package manager pip can be installed using the command:
$ /brc_share/brc/username/myenv/bin/easy_install pip
Additional python packages and modules such as argparse can be installed with pip using commands like this:
$ /path/to/myenv/bin/pip install -E /path/to/myenv argparse
The bash shell can make use of the activate
and deactivate
commands provided by virtualenv. Activate a virtual environment as follows:
$ . /path/to/myenv/bin/activate
Then install pip:
(myenv)$ easy_install pip
Then install extra python packages:
(myenv)$ pip install argparse
Leave the virtual python environment and go back to using the system python environment as follows:
(myenv)$ deactivate