Alex and Matthew playing with python parallel programming

Ipython1 - http://ipython.scipy.org/moin/IPython1 Docs therefore - http://ipython.scipy.org/doc/ipython1/html

Then we copied the sshx script from the ipython1 distro into /imaging/local/linux/bin

Shell setup

The engines use the ~/.bashrc to initialize environment variables for the shell (this does not happen by default because sh is used, which does not read startup scripts). The problem being that Alex is set up with the usual default tcsh, which uses ~/.cshrc.

Accordingly, we added these lines to ~/bashrc:

# FSL stuff - if we are using FSL
export FSLDIR=/imaging/local/linux/bin/fsl-4.0.1
. ${FSLDIR}/etc/fslconf/fsl.sh
export PATH=${FSLDIR}/bin:$PATH

cbu_img_sys=/imaging/local/linux
# If on linux, set stuff for libraries, and python
if [ "`/bin/uname`" = "Linux" ]; then
    export PATH=${cbu_img_sys}/bin:$PATH
    export LD_LIBRARY_PATH=${cbu_img_sys}/lib
fi

Cluster setup

We made a tiny script to test with:

   1 controller = dict(host='l42',
   2                   engine_port=None, # default is 10105
   3                   control_port=None,
   4                   )
   5 
   6 # keys are hostnames, values are the number of engines on that host
   7 engines = dict(l31=2,
   8                l32=2,
   9                l33=2,
  10                l34=2,
  11                )

Then we ran ipcluster -f small_cluster.py

and got this:

Starting controller:
Starting engines:   
Sarting 2 engines on l34
Sarting 2 engines on l32
Sarting 2 engines on l33
Sarting 2 engines on l31

Your cluster is up and running.

For interactive use, you can make a MultiEngineClient with:

from ipython1.kernel import client
mec = client.MultiEngineClient(('l42',10105))

You can then cleanly stop the cluster from IPython using:

mec.kill(controller=True)