= Running AMUSE scripts on the PARA cluster =

AMUSE can be installed and run on the PARA cluster at the Leiden University. All prerequisite software is 
already installed on the cluster, so you only need to download and build AMUSE (see [http://www.amusecode.org/doc/install/howto-install-AMUSE.html Install Amuse])

{{{
#!sh
> svn co http://www.amusecode.org/svn/trunk amuse-svn
> cd amuse-svn
> ./configure
> make
}}}

Next you need to create a shell script to start your python script (see [http://www.strw.leidenuniv.nl/local/computers/it_local/para/torque.php Torque implementation at Leiden Observatory] for a complete description of all options)

{{{
#!sh
#!/bin/tcsh -f
#PBS -l nodes=4
#
setenv PYTHONPATH path/to/amuse/src:${PYTHONPATH}

cd path/to/the/python/script

mpiexec -n 1 python your_script.py

}}}

  You need to specify {{{-n 1}}} to mpiexec. This will start one version of the python script. In your script you 
  can then start the codes (which will run on different nodes of the para cluster).

The number of nodes reserved for your script (with {{{PBS -l nodes=N}}}) must be the number of instances of the codes you will start in the script plus one for the script itself. So if you're script contains this line:

{{{
#!python
code = Athena(number_of_workers=4)
}}}

You need to reserve 5 nodes.

'''WATCH OUT:''' Amuse by default will start a code using a redirection script and this script needs arguments. Unfortunately, the TORQUE/MPICH2 combination on the para cluster  seems to have an error in spawning codes with arguments. To fix, create a amuserc file in your working directory (same directory as the cd earlier), and set this content.

{{{
[channel]
redirection=none
}}}



After you have written you're start script you can submit your script with (we assume, you've named the start script 
{{{start_script.sh}}})

{{{
#!sh
> qsub start_script.sh
}}}