WALKER'S WEBPAGE ON CLUSTER OPERATION
(Updated F2005 by Johnnie Baker and Wittaya Chanatamas)
(Update F2006 by Johnnie Baker and Roy Heath)

http://www.cs.kent.edu/~walker/classes/pdc.f03/misc/cluster.operation	

This file explains how to use the departmental cluster.  

•	To use the departmental cluster, log in to "csc-hnode.cluster.cs.kent.edu", using your departmental account username and password. (This can be done using SSH. I set the port number to 22.) 

•	Next create at least one host file. For example, you could define a “clusterhosts” file that contains the lines:

csc-hnode.cluster.cs.kent.edu
csc-node01.cluster.cs.kent.edu
csc-node02.cluster.cs.kent.edu
csc-node03.cluster.cs.kent.edu
csc-node04.cluster.cs.kent.edu
csc-node05.cluster.cs.kent.edu
csc-node06.cluster.cs.kent.edu
csc-node07.cluster.cs.kent.edu
		...
csc-node18.cluster.cs.kent.edu

•	When all of the nodes are working, there should be 18 in total. 

•	The first node, csc-hnode, is the server, control unit, or manager.  Also known as the 'head' node. 

•	You may want to create more than one of these type files, so that you can switch to another node if one of the nodes is down. The control unit (csc-hnode) should always be included.  For example, “clusterhosts0_3” could contain the lines 

csc-hnode.cluster.cs.kent.edu
csc-node01.cluster.cs.kent.edu
csc-node02.cluster.cs.kent.edu
csc-node03.cluster.cs.kent.edu
			
•	Create a file containing the following simple “Hello World” program, written in C. Here this file is named demo.c

#include <stdio.h>
#include "mpi.h"

int main(int argc, char **argv)
{
        int myid;           /* my rank  */
        int numprocs;       /* the cluster size */

        MPI_Init(&argc, &argv);
        MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
        MPI_Comm_rank(MPI_COMM_WORLD,&myid);

            printf("Hello World from %d of %d\n", myid, numprocs);

            MPI_Finalize();

        return 0;
}

•	Start LAM/MPI on all nodes listed in your “obrhost0-3” file using the following command:

[csc-hnode.cluster.cs.kent.edu]-> lamboot -v clusterhosts0_3

•	You should obtain the following type of response:

LAM 7.1.2/MPI 2 C++/ROMIO - Indiana University

n-1<4217> ssi:boot:base:linear: booting n0 (csc-hnode.cluster.cs.kent.edu)
n-1<4217> ssi:boot:base:linear: booting n1 (csc-node01.cluster.cs.kent.edu)
n-1<4217> ssi:boot:base:linear: booting n2 (csc-node02.cluster.cs.kent.edu)
n-1<4217> ssi:boot:base:linear: booting n3 (csc-node03.cluster.cs.kent.edu)
n-1<4217> ssi:boot:base:linear: finished

•	The program “demo.c” is compiled as follows:

 [csc-hnode.cluster.cs.kent.edu]-> mpicc -o demo demo.c

•	This should create an executable file in your directory under the title of demo

•	Finally, you can run this executable file on all the nodes in your obrhost file using mpirun and  N as follows:

[csc-hnode.cluster.cs.kent.edu]-> mpirun N ~/demo

•	The filename “~/demo” above assumes that “demo” is in your top directory. If the file “demo” is in the directory “mpi”, then you should use “~/mpi/demo” instead.
•	You should receive a response of the following type:

Hello World from 0 of 4
Hello World from 1 of 4
Hello World from 2 of 4
Hello World from 3 of 4

•	The program could be run on the first three nodes using the command:

[csc-hnode.cluster.cs.kent.edu]-> mpirun n0-2 ~/demo

•	This should produce a response similar to the following:

Hello World from 0 of 3
Hello World from 1 of 3
Hello World from 2 of 3

•	When you are done with MPI, please don’t forget to wipe out all of your LAM/MPI processes. Failure to do so may cause the cluster to go down if the system is overloaded. To turn off the notes in the file obrhosts0-3, use the following command:

[csc-hnode.cluster.cs.kent.edu]-> wipe -v clusterhosts0_3

•	You should receive the following type of response:

LAM 7.1.2/MPI 2 C++/ROMIO - Indiana University

n-1<4379> ssi:boot:base:linear: booting n0 (csc-hnode.cluster.cs.kent.edu)
n-1<4379> ssi:boot:base:linear: booting n1 (csc-node01.cluster.cs.kent.edu)
n-1<4379> ssi:boot:base:linear: booting n2 (csc-node02.cluster.cs.kent.edu)
n-1<4379> ssi:boot:base:linear: booting n3 (csc-node03.cluster.cs.kent.edu)
n-1<4379> ssi:boot:base:linear: finished

•	DON'T FORGET TO DO THIS!  Do NOT leave mpi processes running on these machines after you log out!!

•	Some final notes on using the cluster:

•	Stepping back a bit and considering the 19 machines as a whole, please consider csc-hnode.cluster.cs.kent.edu to be the "server" or "control unit" and csc-node01.cluster.cs.kent.edu through csc-node18.cluster.cs.kent.edu to be the "nodes" available for work.

•	Log onto csc-hnode.cluster.cs.kent.edu, edit files there if you want, and then use it to invoke MPI using mpirun.  However, unless you really need all 19 nodes, please don't use csc-hnode.cluster.cs.kent.edu to run programs -- run your programs on csc-node01  through csc-node18.  For example, use this command: 

mpirun n1-18 demo

to run "demo" on nodes 1 through 18, assuming you are using the clusterhosts file with csc-hnode as node 0.

•	The official LAM user manual is located at /usr/share/doc/lam-7.1.2/user.pdf on the head node of the cluster.  The LAM website has a set of tutorials as well which is located at: http://www.lam-mpi.org/tutorials/

•	Some notes concerning departmental machines with MPI 

•	There is a version of lam mpi available on department machines as well as on the clusters. The architecture of the linux machines and the HP machines is different, so you can't run the same binary programs on both sets of machines.  You need to compile separate versions for each architecture.

•	While you can use these machines to develop your program, your final run for your homework should be on the departmental cluster. This may be particularly useful when you are unable to access the departmental cluster.  

•	The machines with LAM include b1 (HP), b4 (HP),  neptune (linux), and poseidon (linux).  

•	The MPICH2 version of MPI is also available on some of the department machines but is not yet installed and setup on the departmental cluster. 

•	The general LAM web site can be found at:

        http://www.lam-mpi.org/

•	The latest release of the LAM software can be found at:

        http://www.lam-mpi.org/download/

•	Posts to the LAM mailing list are archived on the web:

        http://www.lam-mpi.org/MailArchives/lam/

•	Mirrors of all of these sites are available.  Consult:

        http://www.lam-mpi.org/mirrors/