Installing parXXL

This library needs GNU-make for installation. Usually on gnu/linux systems this is the default `make'. On other systems you might find it in various place, of which some might be

 /usr/local/bin/make
 /usr/local/gnu/bin/make
 /usr/local/bin/gmake

In the following `make' will refer to GNU make.

Building the library

First, please notice that the ./configure && make && make install does not work with parXXL as usual for other libraries. The library is designed so that a first installation is performed in the par/ directory.

Configuring the library

As a first step you will have to configure parXXL for your architecture. In most cases you would run from the par/ directory

 ./configure --prefix=/FINAL/INSTALL/PATH --with-mpi

If you omit `--prefix=/FINAL/INSTALL/PATH' you will only be able to use parXXL from your compilation directory. If you omit `--with-mpi' you will just be able to run parXXL executables locally on one machine with the `svm' run time and not be able to use MPI.

This will typically take a while and provide you with verbose information about what it encoutered. `configure' also has a --help option that will tell you about the tunable parameters.

If you want to use MPI as the communication library, see below, and MPI is not installed in your default include path but in another location, <mpi-path> say, you will have to tell make this location.

 ./configure --prefix=/FINAL/INSTALL/PATH --with-mpi=<mpi-path>

If you have multiple MPI versions on your system it might be difficult for configure to find out a consistent solution. A work around would be to try to indicate where to find the correct MPI startup script. E.g

 ./configure --prefix=/FINAL/INSTALL/PATH MPIRUN=/usr/bin/orterun --with-mpi

uses the script that typically comes with the OPENMPI implementation of MPI.

Our heuristic to find and classify the MPI implementation are quite basic for the moment. Don't hesitate to contact us with proposals for improvement.

Now issue the command

 make -k configure

This will run through all directories of parXXL and collect the header files in an `include' directory. Note that if you want symbolic links rather than copies of include files you can use

 make -k configure LINK=1

If you experience the following error

 CPU you selected does not support x86-64 instruction set

Then try to unset your ARCH environment variable (or set it to a correct value).

Compiling the parXXL library

If configuration all went fine you are ready to compile the library.

 make -k ASSERT=1

If you are lucky this should take some time and end up in producing libraries in a subdirectory of lib/ named after your architecture (libparcpp.so, libparsys.so, ...) Refer to Tuning parXXL and parXXL executables for parameters (like the above `ASSERT=1') that might help you to tune parXXL.

If you have a parallel system with several processors or cores you might want to use all of these capacities to speed up the build process. Therefore you split the build in two

 nice make -k -j NUMBER ASSERT=1 targets
 make -k ASSERT=1

where NUMBER is the number of parallel compile threads that you want to perform. Be careful not to leave that number out since make will then run completely parallel starting all compilations simultaneously. Usually this will either be very slow (because the jobs don't fit into memory) or might even crash you machine.

Installing parXXL in its destination directory

Now that the library is successfully compiled you may install it in the final location that you defined at the beginning with the `--prefix' argument to configure.

 make install

Note that depending on that target location you might need root priviledges to copy the files there. An easy way to ensure that (if you are allowed to do that) is using `sudo'

 make install

Note that you can run the installation for different architectures using the same main parXXL directory (see Running parXXL below).

Installation if configure does'nt work

If configure fails for some reason you should first try to give it arguments that help it to better find some tools. Most important are CC, the C compiler, CXX, the C++ compiler and MPIRUN, the launch executable for MPI. E.g

 ./configure --with-mpi=<mpi-path> MPIRUN=/usr/chere/lib/mpixe CC=/opt/grumble/bin/zaeh CXX=/opt/grumble/bin/weniger 

Observe that these settings are given to configure via its command line and not by placing them in front of the ./configure and thus propagating them through the process environment.

If all that fails, you may drop back to our old installation procedure. First chose an appropriate Makefile-inc.something in the subdirectory <par-path>/arch. Basically there are two choices, either to use MPI (in some version) or to use communication via the shared virtual memory of POSIX threads, SVM.

Then make a softlink here in this directory named `Makefile-inc' for example

 ln -s arch/Makefile-inc.linux-lam Makefile-inc

Probably you also will have to adjust the corresponding makefiles to your needs.

Then issue the following command twice

 make -k PARXXL=<par-path> MPIROOT=<mpi-path>

to compile the library.

Compile and link parXXL executables

After a successful compile of the library you can go to the test algorithm in the directory `tests+benchs'. Inside any of its subdirectories these should now compile with

 make -k PARXXL=/FINAL/INSTALL/PATH

Here /FINAL/INSTALL/PATH is either the path you specified to configure or simply the place where you compile the library.

You may also set your PARXXL environment variable to the 'par' directory, e.g.

 export PARXXL=/FINAL/INSTALL/PATH` 

if you are using a bash-shell or

 setenv PARXXL /FINAL/INSTALL/PATH

or put an adequat statement into your login startup file, e.g .bashrc ou .tcshrc.

Tuning parXXL and parXXL executables

There are some variables with which you may have a finer control of the compilation.

ASSERT=1

Compile with assertions switched on. This is recommended, until you are absolutely sure that your program is running smoothly.

QUIET=0

Control the verbosity of the code that is produced. Compiling all the library with a value of 1 give you much more messages than you can probably handle.

DEBUG=1

Control whether or not parXXL is compiled with debugging. Implies QUIET=0, so be careful.

NOSHARED=1

Control whether or not parXXL is used (or produced) as shared library. This is recommended on architecturers that support it, since it leaves the choice of the run-time (mpi or threads) to the start-up of the program.

NOMULLIBS=1

Control whether or not parXXL is used (or produced) as several seperate libraries, one per namespace, libparcpp.so for par::cpp, libparsys.so for par::sys etc.

Different target architectures

By default parXXL is optimized for the compile host. Therefore the installation procedure installs the produced libraries in an directory hierarchie that reflects the compile architecture. By that you may have compiled versions for different architectures simultaneously. E.g you might compile for both common processor architectures INTEL and AMD. Then the libraries could e.g be found in

 linux/gcc/x86_64/nocona
 linux/gcc/x86_64/athlon64

The parXXL run script `parXXLrun' will usually be able to choose the right one at run time.

Sometimes this may be inadequat. There is flag to the configure script, namely --with-libsuffix=LIBSUFFIX that enforce production of the libraries in ${PARXXL}/lib/[LIBSUFFIX]. Use this together with --with-gcc-arch=[arch] to produce a cross-compiled version of the library. For the above example on x86_64 you could produce three different versions of the library:

 configure --prefix=`pwd` --with-libsuffix=linux/gcc/x86_64/athlon64 --with-gcc-arch=athlon64
 configure --prefix=`pwd` --with-libsuffix=linux/gcc/x86_64/nocona --with-gcc-arch=nocona
 configure --prefix=`pwd` --with-libsuffix=linux/gcc/x86_64 --with-gcc-arch=x86_64

Running parXXL

How you may run a parXXL program depends on the communication model that you have chosen. Suppose you have successfully made the files in tests+benchs/sorting then you'd find as newly produced executables

 test_sortingLongQuick
 test_sortingDoubleQuick
 test_sortingLong
 test_sortingDouble

If you have used the defaults (i.e linked parXXL shared) these should be relatively small. The recommended way to launch a parXXL application is the script parXXLrun that comes in the script subdirectory (of the sources) or in the bin directory (where you installed).

Run with 8 parXXL processes and with one million elements per processor as

 parXXLrun -np 8 --logfile - --logger 0 ./test_sortingDoubleQuick 1000000

If done like this it uses the thread runtime `svm' on the local machine. If you like to run the program on one or several different machines you should give a machine file that lists the machines that are to be used:

 parXXLrun -np 8 -machinefile my_machines.txt --logf=- --logg=0 ./test_sortingDoubleQuick 1000000

my_machines.txt should contain one line with the name for each machine, followed by an optional number of processes that are to be run on it, e.g:

 localhost 2
 remote.host.org 2

With the line above this would place processes 0, 1, 4, 5 on `localhost' and 2, 3, 6 and 7 on remote.host.org.

If the machinefile contains only one host or if only one process is to be executed the job is launched with `ssh' on that host. The svm runtime is used in that case.

In other cases mpi is used to run the job and all arguments are simply passed to `mpirun'.

parXXLrun is also able to transfer and transform some environment variables to the remote execution sites:

 parXXLrun -help

Gives you an overview of the capacities of parXXLrun.

To know more about the arguments that a parXXL application receives try:

 parXXLrun -np 1 test_sortingDoubleQuick -h

It should list you the possible options that are eaten by parXXL and its different components.