Installation of MVAPICH

MVAPICH is an open-source MPI software to exploit the novel features and mechanisms of InfiniBand and other RDMA enabled interconnects to deliver performance and scalability to MPI applications. If your Linux machines/cluster is using InfiniBand and/or Omni-Path, Ethernet/iWARP, and RoCE, so you can use MVAPICH for performing the parallel jobs of your calculations.

Official website: http://mvapich.cse.ohio-state.edu/

Check if your machine is using InfiniBand

Use ibv_devinfo

hca_id: mlx4_0

        transport:                      InfiniBand (0)

        fw_ver:                         2.32.5100

        node_guid:                      7cfe:9003:0017:2dd0

        sys_image_guid:                 7cfe:9003:0017:2dd3

        vendor_id:                      0x02c9

        vendor_part_id:                 4099

        hw_ver:                         0x1

        board_id:                       DEL1100001019

        phys_port_cnt:                  1

        Device ports:

                port:   1

                        state:                  PORT_ACTIVE (4)

                        max_mtu:                4096 (5)

                        active_mtu:             4096 (5)

                        sm_lid:                 1

                        port_lid:               1

                        port_lmc:               0x00

                        link_layer:             InfiniBand

Download

1. Go to download page of MVAPICH website

http://mvapich.cse.ohio-state.edu/register/

2. Register and download the tarball of source code to your machine. For example,

wget http://mvapich.cse.ohio-state.edu/download/mvapich/mv2/mvapich2-2.3.tar.gz

3. Unpack the tarball

tar -xzvf mvapich2-2.3.tar.gz

Installation of MVAPICH

1. Set up configuration using default setting (recommended)

./configure

If you want to install MVAPICH on your private space, just use --prefix option to specify the new directory

./configure --prefix=/home/rangsiman/mvapich2-2.3-local

If configuration setting completed without any error, you should see the output like this.

-----------------------------------------------------------------------------

Hwloc optional build support status (more details can be found above):

Probe / display I/O devices: PCI(linux)

Graphical output (Cairo):

XML input / output:          full

libnuma memory support:      no

Plugin support:              no

-----------------------------------------------------------------------------

Configuration completed.

2. Make and compile the binary

make

make install

If the installation was finished without any error (like below), so the MVAPICH is installed on your machine completely.

...

make[4]: Nothing to be done for `install-data-am'.

make[4]: Leaving directory `/home/rangsiman/mvapich2-2.3/osu_benchmarks'

make[3]: Leaving directory `/home/rangsiman/mvapich2-2.3/osu_benchmarks'

make[2]: Leaving directory `/home/rangsiman/mvapich2-2.3/osu_benchmarks'

make[1]: Leaving directory `/home/rangsiman/mvapich2-2.3'

3. Check the program version

cd /home/rangsiman/mvapich2-2.3-local/bin

./mpirun --version

Output

HYDRA build details:

    Version:                                 3.2.1

    Release Date:                            General Availability Release

    CC:                              gcc

    CXX:                             g++

    F77:                             gfortran

    F90:                             gfortran

    Configure options:                       '--disable-option-checking' '--prefix=/home/rangsiman/usr/local/mvapich2-2.3' '--cache-file=/dev/null' '--srcdir=.' 'CC=gcc' 'CFLAGS= -DNDEBUG -DNVALGRIND -O2' 'LDFLAGS=-L/lib -L/lib -L/lib -Wl,-rpath,/lib -L/lib -Wl,-rpath,/lib -L/lib -L/lib' 'LIBS=-libmad -lrdmacm -libumad -libverbs -ldl -lrt -lm -lpthread ' 'CPPFLAGS= -I/home/rangsiman/mvapich2-2.3/src/mpl/include -I/home/rangsiman/mvapich2-2.3/src/mpl/include -I/home/rangsiman/mvapich2-2.3/src/openpa/src -I/home/rangsiman/mvapich2-2.3/src/openpa/src -D_REENTRANT -I/home/rangsiman/mvapich2-2.3/src/mpi/romio/include -I/include -I/include -I/include -I/include' 'MPLLIBNAME=mpl'

    Process Manager:                         pmi

    Launchers available:                     ssh rsh fork slurm ll lsf sge manual persist

    Topology libraries available:            hwloc

    Resource management kernels available:   user slurm ll lsf sge pbs cobalt

    Checkpointing libraries available:

    Demux engines available:                 poll select

Rangsiman Ketkaew