Passer au contenu principal

Run MPI with containers

Simple test

Simple container with ucx and openmpi.

Bootstrap: docker
From: debian:trixie

%environment
    export LD_LIBRARY_PATH=/usr/local/lib
    
%post
	apt-get update && apt-get install -y build-essential wget rdma-core libibverbs-dev
	wget https://github.com/openucx/ucx/releases/download/v1.18.1/ucx-1.18.1.tar.gz
	tar xzf ucx-1.18.1.tar.gz
	cd ucx-1.18.1
	mkdir build
	cd build
	../configure --prefix=/opt/
	make -j4
	make install
	cd ..
	export OPENMPI_VERSION="4.1.6"
	export OPENMPI_MAJOR_VERSION="v4.1"
	export OPENMPI_MAKE_OPTIONS="-j4"
	mkdir -p /openmpi-src
	cd /openmpi-src
	wget https://download.open-mpi.org/release/open-mpi/${OPENMPI_MAJOR_VERSION}/openmpi-${OPENMPI_VERSION}.tar.gz \
      	&& tar xfz openmpi-${OPENMPI_VERSION}.tar.gz
	cd openmpi-${OPENMPI_VERSION} && ./configure --with-ucx=/opt
	make all ${OPENMPI_MAKE_OPTIONS}
	make install
	cd /
	rm -rf /openmpi-src

To build it:

singularity build -f openmpitest.sif openmpi.def

Then we compile an MPI application inside the container. For example osu-benchmarks.

wget https://mvapich.cse.ohio-state.edu/download/mvapich/osu-micro-benchmarks-7.5-1.tar.gz
tar -xvf osu-micro-benchmarks-7.5-1.tar.gz

singularity shell openmpitest.sif

cd osu-micro-benchmarks-7.5-1
./configure CC=/usr/local/bin/mpicc CXX=/usr/local/bin/mpicxx --prefix=/scratch/$user/osu_install
make install

Then you can use the following job:

#!/bin/bash

#SBATCH -N 2
#SBATCH -n 2
#SBATCH -o mpi-%j.out
#SBATCH -e mpi-%j.err

module purge
module load singularityce
module load openmpi


export SINGULARITY_BINDPATH=/usr/lib64/libmunge.so.2,/var/run/munge,/scratch

srun --mpi=pmix singularity run openmpitest.sif /scratch/$user/osu-install/libexec/osu-micro-benchmarks/mpi/collective/osu_alltoall
Retour en haut