Running the Isca framework on the cluster

Installation

First of all define a folder ${WORK} on the /work filesystem (somewhere where you have write permissions):

export WORK=/work/FAC/...
mkdir -p ${WORK}

Load the following relevant modules and create a python virtual environment:

module load gcc/10.4.0
module load mvapich2/2.3.7
module load netcdf-c/4.8.1-mpi
module load netcdf-fortran/4.5.4
module load python/3.9.13

python -m venv ${WORK}/isca_venv

Install the required python modules:

${WORK}/isca_venv/bin/pip install dask f90nml ipykernel Jinja2 numpy pandas pytest sh tqdm xarray

Download and install the Isca framework:

cd ${WORK}
git clone https://github.com/ExeClim/Isca
cd Isca/src/extra/python
${WORK}/isca_venv/bin/pip install -e .

Patch the Isca makefile:

sed -i 's/-fdefault-double-8$/-fdefault-double-8 \\\n           -fallow-invalid-boz -fallow-argument-mismatch/' ${WORK}/Isca/src/extra/python/isca/templates/mkmf.template.gfort

Compiling and running the Held-Suarez dynamical core test case

Compilation takes place automatically at runtime. After logging in to the cluster you always need to setup the environment with:

module load gcc/10.4.0
module load mvapich2/2.3.7
module load netcdf-c/4.8.1-mpi
module load netcdf-fortran/4.5.4
module load python/3.9.13

export GFDL_BASE=${WORK}/Isca
export GFDL_ENV=gfortran
export GFDL_WORK=${WORK}/isca_work
export GFDL_DATA=${WORK}/isca_gfdl_data

export C_INCLUDE_PATH=${NETCDF_C_ROOT}/include
export LIBRARY_PATH=${NETCDF_C_ROOT}/lib

source ${WORK}/isca_venv/bin/activate

Then you can change to the directory of the Held-Suarez experiment and run:

cd $GFDL_BASE/exp/test_cases/held_suarez
python held_suarez_test_case.py