JupyterLab on the curnagl cluster

JupyterLab can be run on the curnagl cluster for testing purposes, only as an intermediate step in the porting of applications from regular workstations to curnagl.

The installation is made inside a python virtual environment, and this tutorial covers the installation of the following kernels: IPyKernel (python), IRKernel (R), IJulia (julia), MATLAB kernel (matlab), IOctave (octave), stata_kernel (stata) and sas_kernel (sas).

Creating the virtual environment

First create/choose a folder ${WORK} under the /scratch or the /work filesystems under your project (ex. WORK=/work/FAC/.../my_project). The following needs to be run only once on the cluster (preferably on an interactive computing node):

module load gcc python
python -m venv ${WORK}/venv
source ${WORK}/venv/bin/activate
pip install jupyterlab ipykernel numpy matplotlib

The IPyKernel is automatically available. The other kernels need to be installed according to your needs

Installing the kernels

Each time you start a new session on the cluster, remember to define the variable ${WORK} according to the math you chose when creating the virtual environment.

IRKernel

module load gcc r
export R_LIBS_USER=${WORK}/venv/lib/Rlibs
mkdir ${R_LIBS_USER}
echo "install.packages('IRkernel', repos='https://stat.ethz.ch/CRAN/', lib=Sys.getenv('R_LIBS_USER'))" | R --no-save
echo "IRkernel::installspec()" | R --no-save

IJulia

module load gcc julia
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs
julia -e 'using Pkg; Pkg.add("IJulia")'

MATLAB kernel

module load matlab/2021b
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)
pip install matlab_kernel
pip install matlabengine==9.11.19

IOctave

module load gcc octave gnuplot
pip install octave_kernel
echo "c.OctaveKernel.plot_settings = dict(backend='gnuplot')" > ~/.jupyter/octave_kernel_config.py

stata_kernel

module load stata-se
pip install stata_kernel
python -m stata_kernel.install

sas_kernel

module load sas
pip install sas_kernel
sed -i "s/'\/opt\/sasinside\/SASHome/'$(echo ${SAS_ROOT} | sed 's/\//\\\//g')/g" venv/lib64/python3.9/site-packages/saspy/sascfg.py

Running JupyterLab

Depending on the kernels you need, you need the following additional steps:

export R_LIBS_USER=${WORK}/venv/lib/Rlibs

export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs

module load matlab/2021b

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)

module load gcc octave