JupyterLab on the curnagl cluster

JupyterLab can be run on the curnagl cluster for testing purposes, only as an intermediate step in the porting of applications from regular workstations to curnagl.

The installation is made inside a python virtual environment, and this tutorial covers the installation of the following kernels: IPyKernel (python), IRKernel (R), IJulia (julia), MATLAB kernel (matlab), IOctave (octave), stata_kernel (stata) and sas_kernel (sas).

Creating the virtual environment

First create/choose a folder ${WORK} under the /scratch or the /work filesystems under your project (ex. WORK=/work/FAC/.../my_project). The following needs to be run only once on the cluster (preferably on an interactive computing node):

module load gcc python
python -m venv ${WORK}/venv
source ${WORK}/venv/bin/activate
pip install jupyterlab ipykernel numpy matplotlib
deactivate

The IPyKernel is automatically available. The other kernels need to be installed according to your needs

Installing the kernels

Each time you start a new session on the cluster, remember to define the variable ${WORK} according to the math you chose when creating the virtual environment.

You also need to activate the virtual environment with:

source ${WORK}/venv/bin/activate

IRKernel

module load gcc r
export R_LIBS_USER=${WORK}/venv/lib/Rlibs
mkdir ${R_LIBS_USER}
echo "install.packages('IRkernel', repos='https://stat.ethz.ch/CRAN/', lib=Sys.getenv('R_LIBS_USER'))" | R --no-save
echo "IRkernel::installspec()" | R --no-save

IJulia

module load gcc julia
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs
julia -e 'using Pkg; Pkg.add("IJulia")'

MATLAB kernel

#module load matlab/2021b
#export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)
source ${WORK}/venv/bin/activate
pip install matlab_kernel
pip install matlabengine==9.11.19
deactivate

IOctave

#module load gcc octave gnuplot
source ${WORK}/venv/bin/activate
pip install octave_kernel
echo "c.OctaveKernel.plot_settings = dict(backend='gnuplot')" > ~/.jupyter/octave_kernel_config.py
deactivate

stata_kernel

#module load stata-se
source ${WORK}/venv/bin/activate
pip install stata_kernel
python -m stata_kernel.install
deactivate

sas_kernel

#module load sas
source ${WORK}/venv/bin/activate
pip install sas_kernel
sed -i "s/'\/opt\/sasinside\/SASHome/'$(echo ${SAS_ROOT} | sed 's/\//\\\//g')/g" ${WORK}/venv/lib64/python3.9/site-packages/saspy/sascfg.py
deactivate

Running JupyterLab

Depending on the kernels you need, you need the following additional steps:

export
R_LIBS_USER=${WORK}/venv/lib/Rlibs

export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs

module#module load matlab/2021b

#export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)

module load gcc octave

source ${WORK}/venv/bin/activate

export R_LIBS_USER=${WORK}/venv/lib/Rlibs

export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs