JupyterLab on the curnagl cluster

JupyterLab can be run on the curnagl cluster for testing purposes, only as an intermediate step in the porting of applications from regular workstations to curnagl.

The installation is made inside a python virtual environment, and this tutorial covers the installation of the following kernels: IPyKernel (python), IRKernel (R), IJulia (julia), MATLAB kernel (matlab), IOctave (octave), stata_kernel (stata) and sas_kernel (sas).

If the workstation is outside of the campus, first connect to the VPN.

Creating the virtual environment

First create/choose a folder ${WORK} under the /scratch or the /work filesystems under your project (ex. WORK=/work/FAC/.../my_project). The following needs to be run only once on the cluster (preferably on an interactive computing node):

module load gcc python
python -m venv ${WORK}/venv
source ${WORK}/venv/bin/activate
pip install jupyterlab ipykernel numpy matplotlib
deactivate

The IPyKernel is automatically available. The other kernels need to be installed according to your needs

Installing the kernels

Each time you start a new session on the cluster, remember to define the variable ${WORK} according to the math you chose when creating the virtual environment.

IRKernel

module load gcc r
export R_LIBS_USER=${WORK}/venv/lib/Rlibs
mkdir ${R_LIBS_USER}
echo "install.packages('IRkernel', repos='https://stat.ethz.ch/CRAN/', lib=Sys.getenv('R_LIBS_USER'))" | R --no-save
echo "IRkernel::installspec()" | R --no-save

IJulia

module load gcc julia
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs
julia -e 'using Pkg; Pkg.add("IJulia")'

MATLAB kernel

#module load matlab/2021b
#export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)
source ${WORK}/venv/bin/activate
pip install matlab_kernel
pip install matlabengine==9.11.19
deactivate

IOctave

#module load gcc octave gnuplot
source ${WORK}/venv/bin/activate
pip install octave_kernel
echo "c.OctaveKernel.plot_settings = dict(backend='gnuplot')" > ~/.jupyter/octave_kernel_config.py
deactivate

stata_kernel

#module load stata-se
source ${WORK}/venv/bin/activate
pip install stata_kernel
python -m stata_kernel.install
deactivate

sas_kernel

#modulemodule load sas
source ${WORK}/venv/bin/activate
pip install sas_kernel
sed -i "s/'\/opt\/sasinside\/SASHome/'$(echo ${SAS_ROOT} | sed 's/\//\\\//g')/g" ${WORK}/venv/lib64/python3.9/site-packages/saspy/sascfg.py
deactivate

Running JupyterLab

DependingBefore onrunning the kernels you need,JupyterLab, you need theto followingstart additionalan steps:interactive session!

Sinteractive

#module

If loadyou matlab/2021bdon't #exportneed LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlinkall -fof ${MATLAB_ROOT}/bin/glnxa64/)the kernels, the corresponding lines can be commented below. The execution order is important, in the sense that loading the gcc module should always be done before activating virtual environments.

# IOctave
module load gcc octave gnuplot

source# ${WORK}/venv/bin/activateIRKernel
export R_LIBS_USER=${WORK}/venv/lib/Rlibs

# IJulia
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs

# JupyterLab environment
source ${WORK}/venv/bin/activate

# Launch JupyterLab (on the shell a link that can be copied on the browser will appear)
jupyter-lab

deactivate

Before you can paste the link to the browser, you will need to establish an SSH tunnel to the interactive node. On curnagl, you can find the name of the node on which you are running with

hostname

From a UNIX-like workstation, you can establish the SSH tunnel with the following command (replace <username> with your user name, and <hostname> with the name of the node you obtained above):

ssh -n -N -J <username>@curnagl.dcsr.unil.ch -L 8888:localhost:8888 <hostname>

You will be prompted for your password. When you have finished, you can close the tunnel with Ctrl-C.