JupyterLab on the curnagl cluster

JupyterLab can be run on the curnagl cluster for testing purposes, only as an intermediate step in the porting of applications from regular workstations to curnagl.

The installation is made inside a python virtual environment, and this tutorial covers the installation of the following kernels: IPyKernel (python), IRKernel (R), IJulia (julia), MATLAB kernel (matlab), IOctave (octave), stata_kernel (stata) and sas_kernel (sas).

If the workstation is outside of the campus, first connect to the VPN.

Creating the virtual environment

First create/choose a folder ${WORK} under the /scratch or the /work filesystems under your project (ex. WORK=/work/FAC/.../my_project). The following needs to be run only once on the cluster (preferably on an interactive computing node):

module load gcc python
python -m venv ${WORK}/jlab_venv
source ${WORK}/jlab_venv/bin/activate
pip install jupyterlab ipykernel numpy matplotlib
deactivate

The IPyKernel is automatically available. The other kernels need to be installed according to your needs

Installing the kernels

Each time you start a new session on the cluster, remember to define the variable ${WORK} according to the path you chose when creating the virtual environment.

IRKernel

module load gcc r
export R_LIBS_USER=${WORK}/jlab_venv/lib/Rlibs
mkdir ${R_LIBS_USER}
echo "install.packages('IRkernel', repos='https://stat.ethz.ch/CRAN/', lib=Sys.getenv('R_LIBS_USER'))" | R --no-save
echo "IRkernel::installspec()" | R --no-save

IJulia

module load gcc julia
export JULIA_DEPOT_PATH=${WORK}/jlab_venv/lib/Jlibs
julia -e 'using Pkg; Pkg.add("IJulia")'

MATLAB kernel

source ${WORK}/jlab_venv/bin/activate
pip install matlab_kernel pip install matlabengine==9.11.19
deactivate

IOctave

source ${WORK}/jlab_venv/bin/activate
pip install octave_kernel
echo "c.OctaveKernel.plot_settings = dict(backend='gnuplot')" > ~/.jupyter/octave_kernel_config.py
deactivate

stata_kernel

source${WORK}/jlab_venv/bin/pip install stata_kernel
${WORK}/jlab_venv/bin/activate
pip install stata_kernel
python -m stata_kernel.install
deactivate

sas_kernel

module load sas
source ${WORK}/jlab_venv/bin/activate
pip install sas_kernel
sed -i "s/'\/opt\/sasinside\/SASHome/'$(echo ${SAS_ROOT} | sed 's/\//\\\//g')/g" ${WORK}/jlab_venv/lib64/python3.9/site-packages/saspy/sascfg.py
deactivate

Running JupyterLab

Before running JupyterLab, you need to start an interactive session!

Sinteractive

If you didn't install all of the kernels, the corresponding lines should be ignored in the commands below. The execution order is important, in the sense that loading the gcc module should always be done before activating virtual environments.

# IOctaveLoad python
module load gcc python

# IOctave (optional)
module load octave gnuplot

# IRKernel (optional)
export R_LIBS_USER=${WORK}/jlab_venv/lib/Rlibs

# IJulia (optional)
export JULIA_DEPOT_PATH=${WORK}/jlab_venv/lib/Jlibs

# JupyterLab environment
source ${WORK}/jlab_venv/bin/activate

# Launch JupyterLab (on the shell a link that can be copied on the browser will appear)
jupyter-lab

deactivate

Before you can copy and paste the link into your favorite browser, you will need to establish an SSH tunnel to the interactive node. On curnagl, you can find the name of the node on which you are running with

hostname

From a UNIX-like workstation, you can establish the SSH tunnel to the curnagl node with the following command (replace <username> with your user name, and <hostname> with the name of the node you obtained above):

ssh -n -N -J <username>@curnagl.dcsr.unil.ch -L 8888:localhost:8888 <hostname>

You will be prompted for your password. When you have finished, you can close the tunnel with Ctrl-C.

Note on Python/R/Julia modules and packages

The modules you install manually from JupyterLab in Python, R or Julia end up inside the JupyterLab virtual environment (${WORK}/jlab_venv). They are hence isolated and independent from your Python/R/Julia instances outside of the virtual environment.