JupyterLab on the curnagl cluster
JupyterLab can be run on the curnagl cluster for testing purposes, only as an intermediate step in the porting of applications from regular workstations to curnagl.
The installation is made inside a python virtual environment, and this tutorial covers the installation of the following kernels: IPyKernel (python), IRKernel (R), IJulia (julia), MATLAB kernel (matlab), IOctave (octave), stata_kernel (stata) and sas_kernel (sas).
Creating the virtual environment
First create/choose a folder ${WORK} under the /scratch or the /work filesystems under your project (ex. WORK=/work/FAC/.../my_project). The following needs to be run only once on the cluster (preferably on an interactive computing node):
module load gcc python
python -m venv ${WORK}/venv
source ${WORK}/venv/bin/activate
pip install jupyterlab ipykernel numpy matplotlib
deactivate
The IPyKernel is automatically available. The other kernels need to be installed according to your needs
Installing the kernels
Each time you start a new session on the cluster, remember to define the variable ${WORK} according to the math you chose when creating the virtual environment.
IRKernel
module load gcc r
export R_LIBS_USER=${WORK}/venv/lib/Rlibs
mkdir ${R_LIBS_USER}
echo "install.packages('IRkernel', repos='https://stat.ethz.ch/CRAN/', lib=Sys.getenv('R_LIBS_USER'))" | R --no-save
echo "IRkernel::installspec()" | R --no-save
IJulia
module load gcc julia
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs
julia -e 'using Pkg; Pkg.add("IJulia")'
MATLAB kernel
#module load matlab/2021b
#export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)
source ${WORK}/venv/bin/activate
pip install matlab_kernel
pip install matlabengine==9.11.19
deactivate
IOctave
#module load gcc octave gnuplot
source ${WORK}/venv/bin/activate
pip install octave_kernel
echo "c.OctaveKernel.plot_settings = dict(backend='gnuplot')" > ~/.jupyter/octave_kernel_config.py
deactivate
stata_kernel
#module load stata-se
source ${WORK}/venv/bin/activate
pip install stata_kernel
python -m stata_kernel.install
deactivate
sas_kernel
#module load sas
source ${WORK}/venv/bin/activate
pip install sas_kernel
sed -i "s/'\/opt\/sasinside\/SASHome/'$(echo ${SAS_ROOT} | sed 's/\//\\\//g')/g" ${WORK}/venv/lib64/python3.9/site-packages/saspy/sascfg.py
deactivate
Running JupyterLab
Depending on the kernels you need, you need the following additional steps:
#module load matlab/2021b
#export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)
module load gcc octave
source ${WORK}/venv/bin/activate
export R_LIBS_USER=${WORK}/venv/lib/Rlibs
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs