JupyterLab on the curnagl cluster
JupyterLab can be run on the curnagl cluster for testing purposes, only as an intermediate step in the porting of applications from regular workstations to curnagl.
The installation is made inside a python virtual environment, and this tutorial covers the installation of the following kernels: IPyKernel (python), IRKernel (R), IJulia (julia), MATLAB kernel (matlab), IOctave (octave), stata_kernel (stata) and sas_kernel (sas).
Creating the virtual environment
First create/choose a folder ${WORK} under the /scratch or the /work filesystems under your project (ex. WORK=/work/FAC/.../my_project). The following needs to be run only once on the cluster (preferably on an interactive computing node):
module load gcc python
python -m venv ${WORK}/venv
source ${WORK}/venv/bin/activate
pip install jupyterlab ipykernel numpy matplotlib
The IPyKernel is automatically available. The other kernels need to be installed according to your needs
Installing the kernels
Each time you start a new session on the cluster, remember to define the variable ${WORK} according to the math you chose when creating the virtual environment.
You also need to activate the virtual environment with:
source ${WORK}/venv/bin/activate
IRKernel
module load gcc r
export R_LIBS_USER=${WORK}/venv/lib/Rlibs
mkdir ${R_LIBS_USER}
echo "install.packages('IRkernel', repos='https://stat.ethz.ch/CRAN/', lib=Sys.getenv('R_LIBS_USER'))" | R --no-save
echo "IRkernel::installspec()" | R --no-save
IJulia
module load gcc julia
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs
julia -e 'using Pkg; Pkg.add("IJulia")'
MATLAB kernel
module load matlab/2021b
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)
pip install matlab_kernel
pip install matlabengine==9.11.19
IOctave
module load gcc octave gnuplot
pip install octave_kernel
echo "c.OctaveKernel.plot_settings = dict(backend='gnuplot')" > ~/.jupyter/octave_kernel_config.py
stata_kernel
module load stata-se
pip install stata_kernel
python -m stata_kernel.install
sas_kernel
module load sas
pip install sas_kernel
sed -i "s/'\/opt\/sasinside\/SASHome/'$(echo ${SAS_ROOT} | sed 's/\//\\\//g')/g" venv/lib64/python3.9/site-packages/saspy/sascfg.py
Running JupyterLab
Depending on the kernels you need, you need the following additional steps:
export R_LIBS_USER=${WORK}/venv/lib/Rlibs
export JULIA_DEPOT_PATH=${WORK}/venv/lib/Jlibs
module load matlab/2021b
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:$(readlink -f ${MATLAB_ROOT}/bin/glnxa64/)
module load gcc octave