Skip to main content

Offline installation on Jura

Installing new software on Jura is complicated because the cluster does not have Internet access. This page covers the installation of R, BioConductor, and Python packages.

R packages

For packages in CRAN

Since Jura cluster is not connected to the Internet, it won't be possible to install R packages directly. A local CRAN mirror has been deployed to ease the installation, it can be used as follows:

source /dcsrsoft/spack/bin/setup_dcsrsoft
module load gcc r
R
>install.packages(c('dplyr','ggplot2','cluster'), repos='http://mirror.dcsr.unil.ch/cran/')
 
For packages not in CRAN

For packages not available in the local CRAN mirror, you will have to go through the following procedure:

STEP 1: download the package on a machine connected to the internet, e.g.:

wget http://cnsgenomics.com/software/gsmr/static/gsmr_1.0.9.tar.gz

STEP 2: transfer the package to Jura using the standard procedure detailed here

STEP 3: log into Jura

STEP 4: launch R

$ source /dcsrsoft/spack/bin/setup_dcsrsoft
$ module load gcc r
$ R

STEP 5: install the package with:

> install.packages("/path/to/gsmr_1.0.9.tar.gz", repos = NULL, type="source")
 
For packages with the source on a Git server

For packages provided on a Git server, you will have to first build a package to transfer to Jura before you can proceed with the installation of the package per se. From a machine connected to the internet:

STEP 1: clone the Git repository in directory hereinafter referred to as "/path/to"

$ git clone https://github.com/jean997/causeSims.git

STEP 2: launch R

$ source /dcsrsoft/spack/bin/setup_dcsrsoft
$ module load gcc r
$ R

STEP 3: build the R package

> require("devtools")
> build("causeSims")

which should output something along these lines:

> build("causeSims")
✔ checking for file ‘/path/to/causeSims/DESCRIPTION’ ...
─ preparing ‘causeSims’:
✔ checking DESCRIPTION meta-information ...
─ checking for LF line-endings in source and make files and shell scripts
─ checking for empty or unneeded directories
─ building ‘causeSims_0.1.0.tar.gz’
  Warning: invalid uid value replaced by that for user 'nobody'
  
[1] "/path/to/causeSims_0.1.0.tar.gz"

Then continue with STEPS 2, 3, 4 & 5 of paragraph "For packages not in CRAN" above.

 

BioConductor packages

BioConductor is a package manager that enhances R with bio-informatic packages. A local BioConductor mirror has been deployed to ease the installation.

First, you have to define a `~/.Rprofile` file with, at least, the following content:

options(
    BioC_mirror = "http://mirror.dcsr.unil.ch/bioconductor",
    repos = "http://mirror.dcsr.unil.ch/cran"
)
options(
    BIOCONDUCTOR_ONLINE_VERSION_DIAGNOSIS = FALSE
)

 

It is very important to have an empty new line at the end of the file!

Then you can launch R:

source /dcsrsoft/spack/bin/setup_dcsrsoft
module load gcc r
R

And install the BioConducter package manager:

> install.packages("http://mirror.dcsr.unil.ch/cran/src/contrib/Archive/BiocManager/BiocManager_1.30.10.tar.gz", repos=NULL, type="source")
> install.packages("http://mirror.dcsr.unil.ch/bioconductor/BiocVersion_3.12.0.tar.gz", repos=NULL, type="source")
> library(BiocManager)

The first step might ask if you want to use a personal library, you can answer yes to both questions.

Finally, you can install the BioConductor packages, for instance edgeR:

> BiocManager::install("edgeR")

At the end of the installation, R might ask you if you want to update BiocManager package. Please don't since newer version is not working with the installed version of R

Python packages

Thankfully Python packages are somewhat easier to deal with - here we use PyTorch as an example

First, on a system that has internet access use pip3 download

$ mkdir torch

$ cd torch

$ pip3 download torch torchvision
Collecting torch
  Downloading https://files.pythonhosted.org/packages/76/58/668ffb25215b3f8231a550a227be7f905f514859c70a65ca59d28f9b7f60/torch-1.5.0-cp37-cp37m-manylinux1_x86_64.whl (752.0MB)
     |████████████████████████████████| 752.0MB 33kB/s 
  Saved ./torch-1.5.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting torchvision
  Downloading https://files.pythonhosted.org/packages/7b/ed/a894f274a7733d6492e438a5831a95b507c5ec777edf6d8c3b97574e08c4/torchvision-0.6.0-cp37-cp37m-manylinux1_x86_64.whl (6.6MB)
     |████████████████████████████████| 6.6MB 15.4MB/s 
  Saved ./torchvision-0.6.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting numpy
  Using cached https://files.pythonhosted.org/packages/1f/df/7988fbbdc8c9b8efb575029498ad84b77e023a3e4623e85068823a102b1d/numpy-1.18.4-cp37-cp37m-manylinux1_x86_64.whl
  Saved ./numpy-1.18.4-cp37-cp37m-manylinux1_x86_64.whl
Collecting future
  Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz (829kB)
     |████████████████████████████████| 829kB 23.1MB/s 
  Saved ./future-0.18.2.tar.gz
Collecting pillow>=4.1.1
  Downloading https://files.pythonhosted.org/packages/ab/f8/d3627cc230270a6a4eedee32974fbc8cb26c5fdb8710dd5ea70133640022/Pillow-7.1.2-cp37-cp37m-manylinux1_x86_64.whl (2.1MB)
     |████████████████████████████████| 2.1MB 15.3MB/s 
  Saved ./Pillow-7.1.2-cp37-cp37m-manylinux1_x86_64.whl
Successfully downloaded torch torchvision numpy future pillow

This will download all the required files which can then be copied to the system without internet access. 

Then we can move `torch` directory to Jura using SFTP server: see 

https://wiki.unil.ch/ci/books/service-de-calcul-haute-performance-%28hpc%29/page/jura

Finally, we can use pip / pip3 to install the package from the downloaded files (torch directory).

$ pip install --user --no-index --find-links=torch torch torchvision

Please be aware that some Python packages take up a lot of space and you may wish to set a non standard installation directory via the --target option of pip - see the Pip documentation for full details.

Back to top