Offline installation on Jura
Installing new software on Jura is complicated because the cluster does not have Internet access. This page covers the installation of R, BioConductor, and Python packages.
R packages
For packages in CRAN
Since Jura cluster is not connected to the Internet, it won't be possible to install R packages directly. A local CRAN mirror has been deployed to ease the installation, it can be used as follows:
module load gcc r
R
>install.packages(c('dplyr','ggplot2','cluster'), repos='http://mirror.dcsr.unil.ch/cran/')
For packages not in CRAN
For packages not available in the local CRAN mirror, you will have to go through the following procedure:
STEP 1: download the package on a machine connected to the internet, e.g.:
wget http://cnsgenomics.com/software/gsmr/static/gsmr_1.0.9.tar.gz
STEP 2: transfer the package to Jura using the standard procedure detailed here
STEP 3: log into Jura
STEP 4: launch R
module load gcc r
R
STEP 5: install the package with:
> install.packages("/path/to/gsmr_1.0.9.tar.gz", repos = NULL, type="source")
For packages with the source on a Git server
For packages provided on a Git server, you will have to first build a package to transfer to Jura before you can proceed with the installation of the package per se. From a machine connected to the internet:
STEP 1: clone the Git repository in directory hereinafter referred to as "/path/to"
git clone https://github.com/jean997/causeSims.git
STEP 2: launch R
module load gcc r
R
STEP 3: build the R package
> require("devtools")
> build("causeSims")
which should output something along these lines:
> build("causeSims")
✔ checking for file ‘/path/to/causeSims/DESCRIPTION’ ...
─ preparing ‘causeSims’:
✔ checking DESCRIPTION meta-information ...
─ checking for LF line-endings in source and make files and shell scripts
─ checking for empty or unneeded directories
─ building ‘causeSims_0.1.0.tar.gz’
Warning: invalid uid value replaced by that for user 'nobody'
[1] "/path/to/causeSims_0.1.0.tar.gz"
Then continue with STEPS 2, 3, 4 & 5 of paragraph "For packages not in CRAN" above.
BioConductor packages
BioConductor is a package manager that enhances R with bio-informatic packages. A local BioConductor mirror has been deployed to ease the installation.
First, you have to define a `~/.Rprofile` file with, at least, the following content:
options(
BioC_mirror = "http://mirror.dcsr.unil.ch/bioconductor",
repos = "http://mirror.dcsr.unil.ch/cran"
)
options(
BIOCONDUCTOR_ONLINE_VERSION_DIAGNOSIS = FALSE
)
It is very important to have an empty new line at the end of the file!
Then you can launch R:
module load gcc r
R
And install the BioConducter package manager:
> install.packages("http://mirror.dcsr.unil.ch/cran/src/contrib/Archive/BiocManager/BiocManager_1.30.10.tar.gz", repos=NULL, type="source")
> install.packages("http://mirror.dcsr.unil.ch/bioconductor/BiocVersion_3.12.0.tar.gz", repos=NULL, type="source")
> library(BiocManager)
The first step might ask if you want to use a personal library, you can answer yes to both questions.
Finally, you can install the BioConductor packages, for instance edgeR
:
> BiocManager::install("edgeR")
At the end of the installation, R might ask you if you want to update BiocManager package. Please don't since newer version is not working with the installed version of R
Python packages
Thankfully Python packages are somewhat easier to deal with - here we use PyTorch as an example
First, on a system that has internet access use pip3 download
mkdir torch
cd torch
pip3 download torch torchvision
Collecting torch
Downloading https://files.pythonhosted.org/packages/76/58/668ffb25215b3f8231a550a227be7f905f514859c70a65ca59d28f9b7f60/torch-1.5.0-cp37-cp37m-manylinux1_x86_64.whl (752.0MB)
|████████████████████████████████| 752.0MB 33kB/s
Saved ./torch-1.5.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting torchvision
Downloading https://files.pythonhosted.org/packages/7b/ed/a894f274a7733d6492e438a5831a95b507c5ec777edf6d8c3b97574e08c4/torchvision-0.6.0-cp37-cp37m-manylinux1_x86_64.whl (6.6MB)
|████████████████████████████████| 6.6MB 15.4MB/s
Saved ./torchvision-0.6.0-cp37-cp37m-manylinux1_x86_64.whl
Collecting numpy
Using cached https://files.pythonhosted.org/packages/1f/df/7988fbbdc8c9b8efb575029498ad84b77e023a3e4623e85068823a102b1d/numpy-1.18.4-cp37-cp37m-manylinux1_x86_64.whl
Saved ./numpy-1.18.4-cp37-cp37m-manylinux1_x86_64.whl
Collecting future
Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz (829kB)
|████████████████████████████████| 829kB 23.1MB/s
Saved ./future-0.18.2.tar.gz
Collecting pillow>=4.1.1
Downloading https://files.pythonhosted.org/packages/ab/f8/d3627cc230270a6a4eedee32974fbc8cb26c5fdb8710dd5ea70133640022/Pillow-7.1.2-cp37-cp37m-manylinux1_x86_64.whl (2.1MB)
|████████████████████████████████| 2.1MB 15.3MB/s
Saved ./Pillow-7.1.2-cp37-cp37m-manylinux1_x86_64.whl
Successfully downloaded torch torchvision numpy future pillow
This will download all the required files which can then be copied to the system without internet access.
Then we can move torch
directory to Jura using SFTP server.
Finally, we can use pip / pip3 to install the package from the downloaded files (torch directory).
$ pip install --user --no-index --find-links=torch torch torchvision
Please be aware that some Python packages take up a lot of space and you may wish to set a non standard installation directory via the --target
option of pip - see the Pip documentation for full details.