Research - Introduction to using the HPC clusters
Several scientific applications require computing and or storage resources that go beyond the processing power of a single multi-core machine. High performance computing (HPC) clusters provide the necessary hardware and software infrastructure to efficiently run computing and data intensive applications. This course provides detailed information on how to efficiently use UNIL compute and storage infrastructure, and also to install/run most of the applications (R and Python based at least) on this infrastructure.
Objectives of this course
- Learn how to run calculations and process data on the UNIL HPC clusters and how to manage data in the UNIL storage facility
- Learn how to install and run in a clean way Python and R applications on the UNIL HPC clusters. The purpose is NOT to learn Python or R programming
- Learn how to run and create Singularity/Apptainer containers
Target audience
This course is meant as an introduction for PhD students, post-docs, researchers of UNIL and CHUV who need to conduct heavy scientific computing and/or manage large amount of data.
Content
This course has been recorded and is available as videos:
First half-day
- Description of the computing and storage facilities of the UNIL, and their cost models
- How to properly organise and manage data in the storage facility
- Learn how to submit jobs on the HPC clusters of the UNIL
- Overview of the software stack available on the cluster
- Good habits for an efficient usage of the processing and storage facilities as shared resources
Second half-day
- Python
- Install and run Python applications on HPC clusters (excepted Jura) from:
- the DCSR software stack
- Pypi repository
- Conda repository
- Sources
- Understand the interest Python/Conda virtual environments
- Install and run Python applications on HPC clusters (excepted Jura) from:
- R
- Install and run R applications on HPC clusters (excepted Jura) from:
- CRAN
- Bioconductor
- Install and run R applications on HPC clusters (excepted Jura) from:
- Properly organize the installed applications in the storage facility
- Be able to be autonomous with running Singularity containers and with creating simple ones.
Length
2 half-days
Prerequisites
- Minimal knowledge of Linux is required, please follow the course dedicated to Linux introduction if you are not familiar with Linux and command-line interface
- You will need a SSH client to connect to the clusters (see How To Access The Clusters). Please contact us at helpdesk@unil.ch (with DCSR in the title) if you have trouble to connect.