Storage and costs

Non-sensitive data (Curnagl)
Summary of storage file systems
The costs page is available here (UNIL login is required).



Storage
Use it for
Copies
Safety
Cost
Accessible from




/users (home)
Configuration files
1
Safe
7-days snapshots
Free, quota is strict (50GB)
Curnagl login node



/scratch
 

Temporary files of running jobs that need large amounts of data
1
> 2weeks-old files are purged when 90% full
Free, no quota

Curnagl login and compute nodes



/work
Data used in current computations
1
No backup
With quota, see "U1 cost" for "GPFS (HPC - normal)" in the costs page
Curnagl login and compute nodes


/nas D2c
Important results of ongoing projects
2

With backup

With quota, add "U1 cost" for "Stockage copie primaire (Isilon)" and "Stockage 2ème copie (Isilon)" in the costs page
Curnagl login node and local mount (see wiki)


/nas D1c
Reloadable datasets of ongoing projects
1

No backup

With quota, see "U1 cost" for "Stockage copie primaire (Isilon)" in the costs page + see blue frame below
Curnagl login node and local mount (see wiki)


LTS (tape)
Data of closed projects, archive
2
On tape
Free, no quota

Once data is on tape: upon request (see step 7 here)



S3
Share data with external collaborators
1
No backup
With quota, see "U1 cost" for "Stockage Objet" in the costs page.

Curnagl login node and local mount (see wiki)




Each PI has 1 TB of free /nas space, shared across all their open projects: 500 GB in D2c or 1 TB in D1c.
Costs are based on the space you actually use, not on your maximum quota.
Where to store your non-sensitive data during a project
This diagram shows where to keep your data at each stage of a project.
Read it from left to right: each column is a project stage, each row a storage system, and the arrows show where data moves. Amber storage has a single copy or is temporary, so move important data out of it.
Letters refer to storage systems, which wiki pages are given below:

A - /scratch
B - /work
C - /nas
D - LTS
E - S3

Each number links to a wiki page with detailed instructions:

Transfert S3 DCSR to other support
Transfer files to/from Curnagl
Access NAS DCSR from the cluster
How to run a job on Curnagl
How much space am I using?
Share files from a bucket with presign keys
Give access to DCSR NAS files to an external collaborator
Filetransfer from the cluster
SWITCHfilesender from the cluster
Email helpdesk@unil.ch to get more information about LTS requests

Sensitive Data (Urblauna)
Summary of storage file systems



Storage
Use it for
Copies
Safety
Cost
Accessible from





/users
(home)

Configuration files
1
Safe
7-days snapshots
Free, quota is strict (50GB)
Urblauna login node



/scratch
 

Temporary files of running jobs that need large amounts of data
1
> 2weeks-old files are purged when 90% full
Free

Urblauna login and compute nodes



/data
Data used in current computations
1
No backup
See "U1 cost" for "GPFS (HPC - sensible)" in the costs page
Urblauna login and compute nodes


/work
(read-only)

Read-only from Urblauna
Software installed from Curnagl (with internet connection)

1
No backup
See "U1 cost" for "GPFS (HPC - normal)" in the costs page
Urblauna login and compute nodes in read-only


/archive (tape)
Data of closed projects or not accessed regularly
2
With backup
See "U1 cost" for "Archivage (sensible)" in the costs page

Urblauna login node, quickly if data still in cache, very slowly otherwise (see wiki)




Where to store your sensitive data during a project

This diagram shows where to keep your data at each stage of a project.
Read it from left to right: each column is a project stage, each row a storage system, and the arrows show where data moves. Amber storage has a single copy or is temporary, so move important data out of it.
Letters refer to storage systems, which wiki pages are given below:

A - /scratch
B - /data
C - /reference
D - /work
E - /archive
F - Tresorit

Each number links to a wiki page with detailed instructions:

Urblauna access and data transfer
Installing your own software
How much space am I using?
Can I collaborate with external collaborators?

Good practices

Move your results out of /scratch and /work (or /data for Urblauna) as soon as your computations are finished.
Do not copy large read-only datasets to /scratch: /work (or /data for Urblauna) performance is more than sufficient for reading them.
Using the 
touch command to prevent files from being purged from /scratch is strictly forbidden.
Move data of closed projects:

to LTS to free up /nas space for non-sensitive data,
to /archive for sensitive data.



Quota increase
To increase a /nas or a /work quota, email helpdesk@unil.ch with "DCSR quota increase" in the subject line and the exact amount required. As this may increase quarterly invoices, we need the PI's approval before applying the change.
More information

Detailed information about storage file systems in Curnagl
Detailed information about storage file systems in Urblauna
Tresorit: file storage and sharing service for your local sensitive data