DCSR NAS

General information

How to access the DCSR NAS

If you don't have a DCSR NAS (Network Attached Storage) storage yet and you would like one, the PI (Principal Investigator) of your project needs to fill up a request application form on the following web interface:

https://conference.unil.ch/research-resource-requests/

Once the DCSR has accepted the request and the access has been granted to you by your PI, you will be able to access your DCSR NAS storage by one of the following method:

Access from a laptop

laptop <-> NAS instructions

Access from the cluster

If you have access to the compute clusters of the DCSR, the data stored on the NAS is directly accessible under /nas. See:

cluster <-> NAS instructions


In case you have asked for the "sensitive data" special option for your storage, please follow the specific instructions to access RECHERCHE-S on the following page:

How to manage sensitive data on UNIL NAS

General structure of the NAS

The general structure of the directories on the DCSR NAS (Network Attached Storage) is:

nasdcsr.unil.ch/RECHERCHE/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>

On mac

smb://nasdcsr.unil.ch/RECHERCHE/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>

On Windows

\\nasdcsr.unil.ch\RECHERCHE\FAC\<your_faculty>\<your_department>\<your_PI>\<your_project>


In case you have asked for the "personal data" special option for your storage, please use RECHERCHE-P with:

nasdcsr.unil.ch/RECHERCHE-P/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>

On mac

smb://nasdcsr.unil.ch/RECHERCHE-P/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>

On Windows

\\nasdcsr.unil.ch\RECHERCHE-P\FAC\<your_faculty>\<your_department>\<your_PI>\<your_project>


In case you have asked for the "sensitive data" special option for your storage, please follow the specific instructions to access RECHERCHE-S on the following page:

How to manage sensitive data on UNIL NAS

What are the /D1c, /D2c, and /LTS subdirectories?

In your project directory you will see at least a D2c and a LTS directory, with the following meanings/purposes:

<your_project>/D2c

D2c means "data 2 copies". This is where you can put your research data during the life time of your project. This NAS directory has an off-site backup (i.e the data is stored in two distinct locations). Snapshots are done on a daily basis for a rolling period of 3 months. Please note that in case you are using the HPC cluster and perform computations on this data, your are advised to copy this data to a fast storage (i.e. /scratch or /work) to speed up computations (see this page).

<your_project>/LTS

LTS means "Long Term Storage". This data will be kept for a very long time on the UNIL storage (a decade, on tapes). This is where you should put the part of your data that needs to be archived e.g. at the end of your project. When you reach this point, you will then be asked to make a description of your data ("meta-data"). See "Long-term storage of research data" section here: https://www.unil.ch/openscience/en/home/menuinst/open-research-data/gerer-les-donnees-de-recherche-research-data-management/stockage--securite.html

Users who explicitly asked for it will also have a D1c directory:

<your_project>/D1c

D1c means "data 1 copy". This data has no backup (and may be lost). This is where you can put e.g. some reference data that you downloaded from an external source, and that can easily be recovered again from the same external source, in case the data get lost in /D1c.

Long term storage at DCSR and other data repositories


Recovering files from the DCSR NAS

Introduction

If you have deleted or corrupted a file you can recover it from the daily snapshots that are performed on the DCSR NAS

From the Curnagl login node


ssh username@curnagl.dcsr.unil.ch
cd /nas/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>/D2c
cd .snapshot
ls
daily_1month_CAMPUS_REC_2021-12-06_12:00 daily_1month_CAMPUS_REC_2021-12-31_12:00 daily_3month_CAMPUS_REC_2021-10-27_18:00 daily_3month_CAMPUS_REC_2021-11-21_18:00 daily_3month_CAMPUS_REC_2021-12-16_18:00
daily_1month_CAMPUS_REC_2021-12-07_12:00 daily_1month_CAMPUS_REC_2022-01-01_12:00 daily_3month_CAMPUS_REC_2021-10-28_18:00 daily_3month_CAMPUS_REC_2021-11-22_18:00 daily_3month_CAMPUS_REC_2021-12-17_18:00
daily_1month_CAMPUS_REC_2021-12-08_12:00 daily_1month_CAMPUS_REC_2022-01-02_12:00 daily_3month_CAMPUS_REC_2021-10-29_18:00 daily_3month_CAMPUS_REC_2021-11-23_18:00 daily_3month_CAMPUS_REC_2021-12-18_18:00
daily_1month_CAMPUS_REC_2021-12-09_12:00 daily_1month_CAMPUS_REC_2022-01-03_12:00 daily_3month_CAMPUS_REC_2021-10-30_18:00 daily_3month_CAMPUS_REC_2021-11-24_18:00 daily_3month_CAMPUS_REC_2021-12-19_18:00
daily_1month_CAMPUS_REC_2021-12-10_12:00 daily_1month_CAMPUS_REC_2022-01-04_12:00 daily_3month_CAMPUS_REC_2021-10-31_18:00 daily_3month_CAMPUS_REC_2021-11-25_18:00 daily_3month_CAMPUS_REC_2021-12-20_18:00
daily_1month_CAMPUS_REC_2021-12-11_12:00 daily_3month_CAMPUS_REC_2021-10-07_18:00 daily_3month_CAMPUS_REC_2021-11-01_18:00 daily_3month_CAMPUS_REC_2021-11-26_18:00 daily_3month_CAMPUS_REC_2021-12-21_18:00
daily_1month_CAMPUS_REC_2021-12-12_12:00 daily_3month_CAMPUS_REC_2021-10-08_18:00 daily_3month_CAMPUS_REC_2021-11-02_18:00 daily_3month_CAMPUS_REC_2021-11-27_18:00 daily_3month_CAMPUS_REC_2021-12-22_18:00
daily_1month_CAMPUS_REC_2021-12-13_12:00 daily_3month_CAMPUS_REC_2021-10-09_18:00 daily_3month_CAMPUS_REC_2021-11-03_18:00 daily_3month_CAMPUS_REC_2021-11-28_18:00 daily_3month_CAMPUS_REC_2021-12-23_18:00
daily_1month_CAMPUS_REC_2021-12-14_12:00 daily_3month_CAMPUS_REC_2021-10-10_18:00 daily_3month_CAMPUS_REC_2021-11-04_18:00 daily_3month_CAMPUS_REC_2021-11-29_18:00 daily_3month_CAMPUS_REC_2021-12-24_18:00
daily_1month_CAMPUS_REC_2021-12-15_12:00 daily_3month_CAMPUS_REC_2021-10-11_18:00 daily_3month_CAMPUS_REC_2021-11-05_18:00 daily_3month_CAMPUS_REC_2021-11-30_18:00 daily_3month_CAMPUS_REC_2021-12-25_18:00
daily_1month_CAMPUS_REC_2021-12-16_12:00 daily_3month_CAMPUS_REC_2021-10-12_18:00 daily_3month_CAMPUS_REC_2021-11-06_18:00 daily_3month_CAMPUS_REC_2021-12-01_18:00 daily_3month_CAMPUS_REC_2021-12-26_18:00
daily_1month_CAMPUS_REC_2021-12-17_12:00 daily_3month_CAMPUS_REC_2021-10-13_18:00 daily_3month_CAMPUS_REC_2021-11-07_18:00 daily_3month_CAMPUS_REC_2021-12-02_18:00 daily_3month_CAMPUS_REC_2021-12-27_18:00
daily_1month_CAMPUS_REC_2021-12-18_12:00 daily_3month_CAMPUS_REC_2021-10-14_18:00 daily_3month_CAMPUS_REC_2021-11-08_18:00 daily_3month_CAMPUS_REC_2021-12-03_18:00 daily_3month_CAMPUS_REC_2021-12-28_18:00
daily_1month_CAMPUS_REC_2021-12-19_12:00 daily_3month_CAMPUS_REC_2021-10-15_18:00 daily_3month_CAMPUS_REC_2021-11-09_18:00 daily_3month_CAMPUS_REC_2021-12-04_18:00 daily_3month_CAMPUS_REC_2021-12-29_18:00
daily_1month_CAMPUS_REC_2021-12-20_12:00 daily_3month_CAMPUS_REC_2021-10-16_18:00 daily_3month_CAMPUS_REC_2021-11-10_18:00 daily_3month_CAMPUS_REC_2021-12-05_18:00 daily_3month_CAMPUS_REC_2021-12-30_18:00
daily_1month_CAMPUS_REC_2021-12-21_12:00 daily_3month_CAMPUS_REC_2021-10-17_18:00 daily_3month_CAMPUS_REC_2021-11-11_18:00 daily_3month_CAMPUS_REC_2021-12-06_18:00 daily_3month_CAMPUS_REC_2021-12-31_18:00
daily_1month_CAMPUS_REC_2021-12-22_12:00 daily_3month_CAMPUS_REC_2021-10-18_18:00 daily_3month_CAMPUS_REC_2021-11-12_18:00 daily_3month_CAMPUS_REC_2021-12-07_18:00 daily_3month_CAMPUS_REC_2022-01-01_18:00
daily_1month_CAMPUS_REC_2021-12-23_12:00 daily_3month_CAMPUS_REC_2021-10-19_18:00 daily_3month_CAMPUS_REC_2021-11-13_18:00 daily_3month_CAMPUS_REC_2021-12-08_18:00 daily_3month_CAMPUS_REC_2022-01-02_18:00
daily_1month_CAMPUS_REC_2021-12-24_12:00 daily_3month_CAMPUS_REC_2021-10-20_18:00 daily_3month_CAMPUS_REC_2021-11-14_18:00 daily_3month_CAMPUS_REC_2021-12-09_18:00 daily_3month_CAMPUS_REC_2022-01-03_18:00
daily_1month_CAMPUS_REC_2021-12-25_12:00 daily_3month_CAMPUS_REC_2021-10-21_18:00 daily_3month_CAMPUS_REC_2021-11-15_18:00 daily_3month_CAMPUS_REC_2021-12-10_18:00 daily_3month_CAMPUS_REC_2022-01-04_18:00
daily_1month_CAMPUS_REC_2021-12-26_12:00 daily_3month_CAMPUS_REC_2021-10-22_18:00 daily_3month_CAMPUS_REC_2021-11-16_18:00 daily_3month_CAMPUS_REC_2021-12-11_18:00 SIQ-743c22b777679f9ed3c08b5b9ac6bceb-restore-latest
daily_1month_CAMPUS_REC_2021-12-27_12:00 daily_3month_CAMPUS_REC_2021-10-23_18:00 daily_3month_CAMPUS_REC_2021-11-17_18:00 daily_3month_CAMPUS_REC_2021-12-12_18:00 SIQ-7b67c2c289b36a2d86a688ade445aa31-latest
daily_1month_CAMPUS_REC_2021-12-28_12:00 daily_3month_CAMPUS_REC_2021-10-24_18:00 daily_3month_CAMPUS_REC_2021-11-18_18:00 daily_3month_CAMPUS_REC_2021-12-13_18:00
daily_1month_CAMPUS_REC_2021-12-29_12:00 daily_3month_CAMPUS_REC_2021-10-25_18:00 daily_3month_CAMPUS_REC_2021-11-19_18:00 daily_3month_CAMPUS_REC_2021-12-14_18:00
daily_1month_CAMPUS_REC_2021-12-30_12:00 daily_3month_CAMPUS_REC_2021-10-26_18:00 daily_3month_CAMPUS_REC_2021-11-20_18:00 daily_3month_CAMPUS_REC_2021-12-15_18:00

Navigate to the date of the last known existence of the file and copy it to a directory of your choice

From Windows

Mount the project directory as described in https://wiki.unil.ch/ci/books/research-data-storage/page/dcsr-nas

Navigate to the project directory and right click

Screenshot 2022-01-05 at 11.37.37.png

Select Properties-> Previous Versions

Screenshot 2022-01-05 at 11.46.19.png

You can then select the snapshot date and copy the missing files or directories.

From MAC

Install the "Owlfiles - File Explorer"  free sofware from the Apple App store:

Go to the App Store ("Apple" menu on the top left of your screen):

Screenshot 2023-02-17 at 13.17.46.png

And search for "Owlfiles" and install it ("Get" button):

Screenshot 2023-02-17 at 13.18.21.png

Start "OwlFiles - File Explorer" and create a new connection as described below:

Screenshot 2022-01-05 at 11.09.48.png

Then navigate through the snapshots to the directories/files you want to restore.






Data management

This page presents the way to move data between your laptop (or a virtual machine/lab server) and the NAS.

Accessing DCSR NAS from home or UNIL network


Note: If you work from home, you first need to connect to UNIL's VPN using Pulse Secure (see VPN instructions).

From Windows

Step 1: right click on Network in File explorer and choose "Map network drive…"

image-1591798613766.png

Step 2: enter the address of the Samba server and click to finish

Ie9map01.PNG

Step 3: enter your Unil credentials and clikc "OK"

map02.PNG

Step 4: click on "This PC" and then on the new network drive

map03.PNG


From Linux

It is possible to use a GUI but there are as much solutions as desktop managers. So here is a way to mount the NAS using command line interface. Don't forget to adapt the username= field in the sudo mount command by replacing unil_username with your username at UNIL (ex. mcurie for Marie Curie)!

# First install the packages
# Debian/Ubuntu version
$ sudo apt install samba cifs-utils
# Fedora/RedHat version
$ sudo dnf install cifs-utils

# In the following, ~/nas is chosen as a mountpoint, but it could be elsewhere
$ mkdir ~/nasdcsr
$ sudo mount -t cifs -o username=unil_username,mfsymlinks,domain=ad,rw,iocharset=utf8,dir_mode=0700,file_mode=0700,uid=$(id -u) //nasdcsr.unil.ch/RECHERCHE ~/nasdcsr

# Perform the reauired operations
$ cd ~/nasdcsr
…

# When finished, don't forget to umount it
$ sudo umount ~/nasdcsr


From MacOS

Step 1: open the Finder, in Go menu, select "Connect to Server"

image-1591856270964.png

Step 2: enter the NAS url (smb://nasdcsr.unil.ch/RECHERCHE), you also may add this server as a favourite by clicking on + symbol

image-1605528817646.png

Step 3: enter your credentials

image-1605528837404.png

Step 4: navigate through the directories to find your working directory and push/fetch files as you see fit

image-1605528863048.png

Accessing DCSR NAS from CHUV network

Windows and Mac users can install the graphical desktop client from DataExpedition as well if they wish so:

Download Expedat from here, installation instruction from here, and product documentation from there.

Here is an example of ExpeDat session:

image-1591796422909.png

On that screenshot you can see:

  1. The server of the NAS
  2. The user information (login and password)
  3. The explorer for the files located on your laptop
  4. The explorer for the files located on the NAS (note that the remote path must be set according to your personal working space)
  5. The get button to fetch files from the NAS to your laptop
  6. The push button to push files from your laptop to the NAS

Sensitive data on DCSR NAS

Since november 2023, new research projects should store their data using Tresorit. Please see the detailed documentation : https://wiki.unil.ch/ci/books/research-data-storage/chapter/donnees-sensibles-tresorit 

Current user of the NAS DCSR RECHERCHE-S/Boxcryptor can continue to use this solution until June 2024.

Request a storage space for sensitive data

To request storage of sensitive data please go to the CI/DCSR "Demande de ressources de recherche" web page

https://conference.unil.ch/research-resource-requests/

and log in with your EDU-id account.

Click on "New request / Nouvelle demande"


image-1600198383347.png

image-1600198806493.pngimage-1600199651933.png


image-1600200172278.png

Don't forget you want to work with sensitive data by selecting the appropriate option above.

image-1600200548256.png

This page allows you to select the options available at DCSR:


image-1600200939342.png

You will then receive a confirmation e-mail that your resource request has been accepted.


How to connect

Mount the NAS to your laptop

Mount the RECHERCHE-S volume of the NAS (see exact addresses below and documentation), and then use a security software to access your data (mandatory, see next section):

- On MacOS:

smb://nasdcsr.unil.ch/RECHERCHE-S

or directly:

smb://nasdcsr.unil.ch/RECHERCHE-S/<ENTITY>/<FAC>/<INSTITUTE>/<PI>/<PROJECT>/D2C

example:

smb://nasdcsr.unil.ch/RECHERCHE-S/FAC/SSP/ISS/username_of_your_pi/project_shortname/D2C

- On Windows:

\\nasdcsr.unil.ch\RECHERCHE-S

or directly:

\\nasdcsr.unil.ch\RECHERCHE-S\<ENTITY>\<FAC>\<INSTITUTE>\<PI>\<PROJECT>\D2C

example:

\\ nasdcsr.unil.ch\RECHERCHE-S\FAC\SSP\ISS\username_of_your_pi\project_shortname\D2C


Mandatory: access the NAS using Boxcryptor (security software)

Request a Boxcryptor password by sending an e-mail to helpdesk@unil.ch with BoxCryptor password in the subject, you shall receive a temporary password to initialize the security software.

Then download and install the Boxcryptor from:

https://www.boxcryptor.com/en/download/

For Mac

1) Install it by dragging it to your applications folder

image-1600235716041.png

2) Confirm your installation by typing your workstation administrator password


image-1600235979332.png

For Windows

1) Double-click on the installer you just downloaded (e.g. Boxcryptor_v2.42.1333_Setup.msi)

image-1610546371006.png

2) Allow installation, accept the terms in the License Agreement and in the Privacy Policy and complete the installation


Start Boxcryptor and sign in with your UNIL e-mail and the Boxcryptor password you received from helpdesk:

image-1600236026508.png

You will have to change your password:

image-1600236441357.png


On MacOS:

Follow the wizard below to connect to your storage by selecting RECHERCHE-S (which should be already mounted before, see previous section):

image-1600236700615.png

Then, the Boxcryptor access to RECHERCHE-S will look like below. You can navigate to your data. A green symbol with a lock indicates a crypted file or folder. The full documentation can be found at https://www.boxcryptor.com/en/help/overview/macos/ :

Screenshot_2021-01-15_at_09_19_12.png

On Windows:

The official documentation is here: https://www.boxcryptor.com/en/help/overview/windows/.

To have your mounted NAS location handled by Boxcryptor you must proceed as follows:

1. From the application tray, right-click on the Boxcryptor icon and select "Settings":

image-1610711316901.png

2. Then, in "Locations", select the drive on which you mounted RECHERCHE-S:

image-1610711505477.png

Finally, please refer to the official documentation in order to encrypt files and folders :https://www.boxcryptor.com/en/help/work-with-files/windows/ to gain some understanding on how Boxcryptor works.

If you now open a FileExplorer you will see something like this:

image-1610712923875.png

You can navigate to your data. A green symbol with a lock indicated a crypted file or folder:

image-1610713306868.png


Give access to DCSR NAS files to an external collaborator

There are two administrative steps:

  1. The external collaborator must ask a UNIL account using this online form (go to "Obtenir un compte informatique personnel"). Warning: in the online form, the external collaborator have to write the name/email of the PI who is sponsoring the UNIL account in the comments section.
  2. the PI (Principal Investigator) to whom the external collaborator is related must use this application to add the collaborator into the appropriate project. Log into the application if necessary on the top right, and click on the "Manage members list / Gérer la liste de membres" icon for your project. The usernames always have 8 characters (e.g. Greta Thunberg username would be: gthunber)

The external collaborator can then access the data of the project stored on the UNIL NAS using VPN:

Moving data between different projects

At some point, you might have to move data from one project to another one for various reasons. On NAS DCSR, such move is not possible by default because of a technical limitation.

So depending on the volume of data to move, there are 2 options. But before going further, please check that quotas on destination projects are high enough to receive the data.

Low amount of data (less than hundreds of GB): copy the data

Copying the data between two projects is possible. From MacOS/Windows computer you can just copy a folder from the source project and paste it in the destination project. From Linux computer, you can use cp or rsync commands. Once the copy is terminated, you can remove the data in the source project.

Note: You have to keep in mind that in such situation the copy pass through your computer, so the speed will be limited by the weakest link (usually your upstream network link, especially if this is performed from home). Excepted for very small amount of data, this should be performed from UNIL network with wired connection, or from Curnagl cluster for those who can access it.

Large amount of data: ask DCSR

DCSR can temporarily remove the technical limitation that prevents moving date between projects. If you need to move large amount of data, you can send a ticket to helpdesk@unil.ch and write something like:

Dear DCSR,
I need to move data between the following projects on NAS DCSR:

Could you temporarily remove the limitation that prevent from moving data between the above projects?
Cheers,
Urusla Lambda

Then, you will be able to cut and paste from MacOS/Windows computer, or tu use mv command from Linux computer.