DCSR NAS
- General information
- Recovering files from the DCSR NAS
- Data management
- Sensitive data on DCSR NAS
- Give access to DCSR NAS files to an external collaborator
- Moving data between different projects
General information
How to access the DCSR NAS
If you don't have a DCSR NAS (Network Attached Storage) storage yet and you would like one, the PI (Principal Investigator) of your project needs to fill up a request application form on the following web interface:
https://conference.unil.ch/research-resource-requests/
Once the DCSR has accepted the request and the access has been granted to you by your PI, you will be able to access your DCSR NAS storage by one of the following method:
Access from a laptop
Access from the cluster
If you have access to the compute clusters of the DCSR, the data stored on the NAS is directly accessible under /nas. See:
In case you have asked for the "sensitive data" special option for your storage, please follow the specific instructions to access RECHERCHE-S on the following page:
How to manage sensitive data on UNIL NAS
General structure of the NAS
The general structure of the directories on the DCSR NAS (Network Attached Storage) is:
nasdcsr.unil.ch/RECHERCHE/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>
On mac
smb://nasdcsr.unil.ch/RECHERCHE/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>
On Windows
\\nasdcsr.unil.ch\RECHERCHE\FAC\<your_faculty>\<your_department>\<your_PI>\<your_project>
In case you have asked for the "personal data" special option for your storage, please use RECHERCHE-P with:
nasdcsr.unil.ch/RECHERCHE-P/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>
On mac
smb://nasdcsr.unil.ch/RECHERCHE-P/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>
On Windows
\\nasdcsr.unil.ch\RECHERCHE-P\FAC\<your_faculty>\<your_department>\<your_PI>\<your_project>
In case you have asked for the "sensitive data" special option for your storage, please follow the specific instructions to access RECHERCHE-S on the following page:
How to manage sensitive data on UNIL NAS
What are the /D1c, /D2c, and /LTS subdirectories?
In your project directory you will see at least a D2c and a LTS directory, with the following meanings/purposes:
<your_project>/D2c
D2c means "data 2 copies". This is where you can put your research data during the life time of your project. This NAS directory has an off-site backup (i.e the data is stored in two distinct locations). Snapshots are done on a daily basis for a rolling period of 3 months. Please note that in case you are using the HPC cluster and perform computations on this data, your are advised to copy this data to a fast storage (i.e. /scratch or /work) to speed up computations (see this page).
<your_project>/LTS
LTS means "Long Term Storage". This data will be kept for a very long time on the UNIL storage (a decade, on tapes). This is where you should put the part of your data that needs to be archived e.g. at the end of your project. When you reach this point, you will then be asked to make a description of your data ("meta-data"). See "Long-term storage of research data" section here: https://www.unil.ch/openscience/en/home/menuinst/open-research-data/gerer-les-donnees-de-recherche-research-data-management/stockage--securite.html
Users who explicitly asked for it will also have a D1c directory:
<your_project>/D1c
D1c means "data 1 copy". This data has no backup (and may be lost). This is where you can put e.g. some reference data that you downloaded from an external source, and that can easily be recovered again from the same external source, in case the data get lost in /D1c.
Long term storage at DCSR and other data repositories
Here is an overview of the options you have for Long Term Storage / Long Term Preservation (LTS/LTP) of your data (warning: french only) https://www.unil.ch/openscience/files/live/sites/openscience/files/Donnees_de_recherche/Files/Cycle_vie_Data_support_stockage.pdf
Recovering files from the DCSR NAS
Introduction
If you have deleted or corrupted a file you can recover it from the daily snapshots that are performed on the DCSR NAS
From the Curnagl login node
ssh username@curnagl.dcsr.unil.ch
cd /nas/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>/D2c
cd .snapshot
ls
daily_1month_CAMPUS_REC_2021-12-06_12:00 daily_1month_CAMPUS_REC_2021-12-31_12:00 daily_3month_CAMPUS_REC_2021-10-27_18:00 daily_3month_CAMPUS_REC_2021-11-21_18:00 daily_3month_CAMPUS_REC_2021-12-16_18:00
daily_1month_CAMPUS_REC_2021-12-07_12:00 daily_1month_CAMPUS_REC_2022-01-01_12:00 daily_3month_CAMPUS_REC_2021-10-28_18:00 daily_3month_CAMPUS_REC_2021-11-22_18:00 daily_3month_CAMPUS_REC_2021-12-17_18:00
daily_1month_CAMPUS_REC_2021-12-08_12:00 daily_1month_CAMPUS_REC_2022-01-02_12:00 daily_3month_CAMPUS_REC_2021-10-29_18:00 daily_3month_CAMPUS_REC_2021-11-23_18:00 daily_3month_CAMPUS_REC_2021-12-18_18:00
daily_1month_CAMPUS_REC_2021-12-09_12:00 daily_1month_CAMPUS_REC_2022-01-03_12:00 daily_3month_CAMPUS_REC_2021-10-30_18:00 daily_3month_CAMPUS_REC_2021-11-24_18:00 daily_3month_CAMPUS_REC_2021-12-19_18:00
daily_1month_CAMPUS_REC_2021-12-10_12:00 daily_1month_CAMPUS_REC_2022-01-04_12:00 daily_3month_CAMPUS_REC_2021-10-31_18:00 daily_3month_CAMPUS_REC_2021-11-25_18:00 daily_3month_CAMPUS_REC_2021-12-20_18:00
daily_1month_CAMPUS_REC_2021-12-11_12:00 daily_3month_CAMPUS_REC_2021-10-07_18:00 daily_3month_CAMPUS_REC_2021-11-01_18:00 daily_3month_CAMPUS_REC_2021-11-26_18:00 daily_3month_CAMPUS_REC_2021-12-21_18:00
daily_1month_CAMPUS_REC_2021-12-12_12:00 daily_3month_CAMPUS_REC_2021-10-08_18:00 daily_3month_CAMPUS_REC_2021-11-02_18:00 daily_3month_CAMPUS_REC_2021-11-27_18:00 daily_3month_CAMPUS_REC_2021-12-22_18:00
daily_1month_CAMPUS_REC_2021-12-13_12:00 daily_3month_CAMPUS_REC_2021-10-09_18:00 daily_3month_CAMPUS_REC_2021-11-03_18:00 daily_3month_CAMPUS_REC_2021-11-28_18:00 daily_3month_CAMPUS_REC_2021-12-23_18:00
daily_1month_CAMPUS_REC_2021-12-14_12:00 daily_3month_CAMPUS_REC_2021-10-10_18:00 daily_3month_CAMPUS_REC_2021-11-04_18:00 daily_3month_CAMPUS_REC_2021-11-29_18:00 daily_3month_CAMPUS_REC_2021-12-24_18:00
daily_1month_CAMPUS_REC_2021-12-15_12:00 daily_3month_CAMPUS_REC_2021-10-11_18:00 daily_3month_CAMPUS_REC_2021-11-05_18:00 daily_3month_CAMPUS_REC_2021-11-30_18:00 daily_3month_CAMPUS_REC_2021-12-25_18:00
daily_1month_CAMPUS_REC_2021-12-16_12:00 daily_3month_CAMPUS_REC_2021-10-12_18:00 daily_3month_CAMPUS_REC_2021-11-06_18:00 daily_3month_CAMPUS_REC_2021-12-01_18:00 daily_3month_CAMPUS_REC_2021-12-26_18:00
daily_1month_CAMPUS_REC_2021-12-17_12:00 daily_3month_CAMPUS_REC_2021-10-13_18:00 daily_3month_CAMPUS_REC_2021-11-07_18:00 daily_3month_CAMPUS_REC_2021-12-02_18:00 daily_3month_CAMPUS_REC_2021-12-27_18:00
daily_1month_CAMPUS_REC_2021-12-18_12:00 daily_3month_CAMPUS_REC_2021-10-14_18:00 daily_3month_CAMPUS_REC_2021-11-08_18:00 daily_3month_CAMPUS_REC_2021-12-03_18:00 daily_3month_CAMPUS_REC_2021-12-28_18:00
daily_1month_CAMPUS_REC_2021-12-19_12:00 daily_3month_CAMPUS_REC_2021-10-15_18:00 daily_3month_CAMPUS_REC_2021-11-09_18:00 daily_3month_CAMPUS_REC_2021-12-04_18:00 daily_3month_CAMPUS_REC_2021-12-29_18:00
daily_1month_CAMPUS_REC_2021-12-20_12:00 daily_3month_CAMPUS_REC_2021-10-16_18:00 daily_3month_CAMPUS_REC_2021-11-10_18:00 daily_3month_CAMPUS_REC_2021-12-05_18:00 daily_3month_CAMPUS_REC_2021-12-30_18:00
daily_1month_CAMPUS_REC_2021-12-21_12:00 daily_3month_CAMPUS_REC_2021-10-17_18:00 daily_3month_CAMPUS_REC_2021-11-11_18:00 daily_3month_CAMPUS_REC_2021-12-06_18:00 daily_3month_CAMPUS_REC_2021-12-31_18:00
daily_1month_CAMPUS_REC_2021-12-22_12:00 daily_3month_CAMPUS_REC_2021-10-18_18:00 daily_3month_CAMPUS_REC_2021-11-12_18:00 daily_3month_CAMPUS_REC_2021-12-07_18:00 daily_3month_CAMPUS_REC_2022-01-01_18:00
daily_1month_CAMPUS_REC_2021-12-23_12:00 daily_3month_CAMPUS_REC_2021-10-19_18:00 daily_3month_CAMPUS_REC_2021-11-13_18:00 daily_3month_CAMPUS_REC_2021-12-08_18:00 daily_3month_CAMPUS_REC_2022-01-02_18:00
daily_1month_CAMPUS_REC_2021-12-24_12:00 daily_3month_CAMPUS_REC_2021-10-20_18:00 daily_3month_CAMPUS_REC_2021-11-14_18:00 daily_3month_CAMPUS_REC_2021-12-09_18:00 daily_3month_CAMPUS_REC_2022-01-03_18:00
daily_1month_CAMPUS_REC_2021-12-25_12:00 daily_3month_CAMPUS_REC_2021-10-21_18:00 daily_3month_CAMPUS_REC_2021-11-15_18:00 daily_3month_CAMPUS_REC_2021-12-10_18:00 daily_3month_CAMPUS_REC_2022-01-04_18:00
daily_1month_CAMPUS_REC_2021-12-26_12:00 daily_3month_CAMPUS_REC_2021-10-22_18:00 daily_3month_CAMPUS_REC_2021-11-16_18:00 daily_3month_CAMPUS_REC_2021-12-11_18:00 SIQ-743c22b777679f9ed3c08b5b9ac6bceb-restore-latest
daily_1month_CAMPUS_REC_2021-12-27_12:00 daily_3month_CAMPUS_REC_2021-10-23_18:00 daily_3month_CAMPUS_REC_2021-11-17_18:00 daily_3month_CAMPUS_REC_2021-12-12_18:00 SIQ-7b67c2c289b36a2d86a688ade445aa31-latest
daily_1month_CAMPUS_REC_2021-12-28_12:00 daily_3month_CAMPUS_REC_2021-10-24_18:00 daily_3month_CAMPUS_REC_2021-11-18_18:00 daily_3month_CAMPUS_REC_2021-12-13_18:00
daily_1month_CAMPUS_REC_2021-12-29_12:00 daily_3month_CAMPUS_REC_2021-10-25_18:00 daily_3month_CAMPUS_REC_2021-11-19_18:00 daily_3month_CAMPUS_REC_2021-12-14_18:00
daily_1month_CAMPUS_REC_2021-12-30_12:00 daily_3month_CAMPUS_REC_2021-10-26_18:00 daily_3month_CAMPUS_REC_2021-11-20_18:00 daily_3month_CAMPUS_REC_2021-12-15_18:00
From Windows
Mount the project directory as described in https://wiki.unil.ch/ci/books/research-data-storage/page/dcsr-nas
Select Properties-> Previous Versions
You can then select the snapshot date and copy the missing files or directories.
From MAC
Install the "Owlfiles - File Explorer" free sofware from the Apple App store:
Go to the App Store ("Apple" menu on the top left of your screen):
And search for "Owlfiles" and install it ("Get" button):
Start "OwlFiles - File Explorer" and create a new connection as described below:
Data management
This page presents the way to move data between your laptop (or a virtual machine/lab server) and the NAS.
Accessing DCSR NAS from home or UNIL network
Note: If you work from home, you first need to connect to UNIL's VPN using Pulse Secure (see VPN instructions).
From Windows
Step 1: right click on Network in File explorer and choose "Map network drive…"
Step 2: enter the address of the Samba server and click to finish
Step 3: enter your Unil credentials and clikc "OK"
- username:
ad\unil_username
(for examplead\ulambda
) - password
Step 4: click on "This PC" and then on the new network drive
From Linux
It is possible to use a GUI but there are as much solutions as desktop managers. So here is a way to mount the NAS using command line interface. Don't forget to adapt the username=
field in the sudo mount
command by replacing unil_username
with your username at UNIL (ex. mcurie
for Marie Curie)!
# First install the packages
# Debian/Ubuntu version
$ sudo apt install samba cifs-utils
# Fedora/RedHat version
$ sudo dnf install cifs-utils
# In the following, ~/nas is chosen as a mountpoint, but it could be elsewhere
$ mkdir ~/nasdcsr
$ sudo mount -t cifs -o username=unil_username,mfsymlinks,domain=ad,rw,iocharset=utf8,dir_mode=0700,file_mode=0700,uid=$(id -u) //nasdcsr.unil.ch/RECHERCHE ~/nasdcsr
# Perform the reauired operations
$ cd ~/nasdcsr
…
# When finished, don't forget to umount it
$ sudo umount ~/nasdcsr
From MacOS
Step 1: open the Finder, in Go menu, select "Connect to Server"
Step 2: enter the NAS url (smb://nasdcsr.unil.ch/RECHERCHE
), you also may add this server as a favourite by clicking on + symbol
Step 3: enter your credentials
Accessing DCSR NAS from CHUV network
Windows and Mac users can install the graphical desktop client from DataExpedition as well if they wish so:
Download Expedat from here, installation instruction from here, and product documentation from there.
Here is an example of ExpeDat session:
On that screenshot you can see:
- The server of the NAS
- The user information (login and password)
- The explorer for the files located on your laptop
- The explorer for the files located on the NAS (note that the remote path must be set according to your personal working space)
- The get button to fetch files from the NAS to your laptop
- The push button to push files from your laptop to the NAS
Sensitive data on DCSR NAS
Since november 2023, new research projects should store their data using Tresorit. Please see the detailed documentation : https://wiki.unil.ch/ci/books/research-data-storage/chapter/donnees-sensibles-tresorit
Current user of the NAS DCSR RECHERCHE-S/Boxcryptor can continue to use this solution until June 2024.
Request a storage space for sensitive data
To request storage of sensitive data please go to the CI/DCSR "Demande de ressources de recherche" web page
https://conference.unil.ch/research-resource-requests/
and log in with your EDU-id account.
Click on "New request / Nouvelle demande"
Don't forget you want to work with sensitive data by selecting the appropriate option above.
This page allows you to select the options available at DCSR:
- Storage resources
- Computing resources
- Computing resources for sensitive data
You will then receive a confirmation e-mail that your resource request has been accepted.
How to connect
Mount the NAS to your laptop
Mount the RECHERCHE-S volume of the NAS (see exact addresses below and documentation), and then use a security software to access your data (mandatory, see next section):
- On MacOS:
smb://nasdcsr.unil.ch/RECHERCHE-S
or directly:
smb://nasdcsr.unil.ch/RECHERCHE-S/<ENTITY>/<FAC>/<INSTITUTE>/<PI>/<PROJECT>/D2C
example:
smb://nasdcsr.unil.ch/RECHERCHE-S/FAC/SSP/ISS/username_of_your_pi/project_shortname/D2C
- On Windows:
\\nasdcsr.unil.ch\RECHERCHE-S
or directly:
\\nasdcsr.unil.ch\RECHERCHE-S\<ENTITY>\<FAC>\<INSTITUTE>\<PI>\<PROJECT>\D2C
example:
\\ nasdcsr.unil.ch\RECHERCHE-S\FAC\SSP\ISS\username_of_your_pi\project_shortname\D2C
Mandatory: access the NAS using Boxcryptor (security software)
Request a Boxcryptor password by sending an e-mail to helpdesk@unil.ch with BoxCryptor password in the subject, you shall receive a temporary password to initialize the security software.
Then download and install the Boxcryptor from:
https://www.boxcryptor.com/en/download/
For Mac
1) Install it by dragging it to your applications folder |
2) Confirm your installation by typing your workstation administrator password |
For Windows
1) Double-click on the installer you just downloaded (e.g. Boxcryptor_v2.42.1333_Setup.msi) |
2) Allow installation, accept the terms in the License Agreement and in the Privacy Policy and complete the installation |
Start Boxcryptor and sign in with your UNIL e-mail and the Boxcryptor password you received from helpdesk:
You will have to change your password:
On MacOS:
Follow the wizard below to connect to your storage by selecting RECHERCHE-S (which should be already mounted before, see previous section):
Then, the Boxcryptor access to RECHERCHE-S will look like below. You can navigate to your data. A green symbol with a lock indicates a crypted file or folder. The full documentation can be found at https://www.boxcryptor.com/en/help/overview/macos/ :
On Windows:
The official documentation is here: https://www.boxcryptor.com/en/help/overview/windows/.
To have your mounted NAS location handled by Boxcryptor you must proceed as follows:
1. From the application tray, right-click on the Boxcryptor icon and select "Settings":
2. Then, in "Locations", select the drive on which you mounted RECHERCHE-S:
Finally, please refer to the official documentation in order to encrypt files and folders :https://www.boxcryptor.com/en/help/work-with-files/windows/ to gain some understanding on how Boxcryptor works.
If you now open a FileExplorer you will see something like this:
Give access to DCSR NAS files to an external collaborator
There are two administrative steps:
- The external collaborator must ask a UNIL account using this online form (go to "Obtenir un compte informatique personnel"). Warning: in the online form, the external collaborator have to write the name/email of the PI who is sponsoring the UNIL account in the comments section.
- the PI (Principal Investigator) to whom the external collaborator is related must use this application to add the collaborator into the appropriate project. Log into the application if necessary on the top right, and click on the "Manage members list / Gérer la liste de membres" icon for your project. The usernames always have 8 characters (e.g. Greta Thunberg username would be: gthunber)
The external collaborator can then access the data of the project stored on the UNIL NAS using VPN:
Moving data between different projects
At some point, you might have to move data from one project to another one for various reasons. On NAS DCSR, such move is not possible by default because of a technical limitation.
So depending on the volume of data to move, there are 2 options. But before going further, please check that quotas on destination projects are high enough to receive the data.
Low amount of data (less than hundreds of GB): copy the data
Copying the data between two projects is possible. From MacOS/Windows computer you can just copy a folder from the source project and paste it in the destination project. From Linux computer, you can use cp or rsync commands. Once the copy is terminated, you can remove the data in the source project.
Note: You have to keep in mind that in such situation the copy pass through your computer, so the speed will be limited by the weakest link (usually your upstream network link, especially if this is performed from home). Excepted for very small amount of data, this should be performed from UNIL network with wired connection, or from Curnagl cluster for those who can access it.
Large amount of data: ask DCSR
DCSR can temporarily remove the technical limitation that prevents moving date between projects. If you need to move large amount of data, you can send a ticket to helpdesk@unil.ch and write something like:
Dear DCSR,
I need to move data between the following projects on NAS DCSR:
- FAC/FBM/ABC/ulambda/project1
- FAC/FBM/ABC/ulambda/project2
- FAC/FBM/ABC/ulambda/project3
Could you temporarily remove the limitation that prevent from moving data between the above projects?
Cheers,
Urusla Lambda
Then, you will be able to cut and paste from MacOS/Windows computer, or tu use mv command from Linux computer.