Data management

This page presents the way to move data between the different locations used for either processing or storing data. The three main locations are hereafter referred to as: laptop, cluster, and NAS.

Laptop <-> Cluster

Note: If you work from home, you first need to connect to Unil's VPN using Pulse Secure (see VPN instructions).

To copy data between your laptop and a cluster you will have to use scp (secure copy).

Linux/Mac

Linux and Mac users simply have to open a terminal.

From laptop to cluster: $ scp /path/to/fileToCopy <username>@<server>:/path/to/dest

From cluster to laptop: $ scp <username>@<server>:/path/to/fileToCopy /path/to/dest

Where:

<username> your UNIL username
<cluster> wally or axiom
<server> <cluster>-front1.unil.ch

Examples

To copy file "file.txt" to your home directory:

scp file.txt ulambda@wally-front1.unil.ch:~

To copy the entire directory "dirA" to your home directory:

scp -r dirA ulambda@wally-front1.unil.ch:~

Windows

Windows users can use WinSCP.

Step 1: create a new session

image-1591861598320.png

Step 2: enter the configuration elements (SCP file protocol, hostname, and credentials). Finally, save the session details

 

image-1591861942154.png

Step 3: enter an explicit session name

image-1591862150318.png

Step 4: login

image-1591862176391.png

Step 5: enter your password

image-1591862202616.png

Step 6: navigate through folders (1 is for local files, 2 is for cluster files) and copy the files in one way or the other

image-1591862220621.png

 

Laptop <-> NAS

Note: If you work from home, you first need to connect to Unil's VPN using Pulse Secure (see VPN instructions).

Samba server

Windows

Step 1: right click on Network in File explorer and choose "Map network drive…"

image-1591798613766.png

Step 2: enter the address of the Samba server and click to finish

Ie9map01.PNG

Step 3: enter your Unil credentials and clikc "OK"

map02.PNG

Step 4: click on "This PC" and then on the new network drive

map03.PNG

 

Linux

It is possible to use a GUI but there are as much solutions as desktop managers. So here is a way to mount the NAS using command line interface. Don't forget to adapt the username= field in the sudo mount command!

# First install the packages
# Debian/Ubuntu version
$ sudo apt install samba cifs-utils
# Fedora/RedHat version
$ sudo dnf install cifs-utils

# In the following, ~/nas is chosen as a mountpoint, but it could be elsewhere
$ mkdir ~/nasdcsr
$ sudo mount -t cifs -o username=unil_username,domain=ad,rw,iocharset=utf8,dir_mode=0700,file_mode=0700,uid=$(id -u) //nasdcsr.unil.ch/RECHERCHE ~/nasdcsr

# Perform the reauired operations
$ cd ~/nasdcsr
…

# When finished, don't forget to umount it
$ sudo umount ~/nasdcsr

 

Mac

Step 1: open the Finder, in Go menu, select "Connect to Server"

image-1591856270964.png

Step 2: enter the NAS url (smb://nasdcsr.unil.ch/RECHERCHE), you also may add this server as a favourite by clicking on + symbol

image-1605528817646.png

Step 3: enter your credentials

image-1605528837404.png

Step 4: navigate through the directories to find your working directory and push/fetch files as you see fit

image-1605528863048.png

Cluster <-> NAS

Note: If you work from home, you first need to connect to Unil's VPN using Pulse Secure (see VPN instructions).

From the login nodes the NAS is available under /nas. The folder hierarchy is:

/nas/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>

Cluster -> NAS

To copy a file to the new NAS:

cp /path/to/file /nas/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>

To copy a folder to the new NAS:

cp -r /path/to/folder /nas/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>

For more complex operations, consider using rsync. For the documentation see the man page:

man rsync

or check out this link.

NAS -> cluster

As above, just swapping the source and destination:

cp /nas/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>/file /path/to/dest
cp -r /nas/FAC/<your_faculty>/<your_department>/<your_PI>/<your_project>/folder /path/to/dest

Accessing DCSR NAS from CHUV network

Windows and Mac users can install the graphical desktop client from DataExpedition as well if they wish so:

Download Expedat from here, installation instruction from here, and product documentation from there.

Here is an example of ExpeDat session: