Transfert S3 DCSR to other support
Data in the S3 DCSR should be transfert to another file system as soon as possible. There is no backup for S3 data. This documentation describes the transfert using Curnagl cluster and the rclone command.
Requirements
- Have an account in the cluster
- Enough space in NAS or work to transfert the data
Rclone configuration
Use a text editor to create a configuration file in your home directory. Be sure to replace the S3 server name and the cryptographic key values with the ones sent in the email S3 form DCSR.
mkdir -p ~/.config/rclone
nano ~/.config/rclone/rclone.conf
The configuration file should look like this :
[s3-dci-ro]
type = s3
provider = Other
access_key_id = T******************M
secret_access_key = S**************************************i
region =
endpoint = https://scl-s3.unil.chFor many different S3 tools, the pair of authentication/cryptographic keys have different names. For Rclone, they are named "access_key_id" and "secret_access_key". Corresponding respectively to "Access key" and "Private key" in the mail sent by DCSR.
Next, secure your key file:
chmod 600 ~/.config/rclone/rclone.conf
Now, s3-dci-ro: and s3-dci-rw: are two S3 configured connection aliases that you can use in Rclone without repeating the connection information in the CLI.
s3-dci-ro: In this connection alias, the cryptographic keys are assigned to a user attached to a read-only policy on the S3 cluster. This prevents you from modifying or accidentally deleting your source data when using this connection alias.
s3-dci-rw: This connection alias allows all the S3 commands on your S3 datasets, including the write and delete commands. Use it very carefully when you are performing sync or delete operations, as there are no backups or snapshots on the S3 cluster to protect the data. Deleted files are permanently lost.