Data Transfer Methods

There are several ways to move data between your own computer (or other systems) and Anunna. Pick the one that matches your operating system and workflow.

File transfer over SSH

Copying files to/from the cluster: scp

From any POSIX-compliant system (Linux/macOS) terminal, files and folders can be transferred to and from the cluster using the SSH-based copy command scp (secure copy).

The syntax of the scp command is in from-to order:

scp <source> <destination>

For example, copying a folder and its contents to the cluster:

scp -pr /home/WUR/[username]/folder_to_transfer [username]@login.anunna.wur.nl:/lustre/scratch/WUR/ABGC/

This example assumes a user in the ABGC group; see the Lustre layout for how the /lustre paths are structured. The -p flag preserves file metadata such as timestamps, and -r allows recursive copying. Further options can be found in the man pages:

man scp

rsync

The rsync protocol, like scp, allows command-line copying of files. Unlike scp, it only transfers the files that have changed between the two systems — it synchronises them, hence the name — which makes it very well suited to regular backups and repeated syncs. Like scp, the syntax is in from-to order.

rsync <source> <destination>

For example:

rsync -av /home/WUR/[username]/folder_to_transfer [username]@login.anunna.wur.nl:/lustre/scratch/WUR/ABGC/

The -a flag preserves file metadata and copies recursively, among other things, and -v gives verbose output. Further options can be found in the man pages:

man rsync

WinSCP

WinSCP is a free and open-source (S)FTP client for Microsoft Windows. By providing the hostname (login.anunna.wur.nl), your username and password, and using the SFTP protocol on port 22, you can log in and transfer files between your local PC and the cluster.

FileZilla

FileZilla is a free and open-source graphical (S)FTP client, available for Linux, macOS, and Windows. By providing the address, username, password, and server type (Unix — see Site Manager → Advanced), you can transfer files between a local system and the cluster. The graphical interface also makes it easy to browse files on Anunna. Detailed instructions can be found on the FileZilla wiki.

Samba/CIFS based protocols

The Common Internet File System (CIFS) is commonly used in and between Windows systems for file sharing. It is only available to clients within WURnet, or when connected through the VPN.

There are two mount points available:

1. your home folder ( \\cifs.anunna.wur.nl\[username] )
2. the Lustre mount ( \\cifs.anunna.wur.nl\lustre )

You can enter these in the location bar of File Explorer.

If you have issues with logging in, the problem is probably your username. In that case, there are a couple of options you could try:

1. Use \WUR\[userid] as username
2. Use [userid] as username
3. Use [WUR e-mail] as username

All of these can be entered if you select "connect using different credentials" in the connection box.

rclone to OneDrive

To easily transfer data to OneDrive, you can use rclone. It is available through the modules system:

module load rclone

To access your own OneDrive space, you first need to configure rclone. In one of the steps, rclone starts a web server, so we first have to use SSH to create a tunnel to the login server so that we can reach that web server from our own computer.

First, connect to Anunna with the tunnel:

ssh user@login.anunna.wur.nl -L53682:127.0.0.1:53682

• Windows PuTTY users: see Putty SSH Tunnelling.

Then load the rclone module (see above) and start the configuration:

rclone config

Create a new config (we use the name remote here, but you can choose your own) and select onedrive as the type. Use the defaults for the next few steps, and choose global for the region. Do not start the advanced config, and do use auto config ("Use web browser to automatically authenticate rclone with remote?"). Copy the URL into your local web browser and enter your WUR credentials when asked. In the next steps select OneDrive; the WUR uses the Business version. If all is well, you now have a working config.

To test it:

rclone tree remote:

If all is well, you see the contents of your own OneDrive. Now you can create folders and copy data to and from your OneDrive:

rclone copy --create-empty-src-dirs --copy-links --progress bin/ remote:test

Another handy tool is rclone ncdu: https://rclone.org/commands/rclone_ncdu/

As soon as you are done, please remove your config:

rclone config delete remote

Anunna is relatively safe, but better safe than sorry.

See also

• Login Nodes
• Storage Systems Overview
• Logging in over SSH
• SSH without password

External links

• WinSCP homepage
• FileZilla homepage
• The Common Internet File System (CIFS) on Wikipedia
• Adding Microsoft OneDrive — rclone documentation