Shared folders
If you work in a group or team and use large volumes of data, it is useful to work within a shared space. User can thus share inputs to their models and make their outputs also easily available. This article explains how to do so within the Lustre file system, that presently supports the HPC.
Identifying a suitable user group
Users access the HPC cluster with their WUR-wide account. This means that all the membership information is also available in the HPC system. To check of which groups is your user a member of, use the following command:
groups <username>
This can result in a rather long list, reflecting permissions in the overall WUR systems. Within these groups you must then identify the one that is closer to match the team or group with which you wish to collaborate.
For instance, if I wish to work together with colleagues at ISRIC, I can search within my groups an appropriate match:
groups duque004 | grep isric
In my case the group des-isric-users looked appropriate. Then next step is to confirm if the other users in my team are also members of the group.
The Lustre file system is accessible in the /lustre
folder and then divided into the /backup
and /nobackup
sections (corresponding to the different usage plans). Inside each of these folders there is a sub-folder named SHARED
in which users are to create their own assets.
You start by creating a folder in this space; it is probably better if it matches the name of your group or team, e.g.:
mkdir /lustre/nobackup/SHARED/myTeamWorkspace
Or in alternative:
cd /lustre/nobackup/SHARED
mkdir myTeamWorkspace
Setting permissions
Three basic steps are involved in stepping permissions correctly:
1. Pass the ownership of the group to the team. In the example below it is applied recursively to all sub-folder and files that may exist:
chgrp -R my-team-group myTeamWorkspace
2. Concede read/write permissions to the group. This allows other members of the group to read and write in the shared folder. If you wish other team members to only read from the folder then remove the w
character from the +rw
bit:
chmod -R g+rw myTeamWorkspace
3. Set default ownership within the group. This guarantees that any new files or folders created within the shared folder are owned by default owned by your team group:
chmod -R g+s myTeamWorkspace
In case the contents of the shared are sensitive or private, and should be accessed by your team, you can block access from any other users with the following command:
chmod -R o-rw myTeamWorkspace
Further reading
An Introduction to Linux Permissions