Storage Systems Overview: Difference between revisions

From HPCwiki
Jump to navigation Jump to search
← Older edit
Haars0011 (talk | contribs)
Bureaucrats, Administrators
261 edits
VisualWikitext
Dawes0011 (talk | contribs)
24 edits
No edit summary
IA migration §6: rewrite Filesystems as the storage overview / landing page; detail content split into Home Directory, Compute Storage, Shared Storage, Archival Storage, Backup Policy, Quotas (via update-page on MediaWiki MCP Server)
 
(14 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Anunna currently has multiple filesystem mounts that are available cluster-wide:
Anunna provides several storage areas, each meant for a different purpose. They differ in size, speed, cost, and — importantly — whether they are backed up. This page is a map of the storage landscape; follow the links for the details of each area.


== Global ==
== Storage areas at a glance ==
* /home - This mount uses NFS to mount the home directories directly from nfs01. Each user has a 200G quota for this filesystem, as it is regularly backed up to tape, and can reliably be restored from up to a week's history.


* /shared - This mount provides a consistent set of binaries for the entire cluster.
{| class="wikitable"
|-
! Area !! Use it for !! Backed up? !! Details
|-
| <code>/home</code> || Personal files: programs, scripts, configuration, small results. Reliable but slower. || Yes || [[Home Directory]]
|-
| <code>/lustre/nobackup</code> || The default area for active compute work. Large and fast. || No || [[Compute Storage]]
|-
| <code>/lustre/backup</code> || Compute data that also needs a safety copy. || Yes (disaster-recovery copy) || [[Compute Storage]]
|-
| <code>/lustre/scratch</code> || Short-lived working data. || No (auto-purged) || [[Compute Storage]]
|-
| <code>/lustre/shared</code> || Data shared between people or groups. || Yes || [[Shared Storage]]
|-
| <code>/archive</code> || Keeping data you are no longer computing on (login nodes, WUR users). Cheaper than Lustre. || Yes || [[Archival Storage]]
|-
| <code>/tmp</code>, <code>/dev/shm</code> || Fast, node-local scratch for the lifetime of a job. || No || [[Compute Storage#Node-local storage|Compute Storage]]
|}


* /lustre - This large mount uses the Lustre filesystem to provide files from multiple redundant servers. Access is provided per group, thus:
There is also a read-only <code>/shared</code> mount that provides a consistent set of binaries and configuration for the whole cluster; it is maintained centrally and is not for your own data.
/lustre/[level]/[partner]/[unit]
e.g.
/lustre/backup/WUR/ABGC/
It comprises of two major parts (and some minor):
* /lustre/backup - In case of disaster, this data is stored a second time on a separate machine. Whilst this backup is purely in case of complete tragedy (such as some immense filesystem error, or multiple component failure), it can potentially be used to revert mistakes if you are very fast about reporting them. There is however no guarantee of this service.
* /lustre/nobackup - This is the 'normal' filesystem for Lustre - no backups, just stored on the filesystem. Without having a backup needed, the cost of data here is not as much as under /lustre/backup, but in case of disaster cannot be recivered.
* /lustre/shared - Same as /lustre/backup, except publicly available. This is where truly shared data lives that isn't assigned to a specific group.


And additionally:
== Choosing where to put your data ==
* /lustre/scratch - A separated, low resilience filesystem. Files here may be removed after some time if the filesystem gets too full (Typically 30 days). You should tidy up this data yourself once work is complete.


=== Private shared directories ===
* '''Running jobs?''' Read and write on [[Compute Storage|Lustre]] (<code>/lustre/nobackup</code> by default), not on your home directory.
If you are working with a group of users on a similar project, you might consider making a [[Shared_folders|Shared directory]] to coordinate. Information on how to do so is in the linked article.
* '''Keeping something safe?''' Your [[Home Directory|home directory]] and the <code>/lustre/backup</code> tier are backed up; <code>/lustre/nobackup</code> and <code>/lustre/scratch</code> are not. See [[Backup Policy]].
 
* '''Done computing, but want to keep the data?''' Move it to [[Archival Storage|/archive]], where long-term data is archived to tape.
== Local ==
* '''Sharing with a team?''' See [[Shared Storage]].
Specific to certain machines are some other filesystems that are available to you:
* /archive - an archive mount only accessible from the login nodes. Files here are sent to the Isilon for deeper storage. The cost of storing data here is much less than on the Lustre, but it cannot be used for compute work. This location is only available to WUR users. Files are able to be reverted via snapshot, and there is a separated backup, however this only comes in fortnightly (14 day) intervals.
 
* /tmp - On each worker node there is a /tmp mount that can be used for temporary local caching. Be advised that you should clean this up, lest your files become a hindrance to other users. You can request a node with free space in your sbatch script like so:
<pre>
#SBATCH --tmp=<required space>
</pre>
 
 
* /dev/shm - On each worker you may also create a virtual filesystem directly into memory, for extremely fast data access. Be advised that this will count against the memory used for your job, but it is also the fastest available filesystem if needed.


Storage is not free; see [[Tariffs]] for costs, and [[Quotas]] for the limits that apply.


== See also ==
== See also ==
* [[Tariffs | Costs associated with resource usage]]
* [[Home Directory]]
 
* [[Compute Storage]]
== External links ==
* [[Shared Storage]]
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]
* [[Archival Storage]]
* [[Backup Policy]]
* [[Quotas]]
* [[Tariffs]]

Latest revision as of 11:54, 18 June 2026

Anunna provides several storage areas, each meant for a different purpose. They differ in size, speed, cost, and — importantly — whether they are backed up. This page is a map of the storage landscape; follow the links for the details of each area.

Storage areas at a glance

Area Use it for Backed up? Details
/home Personal files: programs, scripts, configuration, small results. Reliable but slower. Yes Home Directory
/lustre/nobackup The default area for active compute work. Large and fast. No Compute Storage
/lustre/backup Compute data that also needs a safety copy. Yes (disaster-recovery copy) Compute Storage
/lustre/scratch Short-lived working data. No (auto-purged) Compute Storage
/lustre/shared Data shared between people or groups. Yes Shared Storage
/archive Keeping data you are no longer computing on (login nodes, WUR users). Cheaper than Lustre. Yes Archival Storage
/tmp, /dev/shm Fast, node-local scratch for the lifetime of a job. No Compute Storage

There is also a read-only /shared mount that provides a consistent set of binaries and configuration for the whole cluster; it is maintained centrally and is not for your own data.

Choosing where to put your data

  • Running jobs? Read and write on Lustre (/lustre/nobackup by default), not on your home directory.
  • Keeping something safe? Your home directory and the /lustre/backup tier are backed up; /lustre/nobackup and /lustre/scratch are not. See Backup Policy.
  • Done computing, but want to keep the data? Move it to /archive, where long-term data is archived to tape.
  • Sharing with a team? See Shared Storage.

Storage is not free; see Tariffs for costs, and Quotas for the limits that apply.

See also

Retrieved from "https://wiki.anunna.wur.nl/index.php?title=Storage_Systems_Overview&oldid=2819"