Storage Systems Overview

From HPCwiki
Jump to navigation Jump to search

Anunna provides several storage areas, each meant for a different purpose. They differ in size, speed, cost, and — importantly — whether they are backed up. This page is a map of the storage landscape; follow the links for the details of each area.

Storage areas at a glance

Area Use it for Backed up? Details
/home Personal files: programs, scripts, configuration, small results. Reliable but slower. Yes Home Directory
/lustre/nobackup The default area for active compute work. Large and fast. No Compute Storage
/lustre/backup Compute data that also needs a safety copy. Yes (disaster-recovery copy) Compute Storage
/lustre/scratch Short-lived working data. No (auto-purged) Compute Storage
/lustre/shared Data shared between people or groups. Yes Shared Storage
/archive Keeping data you are no longer computing on (login nodes, WUR users). Cheaper than Lustre. Yes Archival Storage
/tmp, /dev/shm Fast, node-local scratch for the lifetime of a job. No Compute Storage

There is also a read-only /shared mount that provides a consistent set of binaries and configuration for the whole cluster; it is maintained centrally and is not for your own data.

Choosing where to put your data

  • Running jobs? Read and write on Lustre (/lustre/nobackup by default), not on your home directory.
  • Keeping something safe? Your home directory and the /lustre/backup tier are backed up; /lustre/nobackup and /lustre/scratch are not. See Backup Policy.
  • Done computing, but want to keep the data? Move it to /archive, where long-term data is archived to tape.
  • Sharing with a team? See Shared Storage.

Storage is not free; see Tariffs for costs, and Quotas for the limits that apply.

See also