Data Lifecycle Policy: Difference between revisions
IA migration §6: new Data Lifecycle Policy (P2.13) (via create-page on MediaWiki MCP Server) |
No edit summary |
||
| Line 1: | Line 1: | ||
Data on Anunna | Data stored on Anunna should be managed throughout its lifecycle: from active working data, to data that needs to be retained but is no longer actively used, to data that is no longer needed. | ||
Good data lifecycle management keeps storage available for active research, makes efficient use of the different storage tiers and helps keep costs under control. | |||
As a general rule: keep active data on Lustre, move data you want to retain to Archive, and remove data you no longer need. | |||
== Active data == | == Active data == | ||
Data that you are actively working with should be stored on the fast [[Compute Storage|Lustre]] filesystem. | |||
Temporary and intermediate data can be stored in <code>/lustre/scratch</code>. Scratch storage is intended for temporary working data and is not guaranteed to be retained. Data in Scratch may be automatically removed when storage capacity becomes limited. | |||
Do not use Scratch as the only location for data that you need to keep. See [[Compute Storage]] and [[Backup Policy]] for more information. | |||
== Data you want to keep == | |||
When data is no longer actively used but still needs to be retained, move it to [[Archival Storage]]. | |||
Data in the archival storage system that is no longer actively accessed can be moved automatically to long-term tape storage by the [[Archival Storage#Tapeworm|Tapeworm]] service. This frees up capacity on the faster storage tiers while keeping the data available for future use. | |||
== | == Data you no longer need == | ||
Delete intermediate and | Delete temporary, intermediate and obsolete data as soon as it is no longer needed. This frees up storage capacity and helps keep the Anunna environment efficient. | ||
Before deleting data, make sure that it is no longer required and that any necessary copies or backups are available. See the [[Backup Policy]] for information about backup coverage and retention. | |||
== See also == | == See also == | ||
Latest revision as of 15:51, 28 August 2026
Data stored on Anunna should be managed throughout its lifecycle: from active working data, to data that needs to be retained but is no longer actively used, to data that is no longer needed.
Good data lifecycle management keeps storage available for active research, makes efficient use of the different storage tiers and helps keep costs under control.
As a general rule: keep active data on Lustre, move data you want to retain to Archive, and remove data you no longer need.
Active data
Data that you are actively working with should be stored on the fast Lustre filesystem.
Temporary and intermediate data can be stored in /lustre/scratch. Scratch storage is intended for temporary working data and is not guaranteed to be retained. Data in Scratch may be automatically removed when storage capacity becomes limited.
Do not use Scratch as the only location for data that you need to keep. See Compute Storage and Backup Policy for more information.
Data you want to keep
When data is no longer actively used but still needs to be retained, move it to Archival Storage.
Data in the archival storage system that is no longer actively accessed can be moved automatically to long-term tape storage by the Tapeworm service. This frees up capacity on the faster storage tiers while keeping the data available for future use.
Data you no longer need
Delete temporary, intermediate and obsolete data as soon as it is no longer needed. This frees up storage capacity and helps keep the Anunna environment efficient.
Before deleting data, make sure that it is no longer required and that any necessary copies or backups are available. See the Backup Policy for information about backup coverage and retention.