https://wiki.anunna.wur.nl/index.php?action=history&feed=atom&title=Data_Lifecycle_Policy
Data Lifecycle Policy - Revision history
2026-09-29T06:35:29Z
Revision history for this page on the wiki
MediaWiki 1.45.3
https://wiki.anunna.wur.nl/index.php?title=Data_Lifecycle_Policy&diff=3165&oldid=prev
Itter003 at 15:51, 28 August 2026
2026-08-28T15:51:43Z
<p></p>
<table style="background-color: #fff; color: #202122;" data-mw="interface">
<col class="diff-marker" />
<col class="diff-content" />
<col class="diff-marker" />
<col class="diff-content" />
<tr class="diff-title" lang="en">
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td>
<td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 15:51, 28 August 2026</td>
</tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1">Line 1:</td>
<td colspan="2" class="diff-lineno">Line 1:</td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>Data on Anunna <del style="font-weight: bold; text-decoration: none;">moves through a </del>lifecycle: active working data, data <del style="font-weight: bold; text-decoration: none;">you keep </del>but no longer actively <del style="font-weight: bold; text-decoration: none;">use</del>, <del style="font-weight: bold; text-decoration: none;">and </del>data that is no longer needed. <del style="font-weight: bold; text-decoration: none;">Managing that </del>lifecycle keeps storage available and costs under control.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>Data <ins style="font-weight: bold; text-decoration: none;">stored </ins>on Anunna <ins style="font-weight: bold; text-decoration: none;">should be managed throughout its </ins>lifecycle: <ins style="font-weight: bold; text-decoration: none;">from </ins>active working data, <ins style="font-weight: bold; text-decoration: none;">to </ins>data <ins style="font-weight: bold; text-decoration: none;">that needs to be retained </ins>but <ins style="font-weight: bold; text-decoration: none;">is </ins>no longer actively <ins style="font-weight: bold; text-decoration: none;">used</ins>, <ins style="font-weight: bold; text-decoration: none;">to </ins>data that is no longer needed.</div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">Good data </ins>lifecycle <ins style="font-weight: bold; text-decoration: none;">management </ins>keeps storage available <ins style="font-weight: bold; text-decoration: none;">for active research, makes efficient use of the different storage tiers </ins>and <ins style="font-weight: bold; text-decoration: none;">helps keep </ins>costs under control<ins style="font-weight: bold; text-decoration: none;">.</ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">As a general rule: keep active data on Lustre, move data you want to retain to Archive, and remove data you no longer need</ins>.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Active data ==</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Active data ==</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">While </del>you are working <del style="font-weight: bold; text-decoration: none;">on it, data lives </del>on the fast [[Compute Storage|Lustre]] filesystem. Temporary <del style="font-weight: bold; text-decoration: none;">working </del>data <del style="font-weight: bold; text-decoration: none;">on </del><code>/lustre/scratch</code> is not <del style="font-weight: bold; text-decoration: none;">kept indefinitely — it </del>may be <del style="font-weight: bold; text-decoration: none;">purged </del>automatically <del style="font-weight: bold; text-decoration: none;">once </del>the <del style="font-weight: bold; text-decoration: none;">filesystem fills up (see </del>[[Compute Storage]] and [[Backup Policy]]<del style="font-weight: bold; text-decoration: none;">)</del>.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">Data that </ins>you are <ins style="font-weight: bold; text-decoration: none;">actively </ins>working <ins style="font-weight: bold; text-decoration: none;">with should be stored </ins>on the fast [[Compute Storage|Lustre]] filesystem.</div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>Temporary <ins style="font-weight: bold; text-decoration: none;">and intermediate </ins>data <ins style="font-weight: bold; text-decoration: none;">can be stored in </ins><code>/lustre/scratch</code><ins style="font-weight: bold; text-decoration: none;">. Scratch storage is intended for temporary working data and </ins>is not <ins style="font-weight: bold; text-decoration: none;">guaranteed to be retained. Data in Scratch </ins>may be automatically <ins style="font-weight: bold; text-decoration: none;">removed when storage capacity becomes limited.</ins></div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">Do not use Scratch as </ins>the <ins style="font-weight: bold; text-decoration: none;">only location for data that you need to keep. See </ins>[[Compute Storage]] and [[Backup Policy]] <ins style="font-weight: bold; text-decoration: none;">for more information</ins>.</div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div> </div></td></tr>
<tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">== Data you want to keep ==</ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">== Archiving </del>data <del style="font-weight: bold; text-decoration: none;">you keep ==</del></div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">When </ins>data <ins style="font-weight: bold; text-decoration: none;">is no longer actively used but still needs to be retained, move it to [[Archival Storage]].</ins></div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">When you are finished with data but want to keep it, move it to [[Archival Storage|</del>archival storage<del style="font-weight: bold; text-decoration: none;">]]. Datasets there </del>that <del style="font-weight: bold; text-decoration: none;">are </del>no longer actively <del style="font-weight: bold; text-decoration: none;">used are </del>moved to long-term tape <del style="font-weight: bold; text-decoration: none;">archive automatically </del>by the [[Archival Storage#Tapeworm|Tapeworm]] service<del style="font-weight: bold; text-decoration: none;">, freeing </del>the <del style="font-weight: bold; text-decoration: none;">warm </del>storage <del style="font-weight: bold; text-decoration: none;">tier</del>.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">Data in the </ins>archival storage <ins style="font-weight: bold; text-decoration: none;">system </ins>that <ins style="font-weight: bold; text-decoration: none;">is </ins>no longer actively <ins style="font-weight: bold; text-decoration: none;">accessed can be </ins>moved <ins style="font-weight: bold; text-decoration: none;">automatically </ins>to long-term tape <ins style="font-weight: bold; text-decoration: none;">storage </ins>by the [[Archival Storage#Tapeworm|Tapeworm]] service<ins style="font-weight: bold; text-decoration: none;">. This frees up capacity on </ins>the <ins style="font-weight: bold; text-decoration: none;">faster </ins>storage <ins style="font-weight: bold; text-decoration: none;">tiers while keeping the data available for future use</ins>.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>== <del style="font-weight: bold; text-decoration: none;">Removing data </del>==</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>== <ins style="font-weight: bold; text-decoration: none;">Data you no longer need </ins>==</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>Delete intermediate and <del style="font-weight: bold; text-decoration: none;">temporary </del>data <del style="font-weight: bold; text-decoration: none;">you </del>no longer <del style="font-weight: bold; text-decoration: none;">need — it keeps </del>the <del style="font-weight: bold; text-decoration: none;">cluster usable and your costs down. Remember that [[Backup Policy|backups]] are limited, so be sure before you delete</del>.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>Delete <ins style="font-weight: bold; text-decoration: none;">temporary, </ins>intermediate and <ins style="font-weight: bold; text-decoration: none;">obsolete </ins>data <ins style="font-weight: bold; text-decoration: none;">as soon as it is </ins>no longer <ins style="font-weight: bold; text-decoration: none;">needed. This frees up storage capacity and helps keep </ins>the <ins style="font-weight: bold; text-decoration: none;">Anunna environment efficient</ins>.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"><!-- TODO: add the formal data-retention policy if one exists — how long different </del>data is <del style="font-weight: bold; text-decoration: none;">kept, </del>any <del style="font-weight: bold; text-decoration: none;">maximum </del>retention<del style="font-weight: bold; text-decoration: none;">, and what happens to data when an account or project closes</del>. <del style="font-weight: bold; text-decoration: none;">--></del></div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">Before deleting </ins>data<ins style="font-weight: bold; text-decoration: none;">, make sure that it </ins>is <ins style="font-weight: bold; text-decoration: none;">no longer required and that </ins>any <ins style="font-weight: bold; text-decoration: none;">necessary copies or backups are available. See the [[Backup Policy]] for information about backup coverage and </ins>retention.</div></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr>
<tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== See also ==</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== See also ==</div></td></tr>
</table>
Itter003
https://wiki.anunna.wur.nl/index.php?title=Data_Lifecycle_Policy&diff=2931&oldid=prev
Haars001: IA migration §6: new Data Lifecycle Policy (P2.13) (via create-page on MediaWiki MCP Server)
2026-06-19T12:15:53Z
<p>IA migration §6: new Data Lifecycle Policy (P2.13) (via create-page on MediaWiki MCP Server)</p>
<p><b>New page</b></p><div>Data on Anunna moves through a lifecycle: active working data, data you keep but no longer actively use, and data that is no longer needed. Managing that lifecycle keeps storage available and costs under control.<br />
<br />
== Active data ==<br />
<br />
While you are working on it, data lives on the fast [[Compute Storage|Lustre]] filesystem. Temporary working data on <code>/lustre/scratch</code> is not kept indefinitely — it may be purged automatically once the filesystem fills up (see [[Compute Storage]] and [[Backup Policy]]).<br />
<br />
== Archiving data you keep ==<br />
<br />
When you are finished with data but want to keep it, move it to [[Archival Storage|archival storage]]. Datasets there that are no longer actively used are moved to long-term tape archive automatically by the [[Archival Storage#Tapeworm|Tapeworm]] service, freeing the warm storage tier.<br />
<br />
== Removing data ==<br />
<br />
Delete intermediate and temporary data you no longer need — it keeps the cluster usable and your costs down. Remember that [[Backup Policy|backups]] are limited, so be sure before you delete.<br />
<br />
<!-- TODO: add the formal data-retention policy if one exists — how long different data is kept, any maximum retention, and what happens to data when an account or project closes. --><br />
<br />
== See also ==<br />
* [[Storage Systems Overview]]<br />
* [[Archival Storage]]<br />
* [[Backup Policy]]<br />
* [[Data storage best practices]]</div>
Haars001