Filesystems - Revision history

Haars0011 at 15:25, 19 December 2025

2025-12-19T15:25:46Z

← Older revision		Revision as of 17:25, 19 December 2025
Line 50:		Line 50:
	* We have a function to ease uploads (use -h for help) : <code>itape</code>		* We have a function to ease uploads (use -h for help) : <code>itape</code>
	* We have aliases to ease checking of the status of your archive process. (it takes a while) : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.		* We have aliases to ease checking of the status of your archive process. (it takes a while) : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
	* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To ~~fix that, either use <code>irm -f</code> or the icommand to~~ empty it, see <code>irmtrash -h</code>.		* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

			Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5T.
	Because of hardware limitations on the backend tape storage, the ~~filesize~~ limit for our tape archive is 5T.

	== See also ==		== See also ==

Prins0891: add 5T filesize limit for tape

2025-11-28T15:15:36Z

add 5T filesize limit for tape

← Older revision		Revision as of 17:15, 28 November 2025
Line 51:		Line 51:
	* We have aliases to ease checking of the status of your archive process. (it takes a while) : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.		* We have aliases to ease checking of the status of your archive process. (it takes a while) : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
	* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To fix that, either use <code>irm -f</code> or the icommand to empty it, see <code>irmtrash -h</code>.		* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To fix that, either use <code>irm -f</code> or the icommand to empty it, see <code>irmtrash -h</code>.


			Because of hardware limitations on the backend tape storage, the filesize limit for our tape archive is 5T.

	== See also ==		== See also ==

Haars0011: Clearer descriptions

2025-11-21T14:14:17Z

Clearer descriptions

← Older revision		Revision as of 16:14, 21 November 2025
Line 2:		Line 2:

	== Global ==		== Global ==
	* /home - This mount uses NFS to mount the home directories ~~directly~~ from the active master. Each user has a 200G quota for this filesystem, as it is regularly backed up to tape, and can reliably be restored from up to a week's history.		* /home - This mount uses NFS to mount the home directories over the slower internal network from the active master. Each user has a 200G quota for this filesystem,it is regularly backed up to tape, and can reliably be restored from up to a week's history. Use this for programs and configuration files.

	* /shared - This mount provides a consistent set of binaries for the entire cluster.		* /shared - This mount provides a consistent set of binaries and configuration files for the entire cluster.

	* /lustre - This large mount uses the Lustre parallel filesystem to provide files from multiple redundant servers. Access is provided per group, thus:		* /lustre - This large and fast mount uses the Lustre parallel filesystem to provide files from multiple redundant servers over the fast Omnipath network. Access is provided per group, thus:
	/lustre/[level]/[partner]/[unit]		/lustre/[level]/[partner]/[unit]
	e.g.		e.g.

Pelle016: typo

2025-11-17T12:36:30Z

typo

← Older revision		Revision as of 14:36, 17 November 2025
Line 46:		Line 46:

	* The zone is HPC		* The zone is HPC
	* With <code>~~init~~</code> you can init the irods env. Use your account password.		* With <code>iinit</code> you can init the irods env. Use your account password.
	* With <code>ils</code> you can see your available irods collections. You need that as a destination location for <code>itape</code>		* With <code>ils</code> you can see your available irods collections. You need that as a destination location for <code>itape</code>
	* We have a function to ease uploads (use -h for help) : <code>itape</code>		* We have a function to ease uploads (use -h for help) : <code>itape</code>

Pelle016: irods commands added

2025-11-12T16:10:53Z

irods commands added

← Older revision		Revision as of 18:10, 12 November 2025
Line 46:		Line 46:

	* The zone is HPC		* The zone is HPC
	* ~~We have a function to ease uploads :~~ <code>~~itape~~</code>		* With <code>init</code> you can init the irods env. Use your account password.
	* With <code>ils</code> you can see your available irods collections. You need that as a destination location for <code>itape</code>		* With <code>ils</code> you can see your available irods collections. You need that as a destination location for <code>itape</code>
			* We have a function to ease uploads (use -h for help) : <code>itape</code>
	* We have aliases to ease checking of the status of your archive process. (it takes a while) : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.		* We have aliases to ease checking of the status of your archive process. (it takes a while) : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
	* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To fix that, either use <code>irm -f</code> or the icommand to empty it, see <code>irmtrash -h</code>.		* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To fix that, either use <code>irm -f</code> or the icommand to empty it, see <code>irmtrash -h</code>.

Pelle016: add command to help use itape command

2025-11-12T14:10:45Z

add command to help use itape command

← Older revision		Revision as of 16:10, 12 November 2025
Line 47:		Line 47:
	* The zone is HPC		* The zone is HPC
	* We have a function to ease uploads : <code>itape</code>		* We have a function to ease uploads : <code>itape</code>
	* We have aliases to ease checking of the status : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.		* With <code>ils</code> you can see your available irods collections. You need that as a destination location for <code>itape</code>
			* We have aliases to ease checking of the status of your archive process. (it takes a while) : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
	* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To fix that, either use <code>irm -f</code> or the icommand to empty it, see <code>irmtrash -h</code>.		* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To fix that, either use <code>irm -f</code> or the icommand to empty it, see <code>irmtrash -h</code>.

Haars0011: Added info on how to empty the iRODS trashbin

2025-07-18T09:14:58Z

Added info on how to empty the iRODS trashbin

← Older revision		Revision as of 11:14, 18 July 2025
Line 48:		Line 48:
	* We have a function to ease uploads : <code>itape</code>		* We have a function to ease uploads : <code>itape</code>
	* We have aliases to ease checking of the status : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.		* We have aliases to ease checking of the status : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
			* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trashbin. The advantage is that you can retrieve it if the removal was in error, the disadvantage is that the data will keep costing money. To fix that, either use <code>irm -f</code> or the icommand to empty it, see <code>irmtrash -h</code>.

	== See also ==		== See also ==

Prins0891 at 07:08, 15 July 2025

2025-07-15T07:08:30Z

← Older revision		Revision as of 09:08, 15 July 2025
Line 37:		Line 37:
	With that you can push data to the WUR tape storage for archiving at very low cost.		With that you can push data to the WUR tape storage for archiving at very low cost.

	More info on how to use it, please see https://irods.~~wurnet~~.nl/.		More info on how to use it, please see https://irods.wur.nl/.

	The best course of action is to loosely follow the course, using your own data, and use your personal space for data upload and transfer to tape.		The best course of action is to loosely follow the course, using your own data, and use your personal space for data upload and transfer to tape.

Haars0011 at 11:52, 8 July 2025

2025-07-08T11:52:09Z

← Older revision		Revision as of 13:52, 8 July 2025
Line 47:		Line 47:
	* The zone is HPC		* The zone is HPC
	* We have a function to ease uploads : <code>itape</code>		* We have a function to ease uploads : <code>itape</code>
	* We have ~~an alias~~ to ease checking : <code>itapestat</code>		* We have aliases to ease checking of the status : <code>itapestat</code> and <code>itapestatnp</code>, the first is for human use, is shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.

	== See also ==		== See also ==

Haars0011 at 08:59, 8 July 2025

2025-07-08T08:59:13Z

← Older revision		Revision as of 10:59, 8 July 2025
Line 37:		Line 37:
	With that you can push data to the WUR tape storage for archiving at very low cost.		With that you can push data to the WUR tape storage for archiving at very low cost.

	More info on how, please see https://irods.wurnet.nl/.		More info on how to use it, please see https://irods.wurnet.nl/.

	The best course of action is to loosely follow the course, using your own data, and use your personal space for data upload and transfer to tape.		The best course of action is to loosely follow the course, using your own data, and use your personal space for data upload and transfer to tape.

	Be sure to check whether the data is correctly stored on tape before you remove your data!		Be sure to check whether the data is correctly stored on tape before you remove your data!

			On Anunna there are some differences and additions to the above site:

			* The zone is HPC
			* We have a function to ease uploads : <code>itape</code>
			* We have an alias to ease checking : <code>itapestat</code>

	== See also ==		== See also ==