Best-Practices - Revision history

Honfi001 at 08:55, 31 August 2026

2026-08-31T08:55:49Z

← Older revision		Revision as of 08:55, 31 August 2026
Line 43:		Line 43:

	* '''Control where your job's output and error files go''', and make sure they do not land in your home directory.		* '''Control where your job's output and error files go''', and make sure they do not land in your home directory.
	* '''Be careful with array jobs that write to Lustre.''' Thousands of tasks hammering the filesystem at once will slow Lustre down, and your own jobs are among the ones that suffer. Keep the number of tasks that can run at once to a couple of hundred — <code>--array=1-10000%200</code> rather than <code>--array=1-10000</code>. See [[Array Jobs]] and [[Workflows/Embarassinly-Parallel]].		* '''Be careful with array jobs that write to Lustre.''' Thousands of tasks hammering the filesystem at once will slow Lustre down, and your own jobs are among the ones that suffer. Keep the number of tasks that can run at once to a couple of hundred — <code>--array=1-10000%200</code> rather than <code>--array=1-10000</code>. See [[Array Jobs]] and [[Workflows/Embarassinly-Parallel\|Embarassinly Parallel Workflows]].
	* '''Test at small scale before you submit at large scale.''' One task that works tells you far more than ten thousand that fail identically.		* '''Test at small scale before you submit at large scale.''' One task that works tells you far more than ten thousand that fail identically.

Honfi001: /* Home directory (/home) */

2026-08-31T08:54:27Z

Home directory (/home)

← Older revision		Revision as of 08:54, 31 August 2026
Line 28:		Line 28:

	* '''Do not store datasets or other large files here.''' You will hit the quota, and jobs fail in confusing ways when a write is refused partway through.		* '''Do not store datasets or other large files here.''' You will hit the quota, and jobs fail in confusing ways when a write is refused partway through.
			* '''Periodically check the size of your ~/.local and ~/.cache folders.''' Some applications, like rstudio, store gigabytes of data there. You can use environment variables to point them at a different location and avoid going over your quota
	* '''Do not install software here if you intend to run it in jobs.''' Install it on Lustre instead, where the compute nodes can read it at speed. See [[Installing Personal Software]].		* '''Do not install software here if you intend to run it in jobs.''' Install it on Lustre instead, where the compute nodes can read it at speed. See [[Installing Personal Software]].
	* '''Do not point job output or error logs here''', especially for array jobs: a thousand tasks writing logs into a 200 GB backed-up filesystem is the fastest way to fill it.		* '''Do not point job output or error logs here''', especially for array jobs: a thousand tasks writing logs into a 200 GB backed-up filesystem is the fastest way to fill it.

Honfi001 at 08:51, 31 August 2026

2026-08-31T08:51:39Z

← Older revision		Revision as of 08:51, 31 August 2026
Line 1:		Line 1:
	{{DISPLAYTITLE:Best Practices}}		{{DISPLAYTITLE:Best Practices}}

			Anunna is a shared machine. A few hundred people use the same login nodes, the same filesystems and the same scheduler, so most of the habits below exist because breaking one costs someone else time as well as you — and usually you first.

			Each entry is a rule and the reason for it. Where the reason needs more than a line, it links to the page that gives it properly.

	== Environment ==		== Environment ==
	* Do not modify your ~/.bashrc or ~/.bash_profile
	* If you need to create aliases or environment variables either setup your own module or edit your ~/.bash_aliases
	* Setup environment variables with your lustre folder locations ( see [[HPC Advanced/Preparation Exercise]] part 1)

	== Login ~~node~~ ==		Your shell environment is read on every login and by every job you submit. A broken one follows you everywhere, including into jobs that were queued before you broke it.
	The login ~~node is meant to be~~ a staging area for ~~your computations~~, ~~whatever~~ work ~~outside of editing files should~~ be ~~done~~ in a job in the compute ~~node~~.
			* '''Do not edit <code>~/.bashrc</code> or <code>~/.bash_profile</code>, and do not let software edit them either.''' These files gate your login. If one of them breaks, every new session breaks with it and you are locked out of the cluster until someone fixes it for you.
			* '''Read what an installer proposes to do before you let it run.''' Plenty of tools append to <code>~/.bashrc</code> as part of their setup, without making much noise about it.
			* '''Keep your own aliases and variables in <code>~/.bash_aliases</code>''' rather than in the login files themselves. See [[Installing Personal Software]].
			* '''Wrap anything more involved than an alias in a module of your own.''' A module is loaded when you want it and unloaded when you do not, which a line in <code>~/.bashrc</code> is not. See [[Environment Modules]].
			* '''Set up the variables that point at your Lustre folders''' — <code>$myScratch</code>, <code>$myBkp</code> and <code>$myNoBkp</code> — so that you are not typing long paths by hand or guessing at them in a job script. [[HPC Advanced/Preparation Exercise]] Part 1 walks through it using the <code>getLustreDir</code> helper.

			== Login nodes ==

			The login nodes are a staging area, not a workplace. They are shared by everyone who is logged in, and they have no scheduler protecting them: a single heavy command there is felt by every other user on that node.

			* '''Use a login node for editing files, submitting jobs and checking on them.''' That is what it is for. See [[Login Nodes]].
			* '''Run the actual work in a job on a compute node''', however short you expect it to be. See [[Batch Jobs]].
			* '''Ask for an interactive job when you need a shell to work in''' — compiling, installing software, debugging, exploring a dataset. You get a real allocation and you are not competing with everyone else's editors. See [[Interactive Jobs]].

			== Home directory (/home) ==

			Your home directory is small, backed up, and not built for job data. It holds a 200 GB quota.

			* '''Do not store datasets or other large files here.''' You will hit the quota, and jobs fail in confusing ways when a write is refused partway through.
			* '''Do not install software here if you intend to run it in jobs.''' Install it on Lustre instead, where the compute nodes can read it at speed. See [[Installing Personal Software]].
			* '''Do not point job output or error logs here''', especially for array jobs: a thousand tasks writing logs into a 200 GB backed-up filesystem is the fastest way to fill it.

	~~shared filesystems~~ (/~~home~~)		== Lustre (/lustre) ==

	~~== Shared Filesystems (/lustre) ==~~		Lustre is a parallel filesystem. It is very good at moving large amounts of data quickly, and much weaker at metadata operations — anything that asks a question ''about'' a file rather than reading its contents, such as opening it, listing a directory or checking whether a path exists. Work with the grain of that and Lustre is fast; work against it and it is slow for you and for everyone else on the cluster.
	~~You have~~ a ~~quota~~ of ~~200GB in you home~~ directory,
	*Do not store datasets or ~~large files in your home directory~~. ~~You have a quota~~ of ~~200 GB~~
	*Do not install software that you ~~intend to run in jobs in your $HOME . Install your software in lustre.~~
	*Do not point the ~~logs of your jobs into your $HOME, specially when running array jobs.~~
	*Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

	~~== Shared Filesystems (~~/~~lustre) ==~~		* '''Avoid directories holding large numbers of small files.''' The cost is in the per-file bookkeeping, not in the bytes.
	Lustre is ~~a parallel filesystem~~ that ~~excels~~ are ~~high read~~ and ~~write speeds but~~ it ~~is limited at handling metadata operations~~		* '''Do metadata-heavy work on the compute node's local <code>/tmp</code>, or in <code>/dev/shm</code>''', and copy the result back to Lustre when it is done. Unpacking an archive, building software and running a package manager all fall into this category. See [[Setting TMPDIR]].
			* '''Know whether the folder you are writing to is backed up, before you rely on it being there.''' Lustre is split into backed-up and non-backed-up areas, and your variables are named after that split. [[Backup Policy]] and [[Data Lifecycle Policy]] are the authority on what is kept and for how long; do not infer it from a folder name.

	~~- Avoid folder with lots of small files~~		== Jobs ==

	~~- Use the compute node~~'s /~~tmp directory or~~ /~~dev~~/~~shm for metadata heavy operations~~		* '''Control where your job's output and error files go''', and make sure they do not land in your home directory.
			* '''Be careful with array jobs that write to Lustre.''' Thousands of tasks hammering the filesystem at once will slow Lustre down, and your own jobs are among the ones that suffer. Keep the number of tasks that can run at once to a couple of hundred — <code>--array=1-10000%200</code> rather than <code>--array=1-10000</code>. See [[Array Jobs]] and [[Workflows/Embarassinly-Parallel]].
			* '''Test at small scale before you submit at large scale.''' One task that works tells you far more than ten thousand that fail identically.

	~~- jobs - general good practices about submitting jobs~~		== Educational courses ==

	* ~~Be mindful when running array jobs that write~~ to ~~lustre. thousands of job~~ are ~~likely going to slow down lustre~~ and ~~consequently your jobs. Limit the number of active jobs~~ to a ~~couple~~ of ~~hundred~~		* '''Ask for help with your course as early as you can, and at least a month in advance.''' That leaves room to customise the setup for what you are teaching and to test it before a room full of students meets it.
	* ~~Control where~~ the ~~log~~ and ~~error output locations of your job and make sure they do not point to your home folder~~		* '''Do not modify your students' <code>~/.bashrc</code> or <code>~/.bash_profile</code>.''' Students may also need to use the HPC for other courses or thesis work. Make any environment changes through a module the students load. See [[Environment Modules]] and [[Hints for courses]].

	== ~~Educational Courses~~ ==		== See also ==
	~~Request assistance for your course a soon as possible, at least with a month in advance, this leaves room for customisation and testing~~

	~~Do not modify~~ your ~~student's~~ ~/.~~bashrc or ~~~/~~.bash_profile file~~, ~~any environment modifications should be done via modules~~		* [[Installing Personal Software]] — where to put your own software, and the <code>~/.bash_aliases</code> recipe
			* [[Environment Modules]] — loading software, and writing a module of your own
			* [[Batch Jobs]] and [[Interactive Jobs]] — getting work onto a compute node
			* [[Shared Storage]] and [[Quotas]] — what each filesystem is for, and how much of it you have
			* [[HPC Advanced/Preparation Exercise]] — the Lustre variables and aliases, set up step by step
			* [[Backup Policy]] and [[Data Lifecycle Policy]] — what is kept, and for how long
			* [[Dos and Don'ts]] and [[Data storage best practices]] — related guidance, in more detail on their own subjects

Honfi001 at 08:29, 31 August 2026

2026-08-31T08:29:42Z

← Older revision		Revision as of 08:29, 31 August 2026
Line 1:		Line 1:
	{{DISPLAYTITLE:Best Practices}}		{{DISPLAYTITLE:Best Practices}}

	- Environment		== Environment ==

	* Do not modify your ~/.bashrc or ~/.bash_profile		* Do not modify your ~/.bashrc or ~/.bash_profile
	* If you need to create aliases or environment variables either setup your own module or edit your ~/.bash_aliases		* If you need to create aliases or environment variables either setup your own module or edit your ~/.bash_aliases
	* Setup environment variables with your lustre folder locations ( see [[HPC Advanced/Preparation Exercise]] part 1)		* Setup environment variables with your lustre folder locations ( see [[HPC Advanced/Preparation Exercise]] part 1)

	- Login node		== Login node ==

	The login node is meant to be a staging area for your computations, whatever work outside of editing files should be done in a job in the compute node.		The login node is meant to be a staging area for your computations, whatever work outside of editing files should be done in a job in the compute node.

	- shared filesystems (/home)		shared filesystems (/home)

			== Shared Filesystems (/lustre) ==
	You have a quota of 200GB in you home directory,		You have a quota of 200GB in you home directory,

	*Do not store datasets or large files in your home directory. You have a quota of 200 GB		*Do not store datasets or large files in your home directory. You have a quota of 200 GB
	*Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.		*Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
Line 20:		Line 18:
	*Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.		*Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

	~~- shared filesystems~~ (/lustre)		== Shared Filesystems (/lustre) ==

	Lustre is a parallel filesystem that excels are high read and write speeds but it is limited at handling metadata operations		Lustre is a parallel filesystem that excels are high read and write speeds but it is limited at handling metadata operations

Line 33:		Line 30:
	* Control where the log and error output locations of your job and make sure they do not point to your home folder		* Control where the log and error output locations of your job and make sure they do not point to your home folder

	~~- Education~~ Courses		== Educational Courses ==
			Request assistance for your course a soon as possible, at least with a month in advance, this leaves room for customisation and testing
	Request ~~assistantance~~ for your course a soon as possible, at least with a month in advance, this leaves room for ~~customization~~ and testing

	Do not modify your student's ~/.bashrc or ~/.bash_profile file, any environment modifications should be done via modules		Do not modify your student's ~/.bashrc or ~/.bash_profile file, any environment modifications should be done via modules

Honfi001 at 08:00, 31 August 2026

2026-08-31T08:00:46Z

@@ Line 2: / Line 2: @@
 - Environment
 - Login node
@@ Line 24: / Line 29: @@
 - jobs - general good practices about submitting jobs
 - Education Courses
@@ Line 29: / Line 37: @@
 Request assistantance for your course a soon as possible, at least with a month in advance, this leaves room for customization and testing
-Do not modify your student's ~/.bashrc or ~/.bash_profile file, any environment modifications should be done via modules.
+Do not modify your student's ~/.bashrc or ~/.bash_profile file, any environment modifications should be done via modules

Honfi001 at 07:38, 31 August 2026

2026-08-31T07:38:50Z

← Older revision		Revision as of 07:38, 31 August 2026
Line 1:		Line 1:
	{{DISPLAYTITLE:Best Practices}}		{{DISPLAYTITLE:Best Practices}}

			- Environment
	- Login node		- Login node

Honfi001 at 07:30, 31 August 2026

2026-08-31T07:30:46Z

← Older revision		Revision as of 07:30, 31 August 2026
Line 1:		Line 1:
	{{DISPLAYTITLE:Best Practices}}		{{DISPLAYTITLE:Best Practices}}


			- Login node

			The login node is meant to be a staging area for your computations, whatever work outside of editing files should be done in a job in the compute node.

			- shared filesystems (/home)

			You have a quota of 200GB in you home directory,

			*Do not store datasets or large files in your home directory. You have a quota of 200 GB
			*Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
			*Do not point the logs of your jobs into your $HOME, specially when running array jobs.
			*Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

			- shared filesystems (/lustre)

			Lustre is a parallel filesystem that excels are high read and write speeds but it is limited at handling metadata operations

			- Avoid folder with lots of small files

			- Use the compute node's /tmp directory or /dev/shm for metadata heavy operations

			- jobs - general good practices about submitting jobs

			- Education Courses

			Request assistantance for your course a soon as possible, at least with a month in advance, this leaves room for customization and testing

			Do not modify your student's ~/.bashrc or ~/.bash_profile file, any environment modifications should be done via modules.

	== Login Node ==		== Login Node ==

Haars001: /* Jobs */

2026-08-28T16:03:14Z

Jobs

← Older revision		Revision as of 16:03, 28 August 2026
Line 14:		Line 14:
	== Jobs ==		== Jobs ==

	*		*If you have jobs that are very much alike, please use [[Array Jobs]]. This will make your life a lot easier.

Haars001 at 16:00, 28 August 2026

2026-08-28T16:00:43Z

← Older revision		Revision as of 16:00, 28 August 2026
Line 10:		Line 10:
	* Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.		* Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
	* Do not point the logs of your jobs into your $HOME, specially when running array jobs. If not defined slurm will point them to the location you run the sbatch command.		* Do not point the logs of your jobs into your $HOME, specially when running array jobs. If not defined slurm will point them to the location you run the sbatch command.
	* Do not edit or ~~left~~ software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.		* Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

	== Jobs ==		== Jobs ==

	*		*

Honfi001 at 15:02, 19 August 2026

2026-08-19T15:02:17Z

← Older revision		Revision as of 15:02, 19 August 2026
Line 1:		Line 1:
	{{DISPLAYTITLE:Best Practices}}		{{DISPLAYTITLE:Best Practices}}

			== Login Node ==

			* Do not run analyses or jobs in the login node.

			== Home folder ($HOME, ~) ==

			* Do not store datasets or large files in your home directory. You have a quota of 200 GB.
			* Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
			* Do not point the logs of your jobs into your $HOME, specially when running array jobs. If not defined slurm will point them to the location you run the sbatch command.
			* Do not edit or left software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

			== Jobs ==

			*