Jump to content

Best-Practices: Difference between revisions

From HPCwiki
No edit summary
Line 1: Line 1:
{{DISPLAYTITLE:Best Practices}}
{{DISPLAYTITLE:Best Practices}}
- Login node
The login node is meant to be a staging area for your computations, whatever work outside of editing files should be done in a job in the compute node.
- shared filesystems (/home)
You have a quota of 200GB in you home directory,
*Do not store datasets or large files in your home directory. You have a quota of 200 GB
*Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
*Do not point the logs of your jobs into your $HOME, specially when running array jobs.
*Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.
- shared filesystems (/lustre)
Lustre is a parallel filesystem that excels are high read and write speeds but it is limited at handling metadata operations
- Avoid folder with lots of small files
- Use the compute node's /tmp directory or /dev/shm for metadata heavy operations
- jobs - general good practices about submitting jobs
- Education Courses
Request assistantance for your course a soon as possible, at least with a month in advance, this leaves room for customization and testing
Do not modify your student's ~/.bashrc or ~/.bash_profile file, any environment modifications should be done via modules.


== Login Node ==
== Login Node ==

Revision as of 07:30, 31 August 2026


- Login node

The login node is meant to be a staging area for your computations, whatever work outside of editing files should be done in a job in the compute node.

- shared filesystems (/home)

You have a quota of 200GB in you home directory,

  • Do not store datasets or large files in your home directory. You have a quota of 200 GB
  • Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
  • Do not point the logs of your jobs into your $HOME, specially when running array jobs.
  • Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

- shared filesystems (/lustre)

Lustre is a parallel filesystem that excels are high read and write speeds but it is limited at handling metadata operations

- Avoid folder with lots of small files

- Use the compute node's /tmp directory or /dev/shm for metadata heavy operations

- jobs - general good practices about submitting jobs

- Education Courses

Request assistantance for your course a soon as possible, at least with a month in advance, this leaves room for customization and testing

Do not modify your student's ~/.bashrc or ~/.bash_profile file, any environment modifications should be done via modules.

Login Node

  • Do not run analyses or jobs in the login node.

Home folder ($HOME, ~)

  • Do not store datasets or large files in your home directory. You have a quota of 200 GB.
  • Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
  • Do not point the logs of your jobs into your $HOME, specially when running array jobs. If not defined slurm will point them to the location you run the sbatch command.
  • Do not edit or let software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

Jobs

  • If you have jobs that are very much alike, please use Array Jobs. This will make your life a lot easier.