Modules: Difference between revisions

From HPCwiki
Jump to navigation Jump to search
No edit summary
Line 25: Line 25:
* Users are encouraged to adapt their code to the new bucket system as soon as possible to avoid disruption
* Users are encouraged to adapt their code to the new bucket system as soon as possible to avoid disruption


==Listing modules==
==Usage==
 
===Listing Modules===


The commands that hereby follow will list the modules available to the user in increansing detail. '''overview''' provides a top level view of the software available without going into detail about the different versions available. It will only list the software and the number of versions. The '''avail''' command will list the different versions of the same software. Finally, '''spider''' will provide a verbose list with all the different versions and the description of each.
The commands that hereby follow will list the modules available to the user in increansing detail. '''overview''' provides a top level view of the software available without going into detail about the different versions available. It will only list the software and the number of versions. The '''avail''' command will list the different versions of the same software. Finally, '''spider''' will provide a verbose list with all the different versions and the description of each.
Line 41: Line 43:
</pre>
</pre>


==Searching modules==
===Searching For Modules===


The same commands used for listing modules can be used for searching, the only difference is that that the name of the module is passed as an argument. Like the listing in the section above, the commands provide different levels of verbosity.  
The same commands used for listing modules can be used for searching, the only difference is that that the name of the module is passed as an argument. Like the listing in the section above, the commands provide different levels of verbosity.  
Line 57: Line 59:
</pre>
</pre>


===Searching for keywords===
===Searching For Keywords===


As a more advanced search feature, one can search for keywords inside of modules. This is useful when searching for which modules contain a specific Python or R extension. There are bundle modules for both languages that contain a list of their extensions. Lmod will also search inside the description of the modules, which can be useful for discoverability.
As a more advanced search feature, one can search for keywords inside of modules. This is useful when searching for which modules contain a specific Python or R extension. There are bundle modules for both languages that contain a list of their extensions. Lmod will also search inside the description of the modules, which can be useful for discoverability.
Line 90: Line 92:


Hence, one would need to load the module '''R-bundle-CRAN/2023.12-foss-2023a''' to have access to the '''terra''' package
Hence, one would need to load the module '''R-bundle-CRAN/2023.12-foss-2023a''' to have access to the '''terra''' package
===Loading Modules===
===List Loaded Modules===
===Remove Modules===

Revision as of 10:56, 23 August 2024

Modules in anunna

Anunna uses modules via Lmod to provide software to their users. A module configures the environment of the user and/or their jobs to enable the desired application to run. These modules are organized in "buckets" for each year. We intend to keep three buckets of software, one for the current year, one for the previous year and another for legacy software. The legacy module should contain software that is older than two years and is still used or relevant.

The point is to have a conveyor belt of software and have more up-to-date software with more modern build tools (GCC, MPI, intel), which makes the software more maintainable. The conveyor belt also enables the use of more modern toolchains (foss, intel), which will enable software to run more efficiently.

For each bucket, we intend to keep one version of software.Currently there are three buckets available

  • legacy - old unmaintained packages
  • 2023 - packages for the 2023 toolchain
  • GPU - mainly CUDA and related packages that are independent of toolchains.

The new modules have been built with the aid of EasyBuild


Module Migration

  • current modules are crowded, disorganized and inefficient
  • Modules are being organized in buckets for each year with a legacy bucket for old software
  • legacy software is no longer maintained
  • Module in the main location are going to be removed at some point (not yet defined)
  • 2023 and legacy bucket are already available
  • Users are encouraged to adapt their code to the new bucket system as soon as possible to avoid disruption

Usage

Listing Modules

The commands that hereby follow will list the modules available to the user in increansing detail. overview provides a top level view of the software available without going into detail about the different versions available. It will only list the software and the number of versions. The avail command will list the different versions of the same software. Finally, spider will provide a verbose list with all the different versions and the description of each.

module overview
module avail
module spider

Searching For Modules

The same commands used for listing modules can be used for searching, the only difference is that that the name of the module is passed as an argument. Like the listing in the section above, the commands provide different levels of verbosity.

module overview <nameOfModule>
module avail <nameOfModule>
module spider <nameOfModule>

Searching For Keywords

As a more advanced search feature, one can search for keywords inside of modules. This is useful when searching for which modules contain a specific Python or R extension. There are bundle modules for both languages that contain a list of their extensions. Lmod will also search inside the description of the modules, which can be useful for discoverability.

This feature can be used with the following command template:


module key <keyword>


To illustrate, say that one needs to find a module with the R packager terra installed. The first step would be to load one of the buckets, for instance 2023.

module load 2023

Then, the next step would be to apply the key template above

module key terra

which yields the following results

The following modules match your search criteria: "terra"
--------------------------------------------------------------------------------------------------------

  R-bundle-CRAN: R-bundle-CRAN/2023.12-foss-2023a
    Bundle of R packages from CRAN

--------------------------------------------------------------------------------------------------------


Hence, one would need to load the module R-bundle-CRAN/2023.12-foss-2023a to have access to the terra package

Loading Modules

List Loaded Modules

Remove Modules