HPCwiki - User contributions [en]

Software

2026-08-26T06:25:22Z

Honfi001: /* Containers */

The software available on Anunna and how to use, install, and manage it.

== Organization ==

* [[Software Overview]] — the software landscape
* [[Environment Modules]] — loading software with modules and buckets
* [[Installing Personal Software]] — installing into your own space
* [[Licensed Software]] — software that needs a licence

== Scripting Languanges ==
* [[Python]]
* [[R]]
* [[Julia]]

== Containers ==
* [[Apptainer]]

== MPI Implemenations ==
* [[OpenMPI]]
* [[IntelMPI]]

Archival Storage

2026-08-25T12:52:03Z

Honfi001: /* Group pages */

<code>/archive</code> is Anunna's warm storage tier for data you want to keep but are not actively computing on. Storing data there costs less than on [[Compute Storage|Lustre]], and datasets that have not been used for a long time are moved on to long-term tape archive — automatically, through the [[#Tapeworm|Tapeworm]] service — to keep the warm tier free for active work.

== The /archive filesystem ==

<code>/archive</code> is a mount that is only accessible from the login nodes, it cannot be used for compute work, and it is only available to WUR users.

<code>/archive</code> is backed up periodically on the same schedule as your [[Home Directory|home directory]] — restorable from roughly a week of history (see [[Backup Policy]]). That backup is separate from the long-term tape archival described below: a backup is a short-term safety copy, while archival moves data off <code>/archive</code> for long-term keeping.

== Tapeworm ==

Tapeworm (https://tapeworm.anunna.wur.nl/) helps you manage data on <code>/archive</code> by identifying datasets that are no longer actively used and preparing them for tape archival. The goal is simple: keep our warm storage available for active work, while safely preserving older data on tape.

With Tapeworm, you can:
* See which of your datasets are being considered for tape archival
* Review planned moves before they happen
* Approve, snooze, or block moves when needed
* Add metadata to help describe archived datasets. The metadata is included on tape and can be used to view/retrieve from tape, should you need to do so in the future

If you do nothing, Tapeworm will continue with the planned move after the review period. That is why we recommend checking your pending actions regularly. You will also receive notification emails about pending actions.

From time to time, the user pages may temporarily be unavailable during maintenance. In that case, Tapeworm will show a short maintenance page instead of the normal interface.

=== How Tapeworm works ===

# Tapeworm scans <code>/archive</code> and builds an index of paths and filesystem metadata.
# At the dataset discovery step, it groups those paths into datasets, determines size, owner, and last active use, and makes them available in the GUI. Very small datasets are filtered out and will not be shown.
# A policy engine checks which discovered datasets look inactive (for example: 60+ days old and larger than 1GB).
# Matching datasets are marked as '''planned''' and shown in your overview.
# You will be notified by email that Tapeworm plans to move data you own.
# You can review and change what should happen, or block the move(s) entirely.
# If no action is taken, after a wait period of 4 weeks, planned moves will become scheduled and then executed.
# Data is moved to tape, and removed from <code>/archive</code>.

=== Who sees what? ===

* '''Regular users''' see only their own datasets and actions.
* '''Group admins/contacts''' see data for their configured group(s), in addition to their own data.

=== User pages ===

==== 1) Overview ====
This is your action page. It shows items that currently need your decision.
[[File:Screenshot from 2026-03-17 15-19-46.png|none|thumb|1400x1400px]]
For each candidate, you can:
* '''Approve''': proceed with the tape move. It will schedule for the next day
* '''Deny''': stop this move, configure an override for this path. Tapeworm will not try to move this dataset/path again, until you choose to remove the override
* '''Snooze''': postpone the decision to a future date
* '''Edit metadata''': add key/value notes for archived data. These values are included on tape and can be used to view/retrieve datasets on tape

You can also select multiple rows and apply actions in bulk.

==== 2) Datasets ====
This page shows your discovered datasets, their sizes, and last activity times. The application has no concept of what data belongs together and should be considered a 'dataset'. If the selections on this page are wrong, you can change how Tapeworm should handle these datasets instead.
[[File:Screenshot from 2026-03-17 15-20-23.png|none|thumb|1400x1400px]]
'''Important:'''
* If a dataset already has an active move candidate, scheduling controls are disabled.
The dataset list is informational; move decisions are handled through the Schedule page.

==== 3) Schedule ====
This page shows move candidates and their status over time.
[[File:Screenshot from 2026-03-17 15-20-58.png|none|thumb|1400x1400px]]
Common statuses:
* '''Planned''': under review
* '''Scheduled''': move is planned for a specific date
* '''Executing / Tape staged / On tape''': move is in progress or completed
* '''Error''': move needs admin attention. You may be contacted, maybe we resolve it ourselves :)

Once a move is already executing or completed, schedule-changing actions are locked.

==== 4) Overrides ====
Overrides tell Tapeworm to ignore specific paths in future planning.
[[File:Screenshot from 2026-03-17 15-21-49.png|none|thumb|1400x1400px]]
Use overrides when:
* a project is still active and needs to remain on /archive
* policy suggestions are not appropriate for that location

If you agree that the dataset can in principle be moved to tape, but you don't (yet) know when, you can choose to postpone/snooze the archival instead of overriding it.

Overrides apply to the selected path and everything below it.

==== 5) History ====
This page shows completed archival moves. When a dataset has been successfully archived and finalized, it is removed from active scheduling pages and moved into history.
[[File:Screenshot from 2026-03-17 15-24-40.png|none|thumb|1400x1400px]]

=== Group pages ===

Group admins have a separate additional set of pages for their group scope:
* '''Group overview'''
* '''Group datasets'''
* '''Group schedule'''
* '''Group overrides'''
* '''Group history'''

If you manage more than one group, you can switch group scope in the selector at the top of the group pages.

=== Notifications (email) ===

Tapeworm sends email updates when actions are pending, dates are approaching, or move state changes happen.

Emails typically include:
* Dataset path
* Size
* Last activity
* Current status
* Review/scheduled date

Notification types you may receive:
* '''Action required''': please approve, snooze, or deny
* '''Reminder''': review date is approaching
* '''Informational''': move status changed (for example scheduled, staging, or completed)
* '''Escalation''': sent to group contacts when no user response is received

Please read these emails carefully — they are your chance to adjust decisions before execution.

=== Best practices for users ===

* Check your '''Overview''' page regularly
* Use '''Snooze''' if you need time to validate impact
* Add '''metadata''' when approving important datasets
* Use '''Overrides''' for known and persisting exceptions
* If unsure, contact HPC support before a scheduled move date

=== Tapeworm FAQ ===

==== What happens if I do nothing? ====
Planned items can move forward automatically after the review window (4 weeks).

==== Can I undo after tape staging? ====
Not directly in Tapeworm. Retrieval is done via the tape/iRODS workflow. See: https://irods.wur.nl/userguide/tape_retrieval/

==== What does "completed" mean? ====
Completed means Tapeworm saw the tape workflow finish and finalized the move. Before finalization, the system verifies the archive in iRODS and only then removes the staged source copy.

==== Why is an action button disabled? ====
Usually because the move has already progressed (executing/staged/on tape/error), so schedule edits are no longer valid.

==== Why do some paths on /archive/ not appear as datasets in Tapeworm? ====
Tapeworm only shows paths that are discovered as datasets and pass a minimum-size threshold. Very small paths, single text files, and other tiny items are intentionally filtered out.

=== Need help? ===
If anything is unclear, or you think a move is incorrect but you cannot alter it in the provided GUI, please open an HPC support ticket.

== Manual tape access with iRODS besides Tapeworm ==

Anunna hosts its own iRODS instance, with which you can push data manually to the WUR tape storage for archiving. For general usage information, see https://irods.wur.nl/. The best course of action is to loosely follow the course using your own data, and use your personal space for data upload and transfer to tape.

'''Be sure to check whether the data is correctly stored on tape before you remove your data.'''

On Anunna there are some differences and additions to the linked site:

* The zone is <code>HPC</code>.
* With <code>iinit</code> you can initialise the iRODS environment. Use your account password.
* With <code>ils</code> you can see your available iRODS collections. You need that as a destination location for <code>itape</code>.
* We have a function to ease uploads (use <code>-h</code> for help): <code>itape</code>.
* We have aliases to ease checking the status of your archive process (it takes a while): <code>itapestat</code> and <code>itapestatnp</code>. The first is for human use — it shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trash bin. The advantage is that you can retrieve it if the removal was in error; the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5 TB.

== Retrieving data from tape ==
As the tape storage is maintained by a different team, it is best to follow their documentation on retrieval.

For that, see here: https://irods.wur.nl/userguide/archiving_research_data_using_icommands/

== See also ==
* [[Storage Systems Overview]]
* [[Backup Policy]]
* [[Compute Storage]]
* [[Tariffs|Costs associated with resource usage]]

== External links ==
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]

Archival Storage

2026-08-25T12:51:14Z

Honfi001: /* Group pages */

<code>/archive</code> is Anunna's warm storage tier for data you want to keep but are not actively computing on. Storing data there costs less than on [[Compute Storage|Lustre]], and datasets that have not been used for a long time are moved on to long-term tape archive — automatically, through the [[#Tapeworm|Tapeworm]] service — to keep the warm tier free for active work.

== The /archive filesystem ==

<code>/archive</code> is a mount that is only accessible from the login nodes, it cannot be used for compute work, and it is only available to WUR users.

<code>/archive</code> is backed up periodically on the same schedule as your [[Home Directory|home directory]] — restorable from roughly a week of history (see [[Backup Policy]]). That backup is separate from the long-term tape archival described below: a backup is a short-term safety copy, while archival moves data off <code>/archive</code> for long-term keeping.

== Tapeworm ==

Tapeworm (https://tapeworm.anunna.wur.nl/) helps you manage data on <code>/archive</code> by identifying datasets that are no longer actively used and preparing them for tape archival. The goal is simple: keep our warm storage available for active work, while safely preserving older data on tape.

With Tapeworm, you can:
* See which of your datasets are being considered for tape archival
* Review planned moves before they happen
* Approve, snooze, or block moves when needed
* Add metadata to help describe archived datasets. The metadata is included on tape and can be used to view/retrieve from tape, should you need to do so in the future

If you do nothing, Tapeworm will continue with the planned move after the review period. That is why we recommend checking your pending actions regularly. You will also receive notification emails about pending actions.

From time to time, the user pages may temporarily be unavailable during maintenance. In that case, Tapeworm will show a short maintenance page instead of the normal interface.

=== How Tapeworm works ===

# Tapeworm scans <code>/archive</code> and builds an index of paths and filesystem metadata.
# At the dataset discovery step, it groups those paths into datasets, determines size, owner, and last active use, and makes them available in the GUI. Very small datasets are filtered out and will not be shown.
# A policy engine checks which discovered datasets look inactive (for example: 60+ days old and larger than 1GB).
# Matching datasets are marked as '''planned''' and shown in your overview.
# You will be notified by email that Tapeworm plans to move data you own.
# You can review and change what should happen, or block the move(s) entirely.
# If no action is taken, after a wait period of 4 weeks, planned moves will become scheduled and then executed.
# Data is moved to tape, and removed from <code>/archive</code>.

=== Who sees what? ===

* '''Regular users''' see only their own datasets and actions.
* '''Group admins/contacts''' see data for their configured group(s), in addition to their own data.

=== User pages ===

==== 1) Overview ====
This is your action page. It shows items that currently need your decision.
[[File:Screenshot from 2026-03-17 15-19-46.png|none|thumb|1400x1400px]]
For each candidate, you can:
* '''Approve''': proceed with the tape move. It will schedule for the next day
* '''Deny''': stop this move, configure an override for this path. Tapeworm will not try to move this dataset/path again, until you choose to remove the override
* '''Snooze''': postpone the decision to a future date
* '''Edit metadata''': add key/value notes for archived data. These values are included on tape and can be used to view/retrieve datasets on tape

You can also select multiple rows and apply actions in bulk.

==== 2) Datasets ====
This page shows your discovered datasets, their sizes, and last activity times. The application has no concept of what data belongs together and should be considered a 'dataset'. If the selections on this page are wrong, you can change how Tapeworm should handle these datasets instead.
[[File:Screenshot from 2026-03-17 15-20-23.png|none|thumb|1400x1400px]]
'''Important:'''
* If a dataset already has an active move candidate, scheduling controls are disabled.
The dataset list is informational; move decisions are handled through the Schedule page.

==== 3) Schedule ====
This page shows move candidates and their status over time.
[[File:Screenshot from 2026-03-17 15-20-58.png|none|thumb|1400x1400px]]
Common statuses:
* '''Planned''': under review
* '''Scheduled''': move is planned for a specific date
* '''Executing / Tape staged / On tape''': move is in progress or completed
* '''Error''': move needs admin attention. You may be contacted, maybe we resolve it ourselves :)

Once a move is already executing or completed, schedule-changing actions are locked.

==== 4) Overrides ====
Overrides tell Tapeworm to ignore specific paths in future planning.
[[File:Screenshot from 2026-03-17 15-21-49.png|none|thumb|1400x1400px]]
Use overrides when:
* a project is still active and needs to remain on /archive
* policy suggestions are not appropriate for that location

If you agree that the dataset can in principle be moved to tape, but you don't (yet) know when, you can choose to postpone/snooze the archival instead of overriding it.

Overrides apply to the selected path and everything below it.

==== 5) History ====
This page shows completed archival moves. When a dataset has been successfully archived and finalized, it is removed from active scheduling pages and moved into history.
[[File:Screenshot from 2026-03-17 15-24-40.png|none|thumb|1400x1400px]]

=== Group pages ===

Group admins have an additional set of pages for their group scope:
* '''Group overview'''
* '''Group datasets'''
* '''Group schedule'''
* '''Group overrides'''
* '''Group history'''

If you manage more than one group, you can switch group scope in the selector at the top of the group pages.

=== Notifications (email) ===

Tapeworm sends email updates when actions are pending, dates are approaching, or move state changes happen.

Emails typically include:
* Dataset path
* Size
* Last activity
* Current status
* Review/scheduled date

Notification types you may receive:
* '''Action required''': please approve, snooze, or deny
* '''Reminder''': review date is approaching
* '''Informational''': move status changed (for example scheduled, staging, or completed)
* '''Escalation''': sent to group contacts when no user response is received

Please read these emails carefully — they are your chance to adjust decisions before execution.

=== Best practices for users ===

* Check your '''Overview''' page regularly
* Use '''Snooze''' if you need time to validate impact
* Add '''metadata''' when approving important datasets
* Use '''Overrides''' for known and persisting exceptions
* If unsure, contact HPC support before a scheduled move date

=== Tapeworm FAQ ===

==== What happens if I do nothing? ====
Planned items can move forward automatically after the review window (4 weeks).

==== Can I undo after tape staging? ====
Not directly in Tapeworm. Retrieval is done via the tape/iRODS workflow. See: https://irods.wur.nl/userguide/tape_retrieval/

==== What does "completed" mean? ====
Completed means Tapeworm saw the tape workflow finish and finalized the move. Before finalization, the system verifies the archive in iRODS and only then removes the staged source copy.

==== Why is an action button disabled? ====
Usually because the move has already progressed (executing/staged/on tape/error), so schedule edits are no longer valid.

==== Why do some paths on /archive/ not appear as datasets in Tapeworm? ====
Tapeworm only shows paths that are discovered as datasets and pass a minimum-size threshold. Very small paths, single text files, and other tiny items are intentionally filtered out.

=== Need help? ===
If anything is unclear, or you think a move is incorrect but you cannot alter it in the provided GUI, please open an HPC support ticket.

== Manual tape access with iRODS besides Tapeworm ==

Anunna hosts its own iRODS instance, with which you can push data manually to the WUR tape storage for archiving. For general usage information, see https://irods.wur.nl/. The best course of action is to loosely follow the course using your own data, and use your personal space for data upload and transfer to tape.

'''Be sure to check whether the data is correctly stored on tape before you remove your data.'''

On Anunna there are some differences and additions to the linked site:

* The zone is <code>HPC</code>.
* With <code>iinit</code> you can initialise the iRODS environment. Use your account password.
* With <code>ils</code> you can see your available iRODS collections. You need that as a destination location for <code>itape</code>.
* We have a function to ease uploads (use <code>-h</code> for help): <code>itape</code>.
* We have aliases to ease checking the status of your archive process (it takes a while): <code>itapestat</code> and <code>itapestatnp</code>. The first is for human use — it shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trash bin. The advantage is that you can retrieve it if the removal was in error; the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5 TB.

== Retrieving data from tape ==
As the tape storage is maintained by a different team, it is best to follow their documentation on retrieval.

For that, see here: https://irods.wur.nl/userguide/archiving_research_data_using_icommands/

== See also ==
* [[Storage Systems Overview]]
* [[Backup Policy]]
* [[Compute Storage]]
* [[Tariffs|Costs associated with resource usage]]

== External links ==
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]

Archival Storage

2026-08-25T12:48:22Z

Honfi001:

<code>/archive</code> is Anunna's warm storage tier for data you want to keep but are not actively computing on. Storing data there costs less than on [[Compute Storage|Lustre]], and datasets that have not been used for a long time are moved on to long-term tape archive — automatically, through the [[#Tapeworm|Tapeworm]] service — to keep the warm tier free for active work.

== The /archive filesystem ==

<code>/archive</code> is a mount that is only accessible from the login nodes, it cannot be used for compute work, and it is only available to WUR users.

<code>/archive</code> is backed up periodically on the same schedule as your [[Home Directory|home directory]] — restorable from roughly a week of history (see [[Backup Policy]]). That backup is separate from the long-term tape archival described below: a backup is a short-term safety copy, while archival moves data off <code>/archive</code> for long-term keeping.

== Tapeworm ==

Tapeworm (https://tapeworm.anunna.wur.nl/) helps you manage data on <code>/archive</code> by identifying datasets that are no longer actively used and preparing them for tape archival. The goal is simple: keep our warm storage available for active work, while safely preserving older data on tape.

With Tapeworm, you can:
* See which of your datasets are being considered for tape archival
* Review planned moves before they happen
* Approve, snooze, or block moves when needed
* Add metadata to help describe archived datasets. The metadata is included on tape and can be used to view/retrieve from tape, should you need to do so in the future

If you do nothing, Tapeworm will continue with the planned move after the review period. That is why we recommend checking your pending actions regularly. You will also receive notification emails about pending actions.

From time to time, the user pages may temporarily be unavailable during maintenance. In that case, Tapeworm will show a short maintenance page instead of the normal interface.

=== How Tapeworm works ===

# Tapeworm scans <code>/archive</code> and builds an index of paths and filesystem metadata.
# At the dataset discovery step, it groups those paths into datasets, determines size, owner, and last active use, and makes them available in the GUI. Very small datasets are filtered out and will not be shown.
# A policy engine checks which discovered datasets look inactive (for example: 60+ days old and larger than 1GB).
# Matching datasets are marked as '''planned''' and shown in your overview.
# You will be notified by email that Tapeworm plans to move data you own.
# You can review and change what should happen, or block the move(s) entirely.
# If no action is taken, after a wait period of 4 weeks, planned moves will become scheduled and then executed.
# Data is moved to tape, and removed from <code>/archive</code>.

=== Who sees what? ===

* '''Regular users''' see only their own datasets and actions.
* '''Group admins/contacts''' see data for their configured group(s), in addition to their own data.

=== User pages ===

==== 1) Overview ====
This is your action page. It shows items that currently need your decision.
[[File:Screenshot from 2026-03-17 15-19-46.png|none|thumb|1400x1400px]]
For each candidate, you can:
* '''Approve''': proceed with the tape move. It will schedule for the next day
* '''Deny''': stop this move, configure an override for this path. Tapeworm will not try to move this dataset/path again, until you choose to remove the override
* '''Snooze''': postpone the decision to a future date
* '''Edit metadata''': add key/value notes for archived data. These values are included on tape and can be used to view/retrieve datasets on tape

You can also select multiple rows and apply actions in bulk.

==== 2) Datasets ====
This page shows your discovered datasets, their sizes, and last activity times. The application has no concept of what data belongs together and should be considered a 'dataset'. If the selections on this page are wrong, you can change how Tapeworm should handle these datasets instead.
[[File:Screenshot from 2026-03-17 15-20-23.png|none|thumb|1400x1400px]]
'''Important:'''
* If a dataset already has an active move candidate, scheduling controls are disabled.
The dataset list is informational; move decisions are handled through the Schedule page.

==== 3) Schedule ====
This page shows move candidates and their status over time.
[[File:Screenshot from 2026-03-17 15-20-58.png|none|thumb|1400x1400px]]
Common statuses:
* '''Planned''': under review
* '''Scheduled''': move is planned for a specific date
* '''Executing / Tape staged / On tape''': move is in progress or completed
* '''Error''': move needs admin attention. You may be contacted, maybe we resolve it ourselves :)

Once a move is already executing or completed, schedule-changing actions are locked.

==== 4) Overrides ====
Overrides tell Tapeworm to ignore specific paths in future planning.
[[File:Screenshot from 2026-03-17 15-21-49.png|none|thumb|1400x1400px]]
Use overrides when:
* a project is still active and needs to remain on /archive
* policy suggestions are not appropriate for that location

If you agree that the dataset can in principle be moved to tape, but you don't (yet) know when, you can choose to postpone/snooze the archival instead of overriding it.

Overrides apply to the selected path and everything below it.

==== 5) History ====
This page shows completed archival moves. When a dataset has been successfully archived and finalized, it is removed from active scheduling pages and moved into history.
[[File:Screenshot from 2026-03-17 15-24-40.png|none|thumb|1400x1400px]]

=== Group pages ===

Group admins have a separate set of pages for their group scope:
* '''Group overview'''
* '''Group datasets'''
* '''Group schedule'''
* '''Group overrides'''
* '''Group history'''

If you manage more than one group, you can switch group scope in the selector at the top of the group pages.

=== Notifications (email) ===

Tapeworm sends email updates when actions are pending, dates are approaching, or move state changes happen.

Emails typically include:
* Dataset path
* Size
* Last activity
* Current status
* Review/scheduled date

Notification types you may receive:
* '''Action required''': please approve, snooze, or deny
* '''Reminder''': review date is approaching
* '''Informational''': move status changed (for example scheduled, staging, or completed)
* '''Escalation''': sent to group contacts when no user response is received

Please read these emails carefully — they are your chance to adjust decisions before execution.

=== Best practices for users ===

* Check your '''Overview''' page regularly
* Use '''Snooze''' if you need time to validate impact
* Add '''metadata''' when approving important datasets
* Use '''Overrides''' for known and persisting exceptions
* If unsure, contact HPC support before a scheduled move date

=== Tapeworm FAQ ===

==== What happens if I do nothing? ====
Planned items can move forward automatically after the review window (4 weeks).

==== Can I undo after tape staging? ====
Not directly in Tapeworm. Retrieval is done via the tape/iRODS workflow. See: https://irods.wur.nl/userguide/tape_retrieval/

==== What does "completed" mean? ====
Completed means Tapeworm saw the tape workflow finish and finalized the move. Before finalization, the system verifies the archive in iRODS and only then removes the staged source copy.

==== Why is an action button disabled? ====
Usually because the move has already progressed (executing/staged/on tape/error), so schedule edits are no longer valid.

==== Why do some paths on /archive/ not appear as datasets in Tapeworm? ====
Tapeworm only shows paths that are discovered as datasets and pass a minimum-size threshold. Very small paths, single text files, and other tiny items are intentionally filtered out.

=== Need help? ===
If anything is unclear, or you think a move is incorrect but you cannot alter it in the provided GUI, please open an HPC support ticket.

== Manual tape access with iRODS besides Tapeworm ==

Anunna hosts its own iRODS instance, with which you can push data manually to the WUR tape storage for archiving. For general usage information, see https://irods.wur.nl/. The best course of action is to loosely follow the course using your own data, and use your personal space for data upload and transfer to tape.

'''Be sure to check whether the data is correctly stored on tape before you remove your data.'''

On Anunna there are some differences and additions to the linked site:

* The zone is <code>HPC</code>.
* With <code>iinit</code> you can initialise the iRODS environment. Use your account password.
* With <code>ils</code> you can see your available iRODS collections. You need that as a destination location for <code>itape</code>.
* We have a function to ease uploads (use <code>-h</code> for help): <code>itape</code>.
* We have aliases to ease checking the status of your archive process (it takes a while): <code>itapestat</code> and <code>itapestatnp</code>. The first is for human use — it shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trash bin. The advantage is that you can retrieve it if the removal was in error; the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5 TB.

== Retrieving data from tape ==
As the tape storage is maintained by a different team, it is best to follow their documentation on retrieval.

For that, see here: https://irods.wur.nl/userguide/archiving_research_data_using_icommands/

== See also ==
* [[Storage Systems Overview]]
* [[Backup Policy]]
* [[Compute Storage]]
* [[Tariffs|Costs associated with resource usage]]

== External links ==
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]

Archival Storage

2026-08-25T12:43:47Z

Honfi001:

<code>/archive</code> is Anunna's warm storage tier for data you want to keep but are not actively computing on. Storing data there costs less than on [[Compute Storage|Lustre]], and datasets that have not been used for a long time are moved on to long-term tape archive — automatically, through the [[#Tapeworm|Tapeworm]] service — to keep the warm tier free for active work.

== The /archive filesystem ==

<code>/archive</code> is a mount that is only accessible from the login nodes, it cannot be used for compute work, and it is only available to WUR users.

<code>/archive</code> is backed up periodically on the same schedule as your [[Home Directory|home directory]] — restorable from roughly a week of history (see [[Backup Policy]]). That backup is separate from the long-term tape archival described below: a backup is a short-term safety copy, while archival moves data off <code>/archive</code> for long-term keeping.

== Tapeworm ==

Tapeworm (https://tapeworm.anunna.wur.nl/) helps you manage data on <code>/archive</code> by identifying datasets that are no longer actively used and preparing them for tape archival. The goal is simple: keep our warm storage available for active work, while safely preserving older data on tape.

With Tapeworm, you can:
* See which of your datasets are being considered for tape archival
* Review planned moves before they happen
* Approve, snooze, or block moves when needed
* Add metadata to help describe archived datasets. The metadata is included on tape and can be used to view/retrieve from tape, should you need to do so in the future

If you do nothing, Tapeworm will continue with the planned move after the review period. That is why we recommend checking your pending actions regularly. You will also receive notification emails about pending actions.

From time to time, the user pages may temporarily be unavailable during maintenance. In that case, Tapeworm will show a short maintenance page instead of the normal interface.

=== How Tapeworm works ===

# Tapeworm scans <code>/archive</code> and builds an index of paths and filesystem metadata.
# At the dataset discovery step, it groups those paths into datasets, determines size, owner, and last active use, and makes them available in the GUI. Very small datasets are filtered out and will not be shown.
# A policy engine checks which discovered datasets look inactive (for example: 60+ days old and larger than 1GB).
# Matching datasets are marked as '''planned''' and shown in your overview.
# You will be notified by email that Tapeworm plans to move data you own.
# You can review and change what should happen, or block the move(s) entirely.
# If no action is taken, after a wait period of 4 weeks, planned moves will become scheduled and then executed.
# Data is moved to tape, and removed from <code>/archive</code>.

=== Who sees what? ===

* '''Regular users''' see only their own datasets and actions.
* '''Group admins/contacts''' see data for their configured group(s), in addition to their own data.

=== User pages ===

==== 1) Overview ====
This is your action page. It shows items that currently need your decision.
[[File:Screenshot from 2026-03-17 15-19-46.png|none|thumb|1400x1400px]]
For each candidate, you can:
* '''Approve''': proceed with the tape move. It will schedule for the next day
* '''Deny''': stop this move, configure an override for this path. Tapeworm will not try to move this dataset/path again, until you choose to remove the override
* '''Snooze''': postpone the decision to a future date
* '''Edit metadata''': add key/value notes for archived data. These values are included on tape and can be used to view/retrieve datasets on tape

You can also select multiple rows and apply actions in bulk.

==== 2) Datasets ====
This page shows your discovered datasets, their sizes, and last activity times. The application has no concept of what data belongs together and should be considered a 'dataset'. If the selections on this page are wrong, you can change how Tapeworm should handle these datasets instead.
[[File:Screenshot from 2026-03-17 15-20-23.png|none|thumb|1400x1400px]]
Important:
* If a dataset already has an active move candidate, scheduling controls are disabled.
The dataset list is informational; move decisions are handled through the Schedule page.

==== 3) Schedule ====
This page shows move candidates and their status over time.
[[File:Screenshot from 2026-03-17 15-20-58.png|none|thumb|1400x1400px]]
Common statuses:
* '''Planned''': under review
* '''Scheduled''': move is planned for a specific date
* '''Executing / Tape staged / On tape''': move is in progress or completed
* '''Error''': move needs admin attention. You may be contacted, maybe we resolve it ourselves :)

Once a move is already executing or completed, schedule-changing actions are locked.

==== 4) Overrides ====
Overrides tell Tapeworm to ignore specific paths in future planning.
[[File:Screenshot from 2026-03-17 15-21-49.png|none|thumb|1400x1400px]]
Use overrides when:
* a project is still active and needs to remain on /archive
* policy suggestions are not appropriate for that location

If you agree that the dataset can in principle be moved to tape, but you don't (yet) know when, you can choose to postpone/snooze the archival instead of overriding it.

Overrides apply to the selected path and everything below it.

==== 5) History ====
This page shows completed archival moves. When a dataset has been successfully archived and finalized, it is removed from active scheduling pages and moved into history.
[[File:Screenshot from 2026-03-17 15-24-40.png|none|thumb|1400x1400px]]

=== Group pages ===

Group admins have a separate set of pages for their group scope:
* '''Group overview'''
* '''Group datasets'''
* '''Group schedule'''
* '''Group overrides'''
* '''Group history'''

If you manage more than one group, you can switch group scope in the selector at the top of the group pages.

=== Notifications (email) ===

Tapeworm sends email updates when actions are pending, dates are approaching, or move state changes happen.

Emails typically include:
* Dataset path
* Size
* Last activity
* Current status
* Review/scheduled date

Notification types you may receive:
* '''Action required''': please approve, snooze, or deny
* '''Reminder''': review date is approaching
* '''Informational''': move status changed (for example scheduled, staging, or completed)
* '''Escalation''': sent to group contacts when no user response is received

Please read these emails carefully — they are your chance to adjust decisions before execution.

=== Best practices for users ===

* Check your '''Overview''' page regularly
* Use '''Snooze''' if you need time to validate impact
* Add '''metadata''' when approving important datasets
* Use '''Overrides''' for known and persisting exceptions
* If unsure, contact HPC support before a scheduled move date

=== Tapeworm FAQ ===

==== What happens if I do nothing? ====
Planned items can move forward automatically after the review window (4 weeks).

==== Can I undo after tape staging? ====
Not directly in Tapeworm. Retrieval is done via the tape/iRODS workflow. See: https://irods.wur.nl/userguide/tape_retrieval/

==== What does "completed" mean? ====
Completed means Tapeworm saw the tape workflow finish and finalized the move. Before finalization, the system verifies the archive in iRODS and only then removes the staged source copy.

==== Why is an action button disabled? ====
Usually because the move has already progressed (executing/staged/on tape/error), so schedule edits are no longer valid.

==== Why do some paths on /archive/ not appear as datasets in Tapeworm? ====
Tapeworm only shows paths that are discovered as datasets and pass a minimum-size threshold. Very small paths, single text files, and other tiny items are intentionally filtered out.

=== Need help? ===
If anything is unclear, or you think a move is incorrect but you cannot alter it in the provided GUI, please open an HPC support ticket.

== Manual tape access with iRODS besides Tapeworm ==

Anunna hosts its own iRODS instance, with which you can push data manually to the WUR tape storage for archiving. For general usage information, see https://irods.wur.nl/. The best course of action is to loosely follow the course using your own data, and use your personal space for data upload and transfer to tape.

'''Be sure to check whether the data is correctly stored on tape before you remove your data.'''

On Anunna there are some differences and additions to the linked site:

* The zone is <code>HPC</code>.
* With <code>iinit</code> you can initialise the iRODS environment. Use your account password.
* With <code>ils</code> you can see your available iRODS collections. You need that as a destination location for <code>itape</code>.
* We have a function to ease uploads (use <code>-h</code> for help): <code>itape</code>.
* We have aliases to ease checking the status of your archive process (it takes a while): <code>itapestat</code> and <code>itapestatnp</code>. The first is for human use — it shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trash bin. The advantage is that you can retrieve it if the removal was in error; the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5 TB.

== Retrieving data from tape ==
As the tape storage is maintained by a different team, it is best to follow their documentation on retrieval.

For that, see here: https://irods.wur.nl/userguide/archiving_research_data_using_icommands/

== See also ==
* [[Storage Systems Overview]]
* [[Backup Policy]]
* [[Compute Storage]]
* [[Tariffs|Costs associated with resource usage]]

== External links ==
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]

Archival Storage

2026-08-25T12:17:33Z

Honfi001:

<code>/archive</code> is Anunna's warm storage tier for data you want to keep but are not actively computing on. Storing data there costs less than on [[Compute Storage|Lustre]], and datasets that have not been used for a long time are moved on to long-term tape archive — automatically, through the [[#Tapeworm|Tapeworm]] service — to keep the warm tier free for active work.

== The /archive filesystem ==

<code>/archive</code> is a mount that is only accessible from the login nodes, it cannot be used for compute work, and it is only available to WUR users.

<code>/archive</code> is backed up periodically on the same schedule as your [[Home Directory|home directory]] — restorable from roughly a week of history (see [[Backup Policy]]). That backup is separate from the long-term tape archival described below: a backup is a short-term safety copy, while archival moves data off <code>/archive</code> for long-term keeping.

== Tapeworm ==

Tapeworm (https://tapeworm.anunna.wur.nl/) helps you manage data on <code>/archive</code> by identifying datasets that are no longer actively used and preparing them for tape archival. The goal is simple: keep our warm storage available for active work, while safely preserving older data on tape.

With Tapeworm, you can:
* See which of your datasets are being considered for tape archival
* Review planned moves before they happen
* Approve, snooze, or block moves when needed
* Add metadata to help describe archived datasets. The metadata is included on tape and can be used to view/retrieve from tape, should you need to do so in the future

If you do nothing, Tapeworm will continue with the planned move after the review period. That is why we recommend checking your pending actions regularly. You will also receive notification emails about pending actions.

From time to time, the user pages may temporarily be unavailable during maintenance. In that case, Tapeworm will show a short maintenance page instead of the normal interface.

=== How Tapeworm works ===

# Tapeworm scans <code>/archive</code> and builds an index of paths and filesystem metadata.
# At the dataset discovery step, it groups those paths into datasets, determines size, owner, and last active use, and makes them available in the GUI. Very small datasets are filtered out and will not be shown.
# A policy engine checks which discovered datasets look inactive (for example: 60+ days old and larger than 1GB).
# Matching datasets are marked as '''planned''' and shown in your overview.
# You will be notified by email that Tapeworm plans to move data you own.
# You can review and change what should happen, or block the move(s) entirely.
# If no action is taken, after a wait period of 4 weeks, planned moves can become scheduled and then executed.
# Data is moved to tape, and removed from <code>/archive</code>.

=== Who sees what? ===

* '''Regular users''' see only their own datasets and actions.
* '''Group admins/contacts''' see data for their configured group(s), in addition to their own data.

=== User pages ===

==== 1) Overview ====
This is your action page. It shows items that currently need your decision.
[[File:Screenshot from 2026-03-17 15-19-46.png|none|thumb|1400x1400px]]
For each candidate, you can:
* '''Approve''': proceed with the tape move. It will schedule for the next day
* '''Deny''': stop this move, configure an override for this path. Tapeworm will not try to move this dataset/path again, until you choose to remove the override
* '''Snooze''': postpone the decision to a future date
* '''Edit metadata''': add key/value notes for archived data. These values are included on tape and can be used to view/retrieve datasets on tape

You can also select multiple rows and apply actions in bulk.

==== 2) Datasets ====
This page shows your discovered datasets, their sizes, and last activity times. The application has no concept of what data belongs together and should be considered a 'dataset'. If the selections on this page are wrong, you can change how Tapeworm should handle these datasets instead.
[[File:Screenshot from 2026-03-17 15-20-23.png|none|thumb|1400x1400px]]
Important:
* If a dataset already has an active move candidate, scheduling controls are disabled.
The dataset list is informational; move decisions are handled through the Schedule page.

==== 3) Schedule ====
This page shows move candidates and their status over time.
[[File:Screenshot from 2026-03-17 15-20-58.png|none|thumb|1400x1400px]]
Common statuses:
* '''Planned''': under review
* '''Scheduled''': move is planned for a specific date
* '''Executing / Tape staged / On tape''': move is in progress or completed
* '''Error''': move needs admin attention. You may be contacted, maybe we resolve it ourselves :)

Once a move is already executing or completed, schedule-changing actions are locked.

==== 4) Overrides ====
Overrides tell Tapeworm to ignore specific paths in future planning.
[[File:Screenshot from 2026-03-17 15-21-49.png|none|thumb|1400x1400px]]
Use overrides when:
* a project is still active and needs to remain on /archive
* policy suggestions are not appropriate for that location

If you agree that the dataset can in principle be moved to tape, but you don't (yet) know when, you can choose to postpone/snooze the archival instead of overriding it.

Overrides apply to the selected path and everything below it.

==== 5) History ====
This page shows completed archival moves. When a dataset has been successfully archived and finalized, it is removed from active scheduling pages and moved into history.
[[File:Screenshot from 2026-03-17 15-24-40.png|none|thumb|1400x1400px]]

=== Group pages ===

Group admins have a separate set of pages for their group scope:
* '''Group overview'''
* '''Group datasets'''
* '''Group schedule'''
* '''Group overrides'''
* '''Group history'''

If you manage more than one group, you can switch group scope in the selector at the top of the group pages.

=== Notifications (email) ===

Tapeworm sends email updates when actions are pending, dates are approaching, or move state changes happen.

Emails typically include:
* Dataset path
* Size
* Last activity
* Current status
* Review/scheduled date

Notification types you may receive:
* '''Action required''': please approve, snooze, or deny
* '''Reminder''': review date is approaching
* '''Informational''': move status changed (for example scheduled, staging, or completed)
* '''Escalation''': sent to group contacts when no user response is received

Please read these emails carefully — they are your chance to adjust decisions before execution.

=== Best practices for users ===

* Check your '''Overview''' page regularly
* Use '''Snooze''' if you need time to validate impact
* Add '''metadata''' when approving important datasets
* Use '''Overrides''' for known and persisting exceptions
* If unsure, contact HPC support before a scheduled move date

=== Tapeworm FAQ ===

==== What happens if I do nothing? ====
Planned items can move forward automatically after the review window (4 weeks).

==== Can I undo after tape staging? ====
Not directly in Tapeworm. Retrieval is done via the tape/iRODS workflow. See: https://irods.wur.nl/userguide/tape_retrieval/

==== What does "completed" mean? ====
Completed means Tapeworm saw the tape workflow finish and finalized the move. Before finalization, the system verifies the archive in iRODS and only then removes the staged source copy.

==== Why is an action button disabled? ====
Usually because the move has already progressed (executing/staged/on tape/error), so schedule edits are no longer valid.

==== Why do some paths on /archive/ not appear as datasets in Tapeworm? ====
Tapeworm only shows paths that are discovered as datasets and pass a minimum-size threshold. Very small paths, single text files, and other tiny items are intentionally filtered out.

=== Need help? ===
If anything is unclear, or you think a move is incorrect but you cannot alter it in the provided GUI, please open an HPC support ticket.

== Manual tape access with iRODS besides Tapeworm ==

Anunna hosts its own iRODS instance, with which you can push data manually to the WUR tape storage for archiving. For general usage information, see https://irods.wur.nl/. The best course of action is to loosely follow the course using your own data, and use your personal space for data upload and transfer to tape.

'''Be sure to check whether the data is correctly stored on tape before you remove your data.'''

On Anunna there are some differences and additions to the linked site:

* The zone is <code>HPC</code>.
* With <code>iinit</code> you can initialise the iRODS environment. Use your account password.
* With <code>ils</code> you can see your available iRODS collections. You need that as a destination location for <code>itape</code>.
* We have a function to ease uploads (use <code>-h</code> for help): <code>itape</code>.
* We have aliases to ease checking the status of your archive process (it takes a while): <code>itapestat</code> and <code>itapestatnp</code>. The first is for human use — it shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trash bin. The advantage is that you can retrieve it if the removal was in error; the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5 TB.

== Retrieving data from tape ==
As the tape storage is maintained by a different team, it is best to follow their documentation on retrieval.

For that, see here: https://irods.wur.nl/userguide/archiving_research_data_using_icommands/

== See also ==
* [[Storage Systems Overview]]
* [[Backup Policy]]
* [[Compute Storage]]
* [[Tariffs|Costs associated with resource usage]]

== External links ==
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]

Archival Storage

2026-08-25T12:16:29Z

Honfi001:

<code>/archive</code> is Anunna's warm storage tier for data you want to keep but are not actively computing on. Storing data there costs less than on [[Compute Storage|Lustre]], and datasets that have not been used for a long time are moved on to long-term tape archive — automatically, through the [[#Tapeworm|Tapeworm]] service — to keep the warm tier free for active work.

== The /archive filesystem ==

<code>/archive</code> is a mount that is only accessible from the login nodes, it cannot be used for compute work, and it is only available to WUR users.

<code>/archive</code> is backed up periodically on the same schedule as your [[Home Directory|home directory]] — restorable from roughly a week of history (see [[Backup Policy]]). That backup is separate from the long-term tape archival described below: a backup is a short-term safety copy, while archival moves data off <code>/archive</code> for long-term keeping.

== Tapeworm ==

Tapeworm (https://tapeworm.anunna.wur.nl/) helps you manage data on <code>/archive</code> by identifying datasets that are no longer actively used and preparing them for tape archival. The goal is simple: keep our warm storage available for active work, while safely preserving older data on tape.

With Tapeworm, you can:
* See which of your datasets are being considered for tape archival
* Review planned moves before they happen
* Approve, snooze, or block moves when needed
* Add metadata to help describe archived datasets. The metadata is included on tape and can be used to view/retrieve from tape, should you need to do so in the future

If you do nothing, Tapeworm will continue with the planned move after the review period. That is why we recommend checking your pending actions regularly. You will also receive notification emails about pending actions.

From time to time, the user pages may temporarily be unavailable during maintenance. In that case, Tapeworm will show a short maintenance page instead of the normal interface.

=== How Tapeworm works ===

# Tapeworm scans <code>/archive</code> and builds an index of paths and filesystem metadata.
# At the dataset discovery step, it groups those paths into datasets, determines size, owner, and last active use, and makes them available in the GUI. Very small datasets are filtered out and will not be shown.
# A policy engine checks which discovered datasets look stale (for example: 60+ days old and larger than 1GB).
# Matching datasets are marked as '''planned''' and shown in your overview.
# You will be notified by email that Tapeworm plans to move data you own.
# You can review and change what should happen, or block the move(s) entirely.
# If no action is taken, after a wait period of 4 weeks, planned moves can become scheduled and then executed.
# Data is moved to tape, and removed from <code>/archive</code>.

=== Who sees what? ===

* '''Regular users''' see only their own datasets and actions.
* '''Group admins/contacts''' see data for their configured group(s), in addition to their own data.

=== User pages ===

==== 1) Overview ====
This is your action page. It shows items that currently need your decision.
[[File:Screenshot from 2026-03-17 15-19-46.png|none|thumb|1400x1400px]]
For each candidate, you can:
* '''Approve''': proceed with the tape move. It will schedule for the next day
* '''Deny''': stop this move, configure an override for this path. Tapeworm will not try to move this dataset/path again, until you choose to remove the override
* '''Snooze''': postpone the decision to a future date
* '''Edit metadata''': add key/value notes for archived data. These values are included on tape and can be used to view/retrieve datasets on tape

You can also select multiple rows and apply actions in bulk.

==== 2) Datasets ====
This page shows your discovered datasets, their sizes, and last activity times. The application has no concept of what data belongs together and should be considered a 'dataset'. If the selections on this page are wrong, you can change how Tapeworm should handle these datasets instead.
[[File:Screenshot from 2026-03-17 15-20-23.png|none|thumb|1400x1400px]]
Important:
* If a dataset already has an active move candidate, scheduling controls are disabled.
The dataset list is informational; move decisions are handled through the Schedule page.

==== 3) Schedule ====
This page shows move candidates and their status over time.
[[File:Screenshot from 2026-03-17 15-20-58.png|none|thumb|1400x1400px]]
Common statuses:
* '''Planned''': under review
* '''Scheduled''': move is planned for a specific date
* '''Executing / Tape staged / On tape''': move is in progress or completed
* '''Error''': move needs admin attention. You may be contacted, maybe we resolve it ourselves :)

Once a move is already executing or completed, schedule-changing actions are locked.

==== 4) Overrides ====
Overrides tell Tapeworm to ignore specific paths in future planning.
[[File:Screenshot from 2026-03-17 15-21-49.png|none|thumb|1400x1400px]]
Use overrides when:
* a project is still active and needs to remain on /archive
* policy suggestions are not appropriate for that location

If you agree that the dataset can in principle be moved to tape, but you don't (yet) know when, you can choose to postpone/snooze the archival instead of overriding it.

Overrides apply to the selected path and everything below it.

==== 5) History ====
This page shows completed archival moves. When a dataset has been successfully archived and finalized, it is removed from active scheduling pages and moved into history.
[[File:Screenshot from 2026-03-17 15-24-40.png|none|thumb|1400x1400px]]

=== Group pages ===

Group admins have a separate set of pages for their group scope:
* '''Group overview'''
* '''Group datasets'''
* '''Group schedule'''
* '''Group overrides'''
* '''Group history'''

If you manage more than one group, you can switch group scope in the selector at the top of the group pages.

=== Notifications (email) ===

Tapeworm sends email updates when actions are pending, dates are approaching, or move state changes happen.

Emails typically include:
* Dataset path
* Size
* Last activity
* Current status
* Review/scheduled date

Notification types you may receive:
* '''Action required''': please approve, snooze, or deny
* '''Reminder''': review date is approaching
* '''Informational''': move status changed (for example scheduled, staging, or completed)
* '''Escalation''': sent to group contacts when no user response is received

Please read these emails carefully — they are your chance to adjust decisions before execution.

=== Best practices for users ===

* Check your '''Overview''' page regularly
* Use '''Snooze''' if you need time to validate impact
* Add '''metadata''' when approving important datasets
* Use '''Overrides''' for known and persisting exceptions
* If unsure, contact HPC support before a scheduled move date

=== Tapeworm FAQ ===

==== What happens if I do nothing? ====
Planned items can move forward automatically after the review window (4 weeks).

==== Can I undo after tape staging? ====
Not directly in Tapeworm. Retrieval is done via the tape/iRODS workflow. See: https://irods.wur.nl/userguide/tape_retrieval/

==== What does "completed" mean? ====
Completed means Tapeworm saw the tape workflow finish and finalized the move. Before finalization, the system verifies the archive in iRODS and only then removes the staged source copy.

==== Why is an action button disabled? ====
Usually because the move has already progressed (executing/staged/on tape/error), so schedule edits are no longer valid.

==== Why do some paths on /archive/ not appear as datasets in Tapeworm? ====
Tapeworm only shows paths that are discovered as datasets and pass a minimum-size threshold. Very small paths, single text files, and other tiny items are intentionally filtered out.

=== Need help? ===
If anything is unclear, or you think a move is incorrect but you cannot alter it in the provided GUI, please open an HPC support ticket.

== Manual tape access with iRODS besides Tapeworm ==

Anunna hosts its own iRODS instance, with which you can push data manually to the WUR tape storage for archiving. For general usage information, see https://irods.wur.nl/. The best course of action is to loosely follow the course using your own data, and use your personal space for data upload and transfer to tape.

'''Be sure to check whether the data is correctly stored on tape before you remove your data.'''

On Anunna there are some differences and additions to the linked site:

* The zone is <code>HPC</code>.
* With <code>iinit</code> you can initialise the iRODS environment. Use your account password.
* With <code>ils</code> you can see your available iRODS collections. You need that as a destination location for <code>itape</code>.
* We have a function to ease uploads (use <code>-h</code> for help): <code>itape</code>.
* We have aliases to ease checking the status of your archive process (it takes a while): <code>itapestat</code> and <code>itapestatnp</code>. The first is for human use — it shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trash bin. The advantage is that you can retrieve it if the removal was in error; the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5 TB.

== Retrieving data from tape ==
As the tape storage is maintained by a different team, it is best to follow their documentation on retrieval.

For that, see here: https://irods.wur.nl/userguide/archiving_research_data_using_icommands/

== See also ==
* [[Storage Systems Overview]]
* [[Backup Policy]]
* [[Compute Storage]]
* [[Tariffs|Costs associated with resource usage]]

== External links ==
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]

Archival Storage

2026-08-25T12:13:17Z

Honfi001:

<code>/archive</code> is Anunna's warm storage tier for data you want to keep but are not actively computing on. Storing data there costs less than on [[Compute Storage|Lustre]], and datasets that have not been used for a long time are moved on to long-term tape archive — automatically, through the [[#Tapeworm|Tapeworm]] service — to keep the warm tier free for active work.

== The /archive filesystem ==

<code>/archive</code> is a mount that is only accessible from the login nodes, it cannot be used for compute work, and it is only available to WUR users.

<code>/archive</code> is backed up periodically on the same schedule as your [[Home Directory|home directory]] — restorable from roughly a week of history (see [[Backup Policy]]). That backup is separate from the long-term tape archival described below: a backup is a short-term safety copy, while archival moves data off <code>/archive</code> for long-term keeping.

== Tapeworm ==

Tapeworm (https://tapeworm.anunna.wur.nl/) helps you manage data on <code>/archive</code> by identifying datasets that are no longer actively used and preparing them for tape archival. The goal is simple: keep our warm storage available for active work, while safely preserving older data on tape.

With Tapeworm, you can:
* See which of your datasets are being considered for tape archival
* Review planned moves before they happen
* Approve, snooze, or block moves when needed
* Add metadata to help describe archived datasets. The metadata is included on tape and can be used to view/retrieve from tape, should you need to do so in the future

If you do nothing, Tapeworm will continue with the planned move after the review period. That is why we recommend checking your pending actions regularly. You will also receive notification emails about pending actions.

From time to time, the user pages may temporarily be unavailable during maintenance. In that case, Tapeworm will show a short maintenance page instead of the normal interface.

=== How Tapeworm works ===

# Tapeworm scans <code>/archive</code> and builds an index of paths and filesystem metadata.
# A dataset discovery step groups those paths into datasets, determines size, owner, and last active use, and makes them available in the GUI. Very small datasets are filtered out and will not be shown.
# A policy engine checks which discovered datasets look stale (for example: 60+ days old and larger than 1GB).
# Matching datasets are marked as '''planned''' and shown in your overview.
# You will be notified by email that Tapeworm plans to move data you own.
# You can review and change what should happen, or block the move(s) entirely.
# If no action is taken, after a wait period of 4 weeks, planned moves can become scheduled and then executed.
# Data is moved to tape, and removed from <code>/archive</code>.

=== Who sees what? ===

* '''Regular users''' see only their own datasets and actions.
* '''Group admins/contacts''' see data for their configured group(s), in addition to their own data.

=== User pages ===

==== 1) Overview ====
This is your action page. It shows items that currently need your decision.
[[File:Screenshot from 2026-03-17 15-19-46.png|none|thumb|1400x1400px]]
For each candidate, you can:
* '''Approve''': proceed with the tape move. It will schedule for the next day
* '''Deny''': stop this move, configure an override for this path. Tapeworm will not try to move this dataset/path again, until you choose to remove the override
* '''Snooze''': postpone the decision to a future date
* '''Edit metadata''': add key/value notes for archived data. These values are included on tape and can be used to view/retrieve datasets on tape

You can also select multiple rows and apply actions in bulk.

==== 2) Datasets ====
This page shows your discovered datasets, their sizes, and last activity times. The application has no concept of what data belongs together and should be considered a 'dataset'. If the selections on this page are wrong, you can change how Tapeworm should handle these datasets instead.
[[File:Screenshot from 2026-03-17 15-20-23.png|none|thumb|1400x1400px]]
Important:
* If a dataset already has an active move candidate, scheduling controls are disabled.
The dataset list is informational; move decisions are handled through the Schedule page.

==== 3) Schedule ====
This page shows move candidates and their status over time.
[[File:Screenshot from 2026-03-17 15-20-58.png|none|thumb|1400x1400px]]
Common statuses:
* '''Planned''': under review
* '''Scheduled''': move is planned for a specific date
* '''Executing / Tape staged / On tape''': move is in progress or completed
* '''Error''': move needs admin attention. You may be contacted, maybe we resolve it ourselves :)

Once a move is already executing or completed, schedule-changing actions are locked.

==== 4) Overrides ====
Overrides tell Tapeworm to ignore specific paths in future planning.
[[File:Screenshot from 2026-03-17 15-21-49.png|none|thumb|1400x1400px]]
Use overrides when:
* a project is still active and needs to remain on /archive
* policy suggestions are not appropriate for that location

If you agree that the dataset can in principle be moved to tape, but you don't (yet) know when, you can choose to postpone/snooze the archival instead of overriding it.

Overrides apply to the selected path and everything below it.

==== 5) History ====
This page shows completed archival moves. When a dataset has been successfully archived and finalized, it is removed from active scheduling pages and moved into history.
[[File:Screenshot from 2026-03-17 15-24-40.png|none|thumb|1400x1400px]]

=== Group pages ===

Group admins have a separate set of pages for their group scope:
* '''Group overview'''
* '''Group datasets'''
* '''Group schedule'''
* '''Group overrides'''
* '''Group history'''

If you manage more than one group, you can switch group scope in the selector at the top of the group pages.

=== Notifications (email) ===

Tapeworm sends email updates when actions are pending, dates are approaching, or move state changes happen.

Emails typically include:
* Dataset path
* Size
* Last activity
* Current status
* Review/scheduled date

Notification types you may receive:
* '''Action required''': please approve, snooze, or deny
* '''Reminder''': review date is approaching
* '''Informational''': move status changed (for example scheduled, staging, or completed)
* '''Escalation''': sent to group contacts when no user response is received

Please read these emails carefully — they are your chance to adjust decisions before execution.

=== Best practices for users ===

* Check your '''Overview''' page regularly
* Use '''Snooze''' if you need time to validate impact
* Add '''metadata''' when approving important datasets
* Use '''Overrides''' for known and persisting exceptions
* If unsure, contact HPC support before a scheduled move date

=== Tapeworm FAQ ===

==== What happens if I do nothing? ====
Planned items can move forward automatically after the review window (4 weeks).

==== Can I undo after tape staging? ====
Not directly in Tapeworm. Retrieval is done via the tape/iRODS workflow. See: https://irods.wur.nl/userguide/tape_retrieval/

==== What does "completed" mean? ====
Completed means Tapeworm saw the tape workflow finish and finalized the move. Before finalization, the system verifies the archive in iRODS and only then removes the staged source copy.

==== Why is an action button disabled? ====
Usually because the move has already progressed (executing/staged/on tape/error), so schedule edits are no longer valid.

==== Why do some paths on /archive/ not appear as datasets in Tapeworm? ====
Tapeworm only shows paths that are discovered as datasets and pass a minimum-size threshold. Very small paths, single text files, and other tiny items are intentionally filtered out.

=== Need help? ===
If anything is unclear, or you think a move is incorrect but you cannot alter it in the provided GUI, please open an HPC support ticket.

== Manual tape access with iRODS besides Tapeworm ==

Anunna hosts its own iRODS instance, with which you can push data manually to the WUR tape storage for archiving. For general usage information, see https://irods.wur.nl/. The best course of action is to loosely follow the course using your own data, and use your personal space for data upload and transfer to tape.

'''Be sure to check whether the data is correctly stored on tape before you remove your data.'''

On Anunna there are some differences and additions to the linked site:

* The zone is <code>HPC</code>.
* With <code>iinit</code> you can initialise the iRODS environment. Use your account password.
* With <code>ils</code> you can see your available iRODS collections. You need that as a destination location for <code>itape</code>.
* We have a function to ease uploads (use <code>-h</code> for help): <code>itape</code>.
* We have aliases to ease checking the status of your archive process (it takes a while): <code>itapestat</code> and <code>itapestatnp</code>. The first is for human use — it shows a paginated status of all your files. The latter dumps all the info, so you can e.g. use grep to filter.
* If you remove data with <code>irm</code> within iRODS, the data isn't actually removed but moved to a trash bin. The advantage is that you can retrieve it if the removal was in error; the disadvantage is that the data will keep costing money. To empty it, see <code>irmtrash -h</code>.

Because of hardware limitations on the backend tape storage, the size limit per file for our tape archive is 5 TB.

== Retrieving data from tape ==
As the tape storage is maintained by a different team, it is best to follow their documentation on retrieval.

For that, see here: https://irods.wur.nl/userguide/archiving_research_data_using_icommands/

== See also ==
* [[Storage Systems Overview]]
* [[Backup Policy]]
* [[Compute Storage]]
* [[Tariffs|Costs associated with resource usage]]

== External links ==
* [http://wiki.lustre.org/index.php/Main_Page Lustre website]

Python

2026-08-24T14:24:06Z

Honfi001:

Python

2026-08-24T14:22:04Z

Honfi001:

Uv

2026-08-24T14:19:50Z

Honfi001: Created page with "{{DISPLAYTITLE:uv}} uv is a tool for installing Python packages and for managing the environments they live in. It does the same work as <code>pip</code> and <code>venv</code>, and it does it a great deal faster. This page explains what uv is, how to load it on Anunna, where it keeps your files, and how to use it for everyday work. It also explains what uv is '''not''' the right tool for: the packages it installs are not built for our hardware, so when the speed of you..."

{{DISPLAYTITLE:uv}}

uv is a tool for installing Python packages and for managing the environments they live in. It does the same work as <code>pip</code> and <code>venv</code>, and it does it a great deal faster.

This page explains what uv is, how to load it on Anunna, where it keeps your files, and how to use it for everyday work. It also explains what uv is '''not''' the right tool for: the packages it installs are not built for our hardware, so when the speed of your calculation matters, the Python modules remain the better choice.

== Introduction ==

=== What uv is ===

uv is a single program that covers work you would otherwise need several separate tools for. On Anunna it is available as a module, already set up for the cluster.

=== What uv does ===

{| class="wikitable"
! uv can !! Otherwise you would use
|-
| Install Python packages || <code>pip</code>
|-
| Create and manage environments || <code>venv</code>
|-
| Download other versions of Python || a Python module, if one exists
|}

=== What it is faster at ===

Installing packages, and building the environments that hold them. uv produces the same result as <code>pip</code> — the very same packages, from the very same place — in a fraction of the time.

That is worth more here than it would be on your own computer. Installing an environment means writing tens of thousands of small files onto a filesystem that everybody on the cluster shares, so finishing sooner is better for you and better for everyone else.

=== What it is not faster at ===

Running your code.

The packages uv installs are general-purpose builds, made to work on any machine in the world. They are exactly what <code>pip</code> would have given you. The software we provide as modules is compiled for the precise processors in each generation of Anunna's nodes, and that is where the speed of a calculation comes from. If your work is heavy, use the modules — see [[Python/Modules]].

The same applies to Python itself. uv is able to download a Python for you, but it is a general-purpose build too. The module is set up to use the Python you have loaded rather than fetching one.

=== What we have already set up ===

Left alone, uv keeps its downloads and environments in your home directory, which is small, backed up, and slower than Lustre. The Anunna module points all of it at Lustre instead, and there is nothing you need to do to make that happen. The next two sections cover loading the module and where your files end up.

== Loading uv ==

uv lives in the '''2025''' bucket:

<syntaxhighlight lang="bash">
module load 2025 uv
</syntaxhighlight>

Loading uv also loads Python 3.13.1, so you do not need to load a Python module first.

Software from different buckets should not be mixed, so anything you use alongside uv should come from the 2025 bucket as well. See [[Environment Modules#Software buckets]].

To check that it worked:

<syntaxhighlight lang="bash">
uv --version
</syntaxhighlight>

A second command, <code>uvx</code>, arrives together with <code>uv</code>. It is described in [[#Other things uv can do]].

== Where uv keeps your files ==

uv accumulates three kinds of thing as you use it: the packages it has downloaded, any Python versions it has fetched, and the tools you have installed with it. On Anunna the module places all of them on Lustre, under a folder called <code>uv</code> in your nobackup location — <code>$myNoBkp/uv</code>. Nobackup is the right tier because everything in there can be downloaded again.

The module does this by setting a handful of variables for you:

{| class="wikitable"
! Variable !! What it points at
|-
| <code>UV_DIR</code> || The folder that holds all of the others. This is the only one you might want to change.
|-
| <code>UV_CACHE_DIR</code> || Packages uv has downloaded. Normally the largest of them.
|-
| <code>UV_PYTHON_INSTALL_DIR</code> || Python versions uv has downloaded.
|-
| <code>UV_TOOL_DIR</code> || Environments for tools installed with <code>uv tool install</code>.
|-
| <code>UV_TOOL_BIN_DIR</code> || The programs those tools provide. The module has already added this to your <code>PATH</code>.
|-
| <code>UV_PYTHON_BIN_DIR</code> || Shortcuts to downloaded Python versions.
|}

=== Putting them somewhere else ===

Every folder in that table sits inside <code>UV_DIR</code>, so setting that one variable moves all of them. Set it '''before''' loading the module:

<syntaxhighlight lang="bash">
export UV_DIR=$myScratch/uv
module load 2025 uv
</syntaxhighlight>

To see where uv is actually working, ask:

<syntaxhighlight lang="bash">
echo $UV_DIR
</syntaxhighlight>

Use that rather than <code>$myNoBkp</code>, which is only set once you have added it to your shell yourself — see [[Installing Personal Software#Aliases and local variables]].

=== Keeping it from growing ===

The cache is meant to grow; that is part of what makes uv fast. It still counts against your Lustre quota, so clear it out from time to time:

<syntaxhighlight lang="bash">
uv cache prune # remove entries nothing is using
uv cache clean # remove everything
</syntaxhighlight>

== Creating an environment and installing packages ==

=== Never build an environment on a login node ===

Building an environment writes thousands of small files at once. A login node is shared by every user on the cluster, so that work is felt by all of them rather than only by you. Ask for an interactive session on a compute node and work there:

<syntaxhighlight lang="bash">
sinteractive -c 4 --mem 16G --time 120
</syntaxhighlight>

See [[Interactive Jobs]] for the options <code>sinteractive</code> accepts.

=== Creating an environment ===

<syntaxhighlight lang="bash">
module load 2025 uv
uv venv $myNoBkp/PythonEnv/my_env
</syntaxhighlight>

Environments belong on Lustre rather than in your home directory, and nobackup is the right tier because an environment can always be built again from the list of packages in it.

If you run <code>uv venv</code> without giving a path, uv creates a folder named <code>.venv</code> in whatever directory you happen to be in. Giving the path yourself is clearer, and it keeps your environments together in one place.

=== Using it ===

Activate the environment whenever you want it:

<syntaxhighlight lang="bash">
source $myNoBkp/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Its name then appears at the front of your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages into it:

<syntaxhighlight lang="bash">
uv pip install numpy pandas
</syntaxhighlight>

Leave it again with <code>deactivate</code>.

What uv builds is an ordinary virtual environment. <code>pip</code> still works inside it if you prefer, and it can be turned into a Jupyter kernel exactly as [[Python/Modules#Jupyter kernels]] describes.

=== The commands you already know ===

{| class="wikitable"
! If you know this !! Use this
|-
| <code>python -m venv my_env</code> || <code>uv venv my_env</code>
|-
| <code>source my_env/bin/activate</code> || unchanged
|-
| <code>pip install numpy</code> || <code>uv pip install numpy</code>
|-
| <code>pip list</code> || <code>uv pip list</code>
|}

== Other things uv can do ==

=== Running a tool without installing it ===

<code>uvx</code> fetches a command-line program, runs it, and does not leave an environment behind:

<syntaxhighlight lang="bash">
uvx ruff check .
</syntaxhighlight>

=== Installing a tool you use often ===

<syntaxhighlight lang="bash">
uv tool install ruff
</syntaxhighlight>

The program lands in your uv folder, which the module has already added to your <code>PATH</code>, so you can run it straight away and in any later session.

=== Using a different version of Python ===

If you need a version of Python we do not provide as a module, uv can fetch one:

<syntaxhighlight lang="bash">
uv python install 3.12
uv venv --python 3.12 $myNoBkp/PythonEnv/py312
</syntaxhighlight>

Remember that these are general-purpose builds, like the packages. They are convenient, and they are the right choice when you need a specific version for compatibility — but if the speed of your calculation matters, a Python module is the better starting point.

'''Do not run <code>uv python update-shell</code>.''' It writes a fixed path into your <code>~/.bashrc</code> and <code>~/.bash_profile</code>. That applies in every shell you ever open, whether or not the module is loaded, and it stops being correct the moment your uv folder moves.

== Using uv in a job script ==

Build the environment '''before''' you submit the job, not inside it. Create it once in an interactive session, as above, and let the job simply use it. An environment built inside the job is rebuilt on every run, and when many jobs start at the same time they all write to the filesystem at once.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

module load 2025 uv
source $myNoBkp/PythonEnv/my_env/bin/activate

python myscript.py
</syntaxhighlight>

Both the <code>module load</code> and the <code>source</code> have to be inside the script. A job does not inherit the shell you submitted it from, so anything you loaded by hand beforehand is not there when the job runs. This is the most common reason a script that worked interactively fails under <code>sbatch</code>.

See [[Batch Jobs]] for what else can go in the <code>#SBATCH</code> lines.

== FAQ ==

; <code>module load uv</code> says there is no such module.
: The bucket has to be loaded first. Use <code>module load 2025 uv</code>. See [[Environment Modules#Software buckets]].

; Where did uv put my files?
: <code>echo $UV_DIR</code> gives the folder they are all under. To hear it from uv itself, <code>uv cache dir</code> and <code>uv tool dir</code>.

; uv warned that it could not find a Lustre directory.
: The module could not work out where your Lustre folders are, so uv has fallen back to your home directory — which will eventually fill up. Choose a location yourself with <code>export UV_DIR=$myNoBkp/uv</code> and then load the module again.

; My home directory is still filling up, even though uv uses Lustre.
: If you used uv before this module existed, an old cache is probably still sitting in <code>~/.cache/uv</code>. uv no longer looks there, so its own cleaning commands cannot reach it. Check its size with <code>du -sh ~/.cache/uv</code> and delete it by hand if you no longer need it.

; Can I still use <code>pip</code> inside a uv environment?
: Yes. It is an ordinary virtual environment and <code>pip</code> behaves normally in it.

== See also ==

* [[Python]]
* [[Python/Modules]]
* [[Python/Conda]]
* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Quotas]]

== External links ==

* [https://docs.astral.sh/uv/ uv documentation]
* [https://github.com/astral-sh/uv uv on GitHub]

FrequentlyAskedQuestions

2026-08-24T10:25:12Z

Honfi001: Created page with "{{DISPLAYTITLE:Frequently Asked Questions}}"

External Collaborator Access

2026-08-24T10:18:33Z

Honfi001:

Anunna is an open service available to users and groups outside of the WUR, hence access to external researchers and entities can be facilitated.

== Giving a collaborator access ==

Every Anunna user needs a valid user account associated with one of the groups registered at Anunna. Therefore external users need to be sponsored by an existing group, that means that the sponsoring group accepts to bear the costs accrued by the external user and the external user will gain access to the folders common to the group — see [[Managing Group Members]] and [[Shared Storage]].

Alternatively, external users, who want to be their own independent entity and want to pay for their own usage of the HPC, a contract must be signed.

For instructions on how to get the process started — see [[Account Application Process]] (the external-user route). As the WUR contact, you will usually need to sponsor the request and name the project.

=== Required Information ===
When requesting an account for an external user please include the following details

* First name
* Last Name
* Valid e-mail address
* Sponsoring group
* Expiration date for the account
* Name or username of supervisor

The supervisor will be responsible for the data of the external user once the account expires.

== See also ==
* [[Account Application Process]]
* [[Who Can Access?]]
* [[Managing Group Members]]
* [[How to Get Help]]

Network & Security

2026-08-19T15:04:05Z

Honfi001:

This page describes how Anunna is connected and how access to it is secured.

== Network ==

Internally inside the cluster, the compute nodes and the[[Filesystems | Lustre filesystem]] are linked by OmniPath, a high-speed, low-latency interconnect, which is what lets the parallel filesystem and multi-node jobs perform well.

From outside, the cluster services are reached through two loadbalancers, which are connected to the internet over 100Gbit links.

SSH to access <code>login.anunna.wur.nl</code>, other services can be reached through their respective URLs. Access to some services are restricted to certain IP ranges, and are thus are only reachable over the WUR VPN or from within WURNET. See [[Log in to Anunna]] for how to connect.

== Authentication ==

Anunna is public facing, servicing organisation withing and outside of the WUR but access requires an account to be activated by the HPC admins.

Users from the WUR can request an account and once the account is activated, they can login with the same credentials as their WUR account.

In a similar fashion, organisations outside of the WUR can request submit account requests to the admins. Once the account is created, the users receive an email with their login details, which they are advised to change when first logging in.

For stronger protection you can add multi-factor authentication to your SSH access using a hardware security key — see [[Using a hardware key for better security]]. You can also configure key-based login so you are not prompted for a password each time; see [[Ssh without password]].

If you use the wrong credentials, your access may be blocked, see [[Unblocking your IP]]

== Data security ==

For how file and folder permissions, work on the cluster, see [[Shared folders]]. For storing confidential data, contact the HPC team via [[Support]].

== See also ==

* [[Cluster Architecture Overview]]
* [[Log in to Anunna]]
* [[Unblocking your IP]]
* [[Using a hardware key for better security]]
* [[Filesystems]]

Best-Practices

2026-08-19T15:02:17Z

Honfi001:

{{DISPLAYTITLE:Best Practices}}

== Login Node ==

* Do not run analyses or jobs in the login node.

== Home folder ($HOME, ~) ==

* Do not store datasets or large files in your home directory. You have a quota of 200 GB.
* Do not install software that you intend to run in jobs in your $HOME . Install your software in lustre.
* Do not point the logs of your jobs into your $HOME, specially when running array jobs. If not defined slurm will point them to the location you run the sbatch command.
* Do not edit or left software edit your ~/.bashrc or ~/.bash_profile. If something goes wrong you will be locked out.

== Jobs ==

*

Best-Practices

2026-08-19T14:52:15Z

Honfi001:

Best-Practices

2026-08-19T14:51:11Z

Honfi001: Created page with "==Best Practices=="

==Best Practices==

Workflows

2026-08-19T14:46:13Z

Honfi001:

Guidelines and tools for building reproducible, efficient research workflows on Anunna.

== In this section ==

* [[Workflow Migration from Laptop to HPC]] — moving your work to the cluster
* [[Reproducibility Guidelines]] — keeping your work reproducible
* [[Workflow Engines (Snakemake, Nextflow)]] — managing multi-step pipelines
* [[Debugging Jobs]] — working out why a job failed
* [[Checkpointing]] — saving and restarting long jobs
* [[Scheduled tasks (cron)]] — running recurring tasks with scrontab

=== Parallel Workflows ===

* [[Workflows/Parallel-Computing|Parallel Computing]]
* [[Workflows/Serial|Serial]]
* [[Workflows/Embarassinly-Parallel|Embarrassingly Parallel]]
* [[Workflows/Multi-threaded|Multi-threaded]]
* [[Workflows/Multi-Process|Multi-Process]]
* Hybrid Workflows
* GPU workflows

Python/uv

2026-08-18T14:27:13Z

Environment Modules

2026-08-18T12:46:02Z

Honfi001:

Anunna provides software through '''environment modules''', managed with [https://lmod.readthedocs.io/ Lmod]. A module configures your shell — and the environment of your jobs — so that a chosen application and the right version of its dependencies are available. Because each module lives in its own self-contained tree, modules let several otherwise-conflicting programs and several versions of the same program coexist on the cluster without clashing.

== Software buckets ==

Modules on Anunna are organised into '''buckets'''. Each bucket is a snapshot of a particular compiler-and-toolchain generation; all the software in a bucket is built with the same compiler. This matters because mixing software built with different compilers in one job can cause conflicts, errors, or silently wrong results — so keeping a job's software within a single bucket keeps it consistent.

The buckets currently available are:

* '''legacy''' — old software that is no longer maintained or updated but is still used in active research.
* '''2023''' — software built with the 2023 compilers and toolchain.
* '''2024''' — software built with the 2024 compilers and toolchain.
* '''2024''' — software built with the 2024 compilers and toolchain.
* '''utilities''' — software that does not depend on a specific compiler or toolchain (for example [[Apptainer]]).
* '''groups''' — module files contributed by groups inside and outside WUR.
* '''GPU''' — CUDA, cuDNN, and related packages that are independent of toolchains.

A bucket has to be loaded before its modules become visible. To use the 2025 bucket:

<syntaxhighlight lang="bash">
module load 2025
</syntaxhighlight>

After this, <code>module avail</code> shows the expanded list of modules from that bucket.

The modules are built with [https://easybuild.io/ EasyBuild], which uses publicly shared recipes called easyconfigs. Their [https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs repository] is a good place to check what software is available upstream.

== Requesting modules ==

If a module you need is not available, submit a software request at [https://ideas.anunna.wur.nl https://ideas.anunna.wur.nl]. There you can follow the progress of your request and upvote requests from other users — the more upvotes, the sooner it is prioritised.

== Listing modules ==

Three commands list the available modules in increasing detail. <code>overview</code> gives a top-level view — just the software names and how many versions exist. <code>avail</code> lists the individual versions. <code>spider</code> gives a verbose list with a description of each.

<syntaxhighlight lang="bash">
module overview
module avail
module spider
</syntaxhighlight>

== Searching for modules ==

The same three commands search when you pass a module name as an argument, again at increasing levels of detail:

<syntaxhighlight lang="bash">
module overview <nameOfModule>
module avail <nameOfModule>
module spider <nameOfModule>
</syntaxhighlight>

=== Searching by keyword ===

You can also search for a keyword inside modules with <code>module key</code>. This is useful for finding which module contains a specific Python or R extension — there are bundle modules for both languages that list their extensions, and Lmod searches the module descriptions too.

<syntaxhighlight lang="bash">
module key <keyword>
</syntaxhighlight>

For example, to find which module provides the R package <code>terra</code>, first load a bucket, then search:

<syntaxhighlight lang="bash">
module load 2023
module key terra
</syntaxhighlight>

which yields:

<syntaxhighlight lang="text">
The following modules match your search criteria: "terra"
--------------------------------------------------------------------------------------------------------

R-bundle-CRAN: R-bundle-CRAN/2023.12-foss-2023a
Bundle of R packages from CRAN

--------------------------------------------------------------------------------------------------------
</syntaxhighlight>

So you would load <code>R-bundle-CRAN/2023.12-foss-2023a</code> to get the <code>terra</code> package.

== Loading modules ==

Load a module by name:

<syntaxhighlight lang="bash">
module load <moduleName>
</syntaxhighlight>

For example, to load Python from the 2023 bucket:

<syntaxhighlight lang="bash">
module load 2023
module load Python/3.11.3
</syntaxhighlight>

'''Specify the version''' you load, as in the example. It is good practice for consistency and reproducibility — if you omit the version, Lmod picks whatever the current default is, and that default can change over time. Naming the exact version in a submit script also turns the script into documentation of the environment it ran in. When you load a module, its dependencies are loaded automatically.

=== Listing loaded modules ===

<syntaxhighlight lang="bash">
module list
</syntaxhighlight>

Following the example above, after loading the 2023 bucket and <code>Python/3.11.3</code>, the list shows the module plus the dependencies pulled in with it:

<syntaxhighlight lang="text">
user001@login201:~$ module list

Currently Loaded Modules:
1) slurm/24.05.1 (S) 5) binutils/2.40-GCCcore-12.3.0 9) Tcl/8.6.13-GCCcore-12.3.0 13) OpenSSL/1.1
2) 2023 6) bzip2/1.0.8-GCCcore-12.3.0 10) SQLite/3.42.0-GCCcore-12.3.0 14) Python/3.11.3-GCCcore-12.3.0
3) GCCcore/12.3.0 7) ncurses/6.4-GCCcore-12.3.0 11) XZ/5.4.2-GCCcore-12.3.0
4) zlib/1.2.13-GCCcore-12.3.0 8) libreadline/8.2-GCCcore-12.3.0 12) libffi/3.4.4-GCCcore-12.3.0

Where:
S: Module is Sticky, requires --force to unload or purge
</syntaxhighlight>

The <code>slurm</code> module is loaded by default and marked '''Sticky''' (<code>S</code>) — it survives a normal unload or purge.

== Removing and switching modules ==

Unload a single module:

<syntaxhighlight lang="bash">
module unload Python/3.11.3
</syntaxhighlight>

This unloads only the named module, not its dependencies. To swap one module for another in a single step:

<syntaxhighlight lang="bash">
module switch <oldModule> <newModule>
</syntaxhighlight>

Some modules refuse to load alongside others — for example, two Java modules cannot be loaded at once. Attempting it produces a conflict message:

<syntaxhighlight lang="text">
Module 'foo/2' conflicts with the currently loaded module(s) 'foo/1'
</syntaxhighlight>

When you see this, unload or switch rather than trying to stack the two modules.

To clear the whole environment, use <code>purge</code>:

<syntaxhighlight lang="bash">
module purge
</syntaxhighlight>

Sticky modules such as <code>slurm</code> survive a plain purge:

<syntaxhighlight lang="text">
user001@login201:~$ module purge
The following modules were not unloaded:
(Use "module --force purge" to unload all):

1) slurm/24.05.1
</syntaxhighlight>

Putting <code>module purge</code> (or <code>module reset</code>) near the top of a job script is good practice: it clears anything that might have been loaded by mistake, so the job starts from a known-clean environment.

== See also ==

* [[Installing Personal Software]]
* [[Batch Jobs]]
* [[Python]]
* [[R]]
* [[Apptainer]]
* [[ABGC/Modules]] — group-specific modules for ABGC

== External links ==

* [https://lmod.readthedocs.io/ Lmod documentation]
* [https://easybuild.io/ EasyBuild]
* [https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs EasyBuild easyconfigs repository]

Tutorials

2026-08-17T15:13:52Z

Honfi001:

Hands-on tutorials for working on Anunna. For instructor-led courses see [[Workshops]], and for course slides and self-study material see [[Training Materials]].

== Apptainer (containers) ==

* [[Tutorials/Apptainer-Introduction|Introduction]]
* [[Tutorials/Apptainer-PullingImages|Pulling images]]
* [[Tutorials/Apptainer-FakerootAndOverlays|Fakeroot and overlays]]
* [[Tutorials/Apptainer-FakerootAndSandbox|Fakeroot and sandbox]]
* [[Tutorials/Apptainer-DefinitionFiles|Definition files]]
* [[Tutorials/Apptainer-GPUs|GPUs]]
* [[Tutorials/Apptainer-Conda|Conda in a Container]]

== Linux ==

* [[Linux Basic/Linux Self Assessment]] — check whether your command-line skills are ready for the HPC and container courses.

== Parallelism ==

* [[Tutorials/Parallelism-Estimating-Pi|Estimating Pi]]

== See also ==
* [[Workshops]]
* [[Training Materials]]
* [[Apptainer]]

Tutorials/Apptainer-Conda

2026-08-17T15:10:11Z

Honfi001:

{{DISPLAYTITLE:Tutorial: Conda in an Apptainer Container}}
= Running a Conda Environment in a Container =

Sometimes a piece of software is published only as a conda package, with no module, no Python-only equivalent and no ready-made image. Building it into a container is the cleanest way to run it on Anunna: the environment is created once, on a compute node, and afterwards it is a single file that behaves the same way every time you use it.

Read [[Python/Conda]] first if you have not already. This tutorial is the last resort described at the end of that page, not the recommended route.

'''Important:''' Before you begin, make sure the following are in place:

* You are running on a '''compute node''', not a login node. Request an interactive session first — see [[Interactive Jobs]].
* Your <code>.sif</code> image files should be stored on '''Lustre''', not in your home directory. SIF files can be large and will eat through your home quota fast.
* Set your Apptainer cache to Lustre as well. Add this to your session (or your <code>.bashrc</code>):

<syntaxhighlight lang="bash">
export APPTAINER_CACHEDIR=$myScratch/apptainer_cache
</syntaxhighlight>

== Getting Started ==

Load the required modules:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
</syntaxhighlight>

== Three things to know first ==

* '''A container should not be your first move.''' Always prefer software we already provide as [[Environment Modules|modules]], or a [[Python/Modules#Virtual environments|virtual environment]] built on one of our Python modules. Those are compiled for this machine and we are far better placed to help you when something goes wrong. [[Support|Ask us]] if what you need is missing.

* '''This tutorial uses micromamba, not conda.''' Micromamba is a single small program that installs conda packages, and it takes them from conda-forge and bioconda rather than from Anaconda's own repository. That keeps you clear of the licensing conditions described in [[Python/Conda#How they differ]].

* '''The image we build is read-only.''' Everything the environment needs is decided up front and baked in. Do not try to install software interactively inside a running container: the changes will not survive, and you lose the one real advantage of doing this, which is that the result is reproducible.

== Step 1: Describe the environment ==

Create a file called <code>environment.yml</code>. This is where you name the environment, list the packages you want, and say which repositories to take them from.

<syntaxhighlight lang="yaml">
name: base
channels:
- conda-forge
- bioconda
dependencies:
- python
- pip
- star
- bwa
- multiqc
</syntaxhighlight>

Two notes on this file. Keep the name as <code>base</code>, because that is the environment micromamba's image already has and installing into it saves a step. And for real work, pin your versions — <code>multiqc=1.29</code> rather than <code>multiqc</code> — otherwise rebuilding the image in six months will quietly give you different software.

== Step 2: Write the definition file ==

A '''definition file''' describes how the container is built, step by step. Create <code>image.def</code>:

<syntaxhighlight lang="bash">
Bootstrap: docker
From: mambaorg/micromamba:latest

%files
environment.yml /environment.yml

%post
micromamba install -n base --file /environment.yml && \
micromamba clean --all --yes
</syntaxhighlight>

Each part in turn:

=== <code>Bootstrap</code> and <code>From</code> ===

Start from a Docker image that already has micromamba installed, so we do not have to install it ourselves.

=== <code>%files</code> ===

Copy <code>environment.yml</code> from your directory on Anunna into the container, so the next step can read it.

=== <code>%post</code> ===

Run micromamba inside the container to install everything listed in the file, then delete its download cache. That last part matters: without <code>clean</code>, the cache is baked into the image and can easily double its size.

== Step 3: Build the image ==

<syntaxhighlight lang="bash">
apptainer build image.sif image.def
</syntaxhighlight>

This takes a few minutes and produces a single file, <code>image.sif</code>, containing the whole environment.

If the build fails complaining about directories from the host, clear Apptainer's bind list for the duration of the build:

<syntaxhighlight lang="bash">
APPTAINER_BIND=' ' apptainer build image.sif image.def
</syntaxhighlight>

== Testing the image ==

Check that the software is really there:

<syntaxhighlight lang="bash">
apptainer run image.sif multiqc --help
</syntaxhighlight>

You should see MultiQC's own help text, beginning with its version number and a usage line. If instead you get an error saying the command was not found, read the note below.

'''Use <code>run</code>, not <code>exec</code>.''' The micromamba image activates the environment as it starts up, and <code>apptainer run</code> goes through that startup step while <code>apptainer exec</code> skips it. With <code>exec</code> you will most likely be told the command cannot be found, even though it is installed.

== Using it in a job ==

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=multiqc
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

module load utilities Apptainer

apptainer run --bind /lustre $myNobackup/images/image.sif multiqc /path/to/results
</syntaxhighlight>

Apptainer does not automatically make every filesystem visible inside the container, so <code>--bind /lustre</code> ensures your data can be read.

== Where to keep your images ==

Image files are large. Keep them on Lustre, under <code>$myNobackup</code>, and not in your home directory.

If several people in your group use the same software, put one copy somewhere shared instead of each keeping their own — see [[Shared Storage]]. A container is a single file and is identical for everyone who runs it, which is precisely what makes sharing it worthwhile.

One habit worth forming: identify an image by what is inside it, using <code>apptainer inspect</code>, rather than by trusting its file name.

== Summary ==

{| class="wikitable"
! Step !! Command
|-
| Load Apptainer || <code>module reset && module load utilities Apptainer</code>
|-
| Describe the environment || Write <code>environment.yml</code>
|-
| Describe the container || Write <code>image.def</code>
|-
| Build the image || <code>apptainer build image.sif image.def</code>
|-
| Run something in it || <code>apptainer run image.sif <command></code>
|-
| Inspect it || <code>apptainer inspect image.sif</code>
|}

== See also ==

* [[Python/Conda]]
* [[Apptainer]]
* [[Tutorials/Apptainer-Introduction]]
* [[Tutorials/Apptainer-DefinitionFiles]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Shared Storage]]

== External links ==

* [https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html micromamba documentation]
* [https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually Writing an environment.yml file]
* [https://apptainer.org/docs/user/main/definition_files.html Apptainer definition files]
* [https://depot.galaxyproject.org/singularity/ Prebuilt bioconda container images]

Tutorials/Apptainer-Conda

2026-08-17T15:09:47Z

Honfi001: Created page with "{{DISPLAYTITLE:Totorial: Conda in an Apptainer Container}} = Running a Conda Environment in a Container = Sometimes a piece of software is published only as a conda package, with no module, no Python-only equivalent and no ready-made image. Building it into a container is the cleanest way to run it on Anunna: the environment is created once, on a compute node, and afterwards it is a single file that behaves the same way every time you use it. Read Python/Conda firs..."

{{DISPLAYTITLE:Totorial: Conda in an Apptainer Container}}
= Running a Conda Environment in a Container =

Sometimes a piece of software is published only as a conda package, with no module, no Python-only equivalent and no ready-made image. Building it into a container is the cleanest way to run it on Anunna: the environment is created once, on a compute node, and afterwards it is a single file that behaves the same way every time you use it.

Read [[Python/Conda]] first if you have not already. This tutorial is the last resort described at the end of that page, not the recommended route.

'''Important:''' Before you begin, make sure the following are in place:

* You are running on a '''compute node''', not a login node. Request an interactive session first — see [[Interactive Jobs]].
* Your <code>.sif</code> image files should be stored on '''Lustre''', not in your home directory. SIF files can be large and will eat through your home quota fast.
* Set your Apptainer cache to Lustre as well. Add this to your session (or your <code>.bashrc</code>):

<syntaxhighlight lang="bash">
export APPTAINER_CACHEDIR=$myScratch/apptainer_cache
</syntaxhighlight>

== Getting Started ==

Load the required modules:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
</syntaxhighlight>

== Three things to know first ==

* '''A container should not be your first move.''' Always prefer software we already provide as [[Environment Modules|modules]], or a [[Python/Modules#Virtual environments|virtual environment]] built on one of our Python modules. Those are compiled for this machine and we are far better placed to help you when something goes wrong. [[Support|Ask us]] if what you need is missing.

* '''This tutorial uses micromamba, not conda.''' Micromamba is a single small program that installs conda packages, and it takes them from conda-forge and bioconda rather than from Anaconda's own repository. That keeps you clear of the licensing conditions described in [[Python/Conda#How they differ]].

* '''The image we build is read-only.''' Everything the environment needs is decided up front and baked in. Do not try to install software interactively inside a running container: the changes will not survive, and you lose the one real advantage of doing this, which is that the result is reproducible.

== Step 1: Describe the environment ==

Create a file called <code>environment.yml</code>. This is where you name the environment, list the packages you want, and say which repositories to take them from.

<syntaxhighlight lang="yaml">
name: base
channels:
- conda-forge
- bioconda
dependencies:
- python
- pip
- star
- bwa
- multiqc
</syntaxhighlight>

Two notes on this file. Keep the name as <code>base</code>, because that is the environment micromamba's image already has and installing into it saves a step. And for real work, pin your versions — <code>multiqc=1.29</code> rather than <code>multiqc</code> — otherwise rebuilding the image in six months will quietly give you different software.

== Step 2: Write the definition file ==

A '''definition file''' describes how the container is built, step by step. Create <code>image.def</code>:

<syntaxhighlight lang="bash">
Bootstrap: docker
From: mambaorg/micromamba:latest

%files
environment.yml /environment.yml

%post
micromamba install -n base --file /environment.yml && \
micromamba clean --all --yes
</syntaxhighlight>

Each part in turn:

=== <code>Bootstrap</code> and <code>From</code> ===

Start from a Docker image that already has micromamba installed, so we do not have to install it ourselves.

=== <code>%files</code> ===

Copy <code>environment.yml</code> from your directory on Anunna into the container, so the next step can read it.

=== <code>%post</code> ===

Run micromamba inside the container to install everything listed in the file, then delete its download cache. That last part matters: without <code>clean</code>, the cache is baked into the image and can easily double its size.

== Step 3: Build the image ==

<syntaxhighlight lang="bash">
apptainer build image.sif image.def
</syntaxhighlight>

This takes a few minutes and produces a single file, <code>image.sif</code>, containing the whole environment.

If the build fails complaining about directories from the host, clear Apptainer's bind list for the duration of the build:

<syntaxhighlight lang="bash">
APPTAINER_BIND=' ' apptainer build image.sif image.def
</syntaxhighlight>

== Testing the image ==

Check that the software is really there:

<syntaxhighlight lang="bash">
apptainer run image.sif multiqc --help
</syntaxhighlight>

You should see MultiQC's own help text, beginning with its version number and a usage line. If instead you get an error saying the command was not found, read the note below.

'''Use <code>run</code>, not <code>exec</code>.''' The micromamba image activates the environment as it starts up, and <code>apptainer run</code> goes through that startup step while <code>apptainer exec</code> skips it. With <code>exec</code> you will most likely be told the command cannot be found, even though it is installed.

== Using it in a job ==

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=multiqc
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

module load utilities Apptainer

apptainer run --bind /lustre $myNobackup/images/image.sif multiqc /path/to/results
</syntaxhighlight>

Apptainer does not automatically make every filesystem visible inside the container, so <code>--bind /lustre</code> ensures your data can be read.

== Where to keep your images ==

Image files are large. Keep them on Lustre, under <code>$myNobackup</code>, and not in your home directory.

If several people in your group use the same software, put one copy somewhere shared instead of each keeping their own — see [[Shared Storage]]. A container is a single file and is identical for everyone who runs it, which is precisely what makes sharing it worthwhile.

One habit worth forming: identify an image by what is inside it, using <code>apptainer inspect</code>, rather than by trusting its file name.

== Summary ==

{| class="wikitable"
! Step !! Command
|-
| Load Apptainer || <code>module reset && module load utilities Apptainer</code>
|-
| Describe the environment || Write <code>environment.yml</code>
|-
| Describe the container || Write <code>image.def</code>
|-
| Build the image || <code>apptainer build image.sif image.def</code>
|-
| Run something in it || <code>apptainer run image.sif <command></code>
|-
| Inspect it || <code>apptainer inspect image.sif</code>
|}

== See also ==

* [[Python/Conda]]
* [[Apptainer]]
* [[Tutorials/Apptainer-Introduction]]
* [[Tutorials/Apptainer-DefinitionFiles]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Shared Storage]]

== External links ==

* [https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html micromamba documentation]
* [https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually Writing an environment.yml file]
* [https://apptainer.org/docs/user/main/definition_files.html Apptainer definition files]
* [https://depot.galaxyproject.org/singularity/ Prebuilt bioconda container images]

Apptainer

2026-08-17T15:06:22Z

Honfi001:

Apptainer is a container platform. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible. You can build a container using Apptainer on your laptop, and then run it on many of the largest HPC clusters in the world, local university or company clusters, a single server, in the cloud, or on a workstation down the hall. Your container is a single file, and you don’t have to worry about how to install all the software you need on each different operating system.

Apptainer is a fork of singularity and, thus, also supports singularity commands and workflows.

==Apptainer on Anunna==

Since Apptainer is not dependent on specific compilers and toolchains, it is now place inside the '''utilities''' bucket. The '''Apptainer''' module the version compiled in Anunna.

One can load apptainer by running the following commands

<pre>module load utilities
module load Apptainer</pre>

==== Initial Setup ====
Apptainer will cache SIF container images generated from remote sources, and any OCI/docker layers used to create them. The cache is created at <code>$HOME/.apptainer/cache</code> by default. The location of the cache can be changed by setting the <code>APPTAINER_CACHEDIR</code> environment variable.

Since there is a limited amount of space available at <code>$HOME</code>, the Apptainer cache can quickly fill your quota. So it is recommended that you set the variable <code>APPTAINER_CACHEDIR</code> to a location in lustre by defined in your <code>$HOME/.bash_aliases</code> file. An example is provided at [[Aliases and local variables]]

Python/Modules

2026-08-17T14:59:54Z

Honfi001: /* See also */

{{DISPLAYTITLE:Python Modules}}
= Modules =
The Python Environment Modules are the only officially supported Python distributions in the HPC. These are compiled for each specific architecture of the HPC and hence are likely going to be more performant than the versions obtained via Mamba or UV.

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

So, for instance, loading Python 3.12.3
<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Jupyter kernels ==

Once your virtual environment has been setup, you can can access it in jupyter, either via the apps or the notebooks page by setting up a kernel. A kernel is nothing more than a json file with details for jupyter on where to find your Python environment. First things first, with your Python modules loaded activate your virtual environment from the example before.

<syntaxhighlight lang="bash">source $myNobackup/PythonEnv/my_env/bin/activate</syntaxhighlight>The install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">(my_env) user001@login200:~$ pip install -U ipykernel</syntaxhighlight>This is the only dependency that Python needs for creating kernels, you can then run <syntaxhighlight lang="bash">(my_env) user001@login200:~$ python -m ipykernel install --user --name=myenv --display-name="Python (myenv)"</syntaxhighlight>This command will create a folder labelled <code>myenv</code> containing a <code>kernel.json</code> file inside of <code>$HOME/.local/share/jupyter/kernels/</code> , which jupyter watches. If your goal is just to create a kernel for yourself, then you are done. The kernel should be visible to you both inside of the jupyter instances, either the ones accessed via the [https://apps.anunna.wur.nl anuna apps portal] or the [https://notebook.anunna.wur.nl notebooks page]. Thus, the kernel will not be visible or accessible to other users.

The --name=myenv flag is entirely optional, it is the directory in which the <code>kernel.json</code> file is generated. If not used, python will simply label it <code>python3</code> . The use of this flag is recommended, specially when working with multiple kernels to avoid overwriting previously created kernels.

=== Shared Kernels ===
Whether for a course or collaboration, sometimes it is convenient to share a kernel with other users. This is possible and the procedure is a variation of the previous one, with just some additional requirements

* The virtual environment you create needs to be in a shared location , e.g. /lutre/shared
* The virtual environment folder needs to be visiible and accessible to other users i.e. it must have read and execute permissions, 755 is recommended

Like before load the bucket and its corresponding Python version and create your project folder and go there<syntaxhighlight lang="bash">module load 2025
module load Python/3.13.1
mkdir /lustre/shared/MyProject
cd /lustre/shared/MyProject

</syntaxhighlight>Then create your virtual environemnt, activate it and install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">python -m venv ./project_env
source /lustre/shared/MyProject/project_env/bin/activate
pip install -U ipykernel</syntaxhighlight>

Here is where things differ, inside the activate kernel run the command<syntaxhighlight lang="bash">
python -m ipykernel install --sys-prefix --name=myproject --display-name="Python (project_env)"
</syntaxhighlight>Note that now the command uses the flag --sys-prefix . This flag sets the installation path of the kernel '''inside''' the virtual environment's directory, in this case, the kernel.json file will be generate in <code>/lustre/shared/MyProject/project_env/share/jupyter/kernels/myproject</code>.

While the kernel has been created, jupyter still will not be able to see it. In order to be able to see and access the kernel you would need to either copy it to your folder<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
cp -r $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>or link it<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
ln -s $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>Any users that run this last step should gain access to the kernel from jupyter. Though note, that only users that have write permissions to virtual environment will be able to install and remove modules from it.

<div style="background-color: #e7f3fe; border-left: 6px solid #2196F3; padding: 10px; margin-bottom: 15px;">
'''Note:''' When using custom kernels do not run <code>!pip</code>, just run <code>pip</code> without the <code>!</code>
</div>

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[Python]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

Python/Modules

2026-08-17T14:49:19Z

Honfi001:

{{DISPLAYTITLE:Python Modules}}
= Modules =
The Python Environment Modules are the only officially supported Python distributions in the HPC. These are compiled for each specific architecture of the HPC and hence are likely going to be more performant than the versions obtained via Mamba or UV.

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

So, for instance, loading Python 3.12.3
<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Jupyter kernels ==

Once your virtual environment has been setup, you can can access it in jupyter, either via the apps or the notebooks page by setting up a kernel. A kernel is nothing more than a json file with details for jupyter on where to find your Python environment. First things first, with your Python modules loaded activate your virtual environment from the example before.

<syntaxhighlight lang="bash">source $myNobackup/PythonEnv/my_env/bin/activate</syntaxhighlight>The install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">(my_env) user001@login200:~$ pip install -U ipykernel</syntaxhighlight>This is the only dependency that Python needs for creating kernels, you can then run <syntaxhighlight lang="bash">(my_env) user001@login200:~$ python -m ipykernel install --user --name=myenv --display-name="Python (myenv)"</syntaxhighlight>This command will create a folder labelled <code>myenv</code> containing a <code>kernel.json</code> file inside of <code>$HOME/.local/share/jupyter/kernels/</code> , which jupyter watches. If your goal is just to create a kernel for yourself, then you are done. The kernel should be visible to you both inside of the jupyter instances, either the ones accessed via the [https://apps.anunna.wur.nl anuna apps portal] or the [https://notebook.anunna.wur.nl notebooks page]. Thus, the kernel will not be visible or accessible to other users.

The --name=myenv flag is entirely optional, it is the directory in which the <code>kernel.json</code> file is generated. If not used, python will simply label it <code>python3</code> . The use of this flag is recommended, specially when working with multiple kernels to avoid overwriting previously created kernels.

=== Shared Kernels ===
Whether for a course or collaboration, sometimes it is convenient to share a kernel with other users. This is possible and the procedure is a variation of the previous one, with just some additional requirements

* The virtual environment you create needs to be in a shared location , e.g. /lutre/shared
* The virtual environment folder needs to be visiible and accessible to other users i.e. it must have read and execute permissions, 755 is recommended

Like before load the bucket and its corresponding Python version and create your project folder and go there<syntaxhighlight lang="bash">module load 2025
module load Python/3.13.1
mkdir /lustre/shared/MyProject
cd /lustre/shared/MyProject

</syntaxhighlight>Then create your virtual environemnt, activate it and install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">python -m venv ./project_env
source /lustre/shared/MyProject/project_env/bin/activate
pip install -U ipykernel</syntaxhighlight>

Here is where things differ, inside the activate kernel run the command<syntaxhighlight lang="bash">
python -m ipykernel install --sys-prefix --name=myproject --display-name="Python (project_env)"
</syntaxhighlight>Note that now the command uses the flag --sys-prefix . This flag sets the installation path of the kernel '''inside''' the virtual environment's directory, in this case, the kernel.json file will be generate in <code>/lustre/shared/MyProject/project_env/share/jupyter/kernels/myproject</code>.

While the kernel has been created, jupyter still will not be able to see it. In order to be able to see and access the kernel you would need to either copy it to your folder<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
cp -r $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>or link it<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
ln -s $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>Any users that run this last step should gain access to the kernel from jupyter. Though note, that only users that have write permissions to virtual environment will be able to install and remove modules from it.

<div style="background-color: #e7f3fe; border-left: 6px solid #2196F3; padding: 10px; margin-bottom: 15px;">
'''Note:''' When using custom kernels do not run <code>!pip</code>, just run <code>pip</code> without the <code>!</code>
</div>

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

Python

2026-08-17T14:47:34Z

Honfi001:

Python/Conda

2026-08-17T14:33:16Z

Honfi001:

{{DISPLAYTITLE:Anaconda}}

Conda is a tool for installing scientific software and keeping it in self-contained folders called environments. You will meet it most often in bioinformatics, data science and machine learning, where a great deal of software is published this way.

This page explains what Anaconda and Mamba are, why we do not recommend them on Anunna, what to use instead, and — if you have no alternative — how to use conda here with the least trouble.

== Introduction ==

=== What is Anaconda ===

Anaconda is a Python distribution made by a company, Anaconda Inc. It arrives as one large download containing Python, the <code>conda</code> command and several hundred scientific packages, already installed and ready to use. It is several gigabytes in size.

It was designed for a personal computer: you install it once and nearly everything you might need is already there. That convenience is why it became so popular.

=== What is Mamba ===

Mamba is a free, open-source tool that does the same job as Anaconda's <code>conda</code> command. It is not a bundle — it installs only the packages you ask for, and nothing else.

Mamba comes from the [https://conda-forge.org/ conda-forge] community. You normally get it by installing [https://github.com/conda-forge/miniforge Miniforge], which provides both <code>conda</code> and <code>mamba</code>. It gives you access to conda-forge and to [https://bioconda.github.io/ bioconda], the repository where most bioinformatics software is published.

=== What they have in common ===

Both build '''environments''' — a self-contained folder with its own Python and its own libraries. You can keep several side by side, each holding different versions of the same software, without them interfering with one another.

Both also install much more than Python packages. <code>pip</code> installs Python code and nothing else, whereas conda and mamba can install the things that code depends on as well: R, compilers, C libraries and command-line programs. That is the main reason people choose them.

Underneath, they are the same system. They use the same package format and the same repositories, so an environment built by one can be used by the other, and the commands are identical — <code>create</code>, <code>install</code>, <code>activate</code>. Moving from one to the other is not a migration; nothing needs converting.

=== How they differ ===

{| class="wikitable"
! !! Anaconda !! Mamba
|-
| '''What you get''' || Python and several hundred packages, whether you wanted them or not. Several gigabytes. || Only the packages you ask for.
|-
| '''Where packages come from''' || Anaconda's own repository. || conda-forge and bioconda.
|-
| '''Cost''' || A paid licence is required for large organisations. || Free and open source.
|}

Speed used to be a clear difference too, as mamba worked out dependencies far faster than conda did. Since conda version 23.10 the two share the same machinery for this, so the gap is much smaller than it once was.

On licensing, Anaconda changed its terms in 2024 and they may not permit research use at an institution the size of WUR — see [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's own statement on academia and research].

== Why they are not recommended in the cluster ==

Both tools do run on Anunna. Neither is a good fit for it. The reasons below apply to Anaconda and Mamba alike, except the last one.

* '''Neither is built for our hardware.''' Conda packages — from Anaconda and from conda-forge equally — are general-purpose builds that have to work on any machine in the world. The software we provide as modules is compiled for the exact processors in each generation of Anunna's nodes. If you care about speed and efficiency, use the modules in the buckets: see [[Python/Modules]].

* '''They bring their own copies of software the cluster already provides.''' Ask for one package and you may also receive compilers, maths libraries, CUDA and MPI as dependencies. These sit alongside Anunna's own versions and can be picked up instead of them, which produces results that are hard to explain.

* '''Their MPI is unlikely to use Anunna's fast network.''' A program that spreads work across several nodes needs MPI built against the cluster's high-speed interconnect. The MPI that comes out of a conda environment is a generic build, so a multi-node job may fall back to ordinary networking and run far slower, or fail to start at all. If your work runs on more than one node, use the MPI modules.

* '''They assume system libraries live where they do on a laptop.''' Those assumptions often do not hold here, and the result is a job that fails part-way through with an error that points nowhere useful.

* '''They fill your home directory.''' A single environment is tens of thousands of small files. Your home directory is small, it is backed up, and it sits on a filesystem measured at roughly ninety times slower than Lustre for large reads and writes. See [[Quotas]] and [[Storage Systems Overview]].

* '''Installing them slows the cluster down for everybody.''' Working out and unpacking an environment writes thousands of small files at once. On a login node, which is shared by every user, that is felt by everyone rather than only by you.

* '''They change your shell without asking.''' The <code>conda init</code> command writes a block into your <code>~/.bashrc</code>. That block is then read again by job scripts and by interactive sessions, and it can leak into containers, causing failures that look nothing like a conda problem.

* '''Anaconda additionally has a licensing problem.''' Its 2024 terms require a paid licence for organisations of 200 or more people, and the free exemption covers teaching rather than research. Other HPC centres have already stopped providing it.

== What to use instead ==

In order — try the first before the second, and so on.

=== Check whether it is already a module ===

Most widely used scientific software is already installed on Anunna and compiled for our hardware. Search for it by name before installing anything yourself:

<syntaxhighlight lang="bash">
module key blast
</syntaxhighlight>

<code>module key</code> searches every bucket at once, so you do not need to load one first. See [[Environment Modules]].

=== A virtual environment on a Python module ===

If you only need Python packages, a virtual environment built on one of our Python modules is the right tool. It is smaller, faster and simpler than a conda environment, and it uses the Python we compiled for this machine.

[[Python/Modules#Virtual environments]] covers this in full. The one thing to remember here is that the environment belongs on Lustre — under <code>$myNobackup</code> — and not in your home directory.

For a faster alternative to <code>pip</code> when installing into such an environment, the <code>uv</code> module is available in the 2025 bucket.

=== A ready-made container ===

Almost every package on bioconda is also published as a prebuilt container image, so you can run the software without building an environment at all. This is often the quickest route for bioinformatics software.

<syntaxhighlight lang="bash">
module load utilities Apptainer
apptainer pull https://depot.galaxyproject.org/singularity/multiqc:1.29--pyhdfd78af_0
apptainer run multiqc_1.29--pyhdfd78af_0.sif multiqc --help
</syntaxhighlight>

Browse [https://depot.galaxyproject.org/singularity/ the image list] for the tool and version you want. See [[Apptainer]], and set <code>APPTAINER_CACHEDIR</code> to a location on Lustre first, as that page describes.

=== When none of these work ===

Some software is published only as a conda package and has no module, no Python-only equivalent and no container. That is a real situation, and the rest of this page is for it.

== If you still need conda ==

Everything below is aimed at avoiding the problems described above. The two that matter most are keeping conda out of your <code>~/.bashrc</code> and never building an environment on a login node.

=== Use Miniforge, not Anaconda ===

[https://github.com/conda-forge/miniforge Miniforge] is a small installer from the conda-forge community. It gives you both <code>conda</code> and <code>mamba</code>, it uses conda-forge instead of Anaconda's repository, and it carries no licence conditions.

=== Install it on Lustre, not in your home directory ===

Start an interactive session first — see below — then:

<syntaxhighlight lang="bash">
cd $myNobackup
curl -L -O https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p $myNobackup/miniforge3
</syntaxhighlight>

<code>-p</code> chooses where it goes. <code>-b</code> installs without asking questions '''and without touching your shell configuration''', which is what you want.

<code>$myNobackup</code> is your Lustre location without backup, which is the right tier here: an environment can always be rebuilt from its list of packages, so it does not need backing up. If you have not set that variable yet, see [[Installing Personal Software#Aliases and local variables]].

=== Do not let it change your shell ===

Never run <code>conda init</code>. Load conda explicitly instead, whenever you need it:

<syntaxhighlight lang="bash">
source $myNoBkp/miniforge3/etc/profile.d/conda.sh
conda activate myenv
</syntaxhighlight>

If you have run <code>conda init</code> in the past, open <code>~/.bashrc</code> and delete everything between the lines <code># >>> conda initialize >>></code> and <code># <<< conda initialize <<<</code>, including those two lines.

=== Keep the package cache off your home directory ===

Conda keeps a cache of downloaded packages and a record of your environments. Point both at Lustre by adding these lines to your <code>~/.bash_aliases</code>:

<syntaxhighlight lang="bash">
export CONDA_PKGS_DIRS=$myNoBkp/conda/pkgs
export CONDA_ENVS_DIRS=$myNoBkp/conda/envs
</syntaxhighlight>

=== Never build an environment on a login node ===

Creating an environment is exactly the kind of work that disturbs the shared filesystem for other users. Ask for an interactive session on a compute node and do it there:

<syntaxhighlight lang="bash">
sinteractive -c 4 --mem 16G --time 120
</syntaxhighlight>

Then create the environment. Listing the packages in a file makes it repeatable:

<syntaxhighlight lang="bash">
source $myNoBkp/miniforge3/etc/profile.d/conda.sh
mamba create -n myenv python=3.12 numpy pandas
</syntaxhighlight>

See [[Interactive Jobs]] for the options <code>sinteractive</code> accepts.

=== Using an environment in a job script ===

Load conda and activate the environment inside the script itself. Note the first line: use plain <code>#!/bin/bash</code>, not <code>#!/bin/bash -i</code>, so the script does not read your shell configuration.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

source $myNobackup/miniforge3/etc/profile.d/conda.sh
conda activate myenv

python myscript.py
</syntaxhighlight>

See [[Batch Jobs]].

=== Cleaning up ===

Environments are large and easy to forget. Check what you have from time to time, remove what you no longer use, and empty the download cache:

<syntaxhighlight lang="bash">
conda env list
conda env remove -n myenv
conda clean --all
</syntaxhighlight>

== See also ==

* [[Python]]
* [[Python/Modules]]
* [[Environment Modules]]
* [[Apptainer]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Installing Personal Software]]
* [[Quotas]]

== External links ==

* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]
* [https://bioconda.github.io/ bioconda]
* [https://depot.galaxyproject.org/singularity/ Prebuilt bioconda container images]
* [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's terms of service for academia and research]

Python/Conda

2026-08-17T14:32:37Z

Honfi001:

Python/Conda

2026-08-17T14:25:48Z

Honfi001:

Python

2026-08-17T14:06:11Z

Honfi001:

Python/Conda

2026-08-17T14:00:05Z

Honfi001:

{{DISPLAYTITLE:Anaconda}}
Conda is a tool for installing scientific software and keeping it in self-contained folders called environments. You will meet it most often in bioinformatics, data science and machine learning, where a great deal of software is published this way.

This page explains what Anaconda and Mamba are, why we do not recommend them on Anunna, what to use instead, and — if you have no alternative — how to use conda here with the least trouble.

== Introduction ==

=== What is Anaconda ===

Anaconda is a Python distribution made by a company, Anaconda Inc. It arrives as one large download containing Python, the <code>conda</code> command and several hundred scientific packages, already installed and ready to use. It is several gigabytes in size.

It was designed for a personal computer: you install it once and nearly everything you might need is already there. That convenience is why it became so popular.

=== What is Mamba ===

Mamba is a free, open-source tool that does the same job as Anaconda's <code>conda</code> command. It is not a bundle — it installs only the packages you ask for, and nothing else.

Mamba comes from the [https://conda-forge.org/ conda-forge] community. You normally get it by installing [https://github.com/conda-forge/miniforge Miniforge], which provides both <code>conda</code> and <code>mamba</code>. It gives you access to conda-forge and to [https://bioconda.github.io/ bioconda], the repository where most bioinformatics software is published.

=== What they have in common ===

Both build '''environments''' — a self-contained folder with its own Python and its own libraries. You can keep several side by side, each holding different versions of the same software, without them interfering with one another.

Both also install much more than Python packages. <code>pip</code> installs Python code and nothing else, whereas conda and mamba can install the things that code depends on as well: R, compilers, C libraries and command-line programs. That is the main reason people choose them.

Underneath, they are the same system. They use the same package format and the same repositories, so an environment built by one can be used by the other, and the commands are identical — <code>create</code>, <code>install</code>, <code>activate</code>. Moving from one to the other is not a migration; nothing needs converting.

=== How they differ ===

{| class="wikitable"
! !! Anaconda !! Mamba
|-
| '''What you get''' || Python and several hundred packages, whether you wanted them or not. Several gigabytes. || Only the packages you ask for.
|-
| '''Where packages come from''' || Anaconda's own repository. || conda-forge and bioconda.
|-
| '''Cost''' || A paid licence is required for large organisations. || Free and open source.
|}

Speed used to be a clear difference too, as mamba worked out dependencies far faster than conda did. Since conda version 23.10 the two share the same machinery for this, so the gap is much smaller than it once was.

On licensing, Anaconda changed its terms in 2024 and they may not permit research use at an institution the size of WUR — see [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's own statement on academia and research].

== Why they are not recommended in the cluster ==

Both tools do run on Anunna. Neither is a good fit for it. The reasons below apply to Anaconda and Mamba alike, except the last one.

* '''Neither is built for our hardware.''' Conda packages — from Anaconda and from conda-forge equally — are general-purpose builds that have to work on any machine in the world. The software we provide as modules is compiled for the exact processors in each generation of Anunna's nodes. If you care about speed and efficiency, use the modules in the buckets: see [[Python/Modules]].

* '''They bring their own copies of software the cluster already provides.''' Ask for one package and you may also receive compilers, maths libraries, CUDA and MPI as dependencies. These sit alongside Anunna's own versions and can be picked up instead of them, which produces results that are hard to explain.

* '''Their MPI is unlikely to use Anunna's fast network.''' A program that spreads work across several nodes needs MPI built against the cluster's high-speed interconnect. The MPI that comes out of a conda environment is a generic build, so a multi-node job may fall back to ordinary networking and run far slower, or fail to start at all. If your work runs on more than one node, use the MPI modules.

* '''They assume system libraries live where they do on a laptop.''' Those assumptions often do not hold here, and the result is a job that fails part-way through with an error that points nowhere useful.

* '''They fill your home directory.''' A single environment is tens of thousands of small files. Your home directory is small, it is backed up, and it sits on a filesystem measured at roughly ninety times slower than Lustre for large reads and writes. See [[Quotas]] and [[Storage Systems Overview]].

* '''Installing them slows the cluster down for everybody.''' Working out and unpacking an environment writes thousands of small files at once. On a login node, which is shared by every user, that is felt by everyone rather than only by you.

* '''They change your shell without asking.''' The <code>conda init</code> command writes a block into your <code>~/.bashrc</code>. That block is then read again by job scripts and by interactive sessions, and it can leak into containers, causing failures that look nothing like a conda problem.

* '''Anaconda additionally has a licensing problem.''' Its 2024 terms require a paid licence for organisations of 200 or more people, and the free exemption covers teaching rather than research. Other HPC centres have already stopped providing it.

== What to use instead ==

In order — try the first before the second, and so on.

=== Check whether it is already a module ===

Most widely used scientific software is already installed on Anunna and compiled for our hardware. Search for it by name before installing anything yourself:

<syntaxhighlight lang="bash">
module key blast
</syntaxhighlight>

<code>module key</code> searches every bucket at once, so you do not need to load one first. See [[Environment Modules]].

=== A virtual environment on a Python module ===

If you only need Python packages, a virtual environment built on one of our Python modules is the right tool. It is smaller, faster and simpler than a conda environment, and it uses the Python we compiled for this machine.

[[Python/Modules#Virtual environments]] covers this in full. The one thing to remember here is that the environment belongs on Lustre — under <code>$myNobackup</code> — and not in your home directory.

For a faster alternative to <code>pip</code> when installing into such an environment, the <code>uv</code> module is available in the 2025 bucket.

=== A ready-made container ===

Almost every package on bioconda is also published as a prebuilt container image, so you can run the software without building an environment at all. This is often the quickest route for bioinformatics software.

<syntaxhighlight lang="bash">
module load utilities Apptainer
apptainer pull https://depot.galaxyproject.org/singularity/multiqc:1.29--pyhdfd78af_0
apptainer run multiqc_1.29--pyhdfd78af_0.sif multiqc --help
</syntaxhighlight>

Browse [https://depot.galaxyproject.org/singularity/ the image list] for the tool and version you want. See [[Apptainer]], and set <code>APPTAINER_CACHEDIR</code> to a location on Lustre first, as that page describes.

=== When none of these work ===

Some software is published only as a conda package and has no module, no Python-only equivalent and no container. That is a real situation, and the rest of this page is for it.

== If you still need conda ==

Everything below is aimed at avoiding the problems described above. The two that matter most are keeping conda out of your <code>~/.bashrc</code> and never building an environment on a login node.

=== Use Miniforge, not Anaconda ===

[https://github.com/conda-forge/miniforge Miniforge] is a small installer from the conda-forge community. It gives you both <code>conda</code> and <code>mamba</code>, it uses conda-forge instead of Anaconda's repository, and it carries no licence conditions.

=== Install it on Lustre, not in your home directory ===

Start an interactive session first — see below — then:

<syntaxhighlight lang="bash">
cd $myNobackup
curl -L -O https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p $myNobackup/miniforge3
</syntaxhighlight>

<code>-p</code> chooses where it goes. <code>-b</code> installs without asking questions '''and without touching your shell configuration''', which is what you want.

<code>$myNobackup</code> is your Lustre location without backup, which is the right tier here: an environment can always be rebuilt from its list of packages, so it does not need backing up. If you have not set that variable yet, see [[Installing Personal Software#Aliases and local variables]].

=== Do not let it change your shell ===

Never run <code>conda init</code>. Load conda explicitly instead, whenever you need it:

<syntaxhighlight lang="bash">
source $myNoBkp/miniforge3/etc/profile.d/conda.sh
conda activate myenv
</syntaxhighlight>

If you have run <code>conda init</code> in the past, open <code>~/.bashrc</code> and delete everything between the lines <code># >>> conda initialize >>></code> and <code># <<< conda initialize <<<</code>, including those two lines.

=== Keep the package cache off your home directory ===

Conda keeps a cache of downloaded packages and a record of your environments. Point both at Lustre by adding these lines to your <code>~/.bash_aliases</code>:

<syntaxhighlight lang="bash">
export CONDA_PKGS_DIRS=$myNoBkp/conda/pkgs
export CONDA_ENVS_DIRS=$myNoBkp/conda/envs
</syntaxhighlight>

=== Never build an environment on a login node ===

Creating an environment is exactly the kind of work that disturbs the shared filesystem for other users. Ask for an interactive session on a compute node and do it there:

<syntaxhighlight lang="bash">
sinteractive -c 4 --mem 16G --time 120
</syntaxhighlight>

Then create the environment. Listing the packages in a file makes it repeatable:

<syntaxhighlight lang="bash">
source $myNoBkp/miniforge3/etc/profile.d/conda.sh
mamba create -n myenv python=3.12 numpy pandas
</syntaxhighlight>

See [[Interactive Jobs]] for the options <code>sinteractive</code> accepts.

=== Using an environment in a job script ===

Load conda and activate the environment inside the script itself. Note the first line: use plain <code>#!/bin/bash</code>, not <code>#!/bin/bash -i</code>, so the script does not read your shell configuration.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

source $myNobackup/miniforge3/etc/profile.d/conda.sh
conda activate myenv

python myscript.py
</syntaxhighlight>

See [[Batch Jobs]].

=== Cleaning up ===

Environments are large and easy to forget. Check what you have from time to time, remove what you no longer use, and empty the download cache:

<syntaxhighlight lang="bash">
conda env list
conda env remove -n myenv
conda clean --all
</syntaxhighlight>

== See also ==

* [[Python]]
* [[Python/Modules]]
* [[Environment Modules]]
* [[Apptainer]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Installing Personal Software]]
* [[Quotas]]

== External links ==

* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]
* [https://bioconda.github.io/ bioconda]
* [https://depot.galaxyproject.org/singularity/ Prebuilt bioconda container images]
* [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's terms of service for academia and research]

Python/Conda

2026-08-17T13:55:35Z

Honfi001:

Conda is a tool for installing scientific software and keeping it in self-contained folders called environments. You will meet it most often in bioinformatics, data science and machine learning, where a great deal of software is published this way.

This page explains what Anaconda and Mamba are, why we do not recommend them on Anunna, what to use instead, and — if you have no alternative — how to use conda here with the least trouble.

== Introduction ==

=== What is Anaconda ===

Anaconda is a Python distribution made by a company, Anaconda Inc. It arrives as one large download containing Python, the <code>conda</code> command and several hundred scientific packages, already installed and ready to use. It is several gigabytes in size.

It was designed for a personal computer: you install it once and nearly everything you might need is already there. That convenience is why it became so popular.

=== What is Mamba ===

Mamba is a free, open-source tool that does the same job as Anaconda's <code>conda</code> command. It is not a bundle — it installs only the packages you ask for, and nothing else.

Mamba comes from the [https://conda-forge.org/ conda-forge] community. You normally get it by installing [https://github.com/conda-forge/miniforge Miniforge], which provides both <code>conda</code> and <code>mamba</code>. It gives you access to conda-forge and to [https://bioconda.github.io/ bioconda], the repository where most bioinformatics software is published.

=== What they have in common ===

Both build '''environments''' — a self-contained folder with its own Python and its own libraries. You can keep several side by side, each holding different versions of the same software, without them interfering with one another.

Both also install much more than Python packages. <code>pip</code> installs Python code and nothing else, whereas conda and mamba can install the things that code depends on as well: R, compilers, C libraries and command-line programs. That is the main reason people choose them.

Underneath, they are the same system. They use the same package format and the same repositories, so an environment built by one can be used by the other, and the commands are identical — <code>create</code>, <code>install</code>, <code>activate</code>. Moving from one to the other is not a migration; nothing needs converting.

=== How they differ ===

{| class="wikitable"
! !! Anaconda !! Mamba
|-
| '''What you get''' || Python and several hundred packages, whether you wanted them or not. Several gigabytes. || Only the packages you ask for.
|-
| '''Where packages come from''' || Anaconda's own repository. || conda-forge and bioconda.
|-
| '''Cost''' || A paid licence is required for large organisations. || Free and open source.
|}

Speed used to be a clear difference too, as mamba worked out dependencies far faster than conda did. Since conda version 23.10 the two share the same machinery for this, so the gap is much smaller than it once was.

On licensing, Anaconda changed its terms in 2024 and they may not permit research use at an institution the size of WUR — see [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's own statement on academia and research].

== Why they are not recommended in the cluster ==

Both tools do run on Anunna. Neither is a good fit for it. The reasons below apply to Anaconda and Mamba alike, except the last one.

* '''Neither is built for our hardware.''' Conda packages — from Anaconda and from conda-forge equally — are general-purpose builds that have to work on any machine in the world. The software we provide as modules is compiled for the exact processors in each generation of Anunna's nodes. If you care about speed and efficiency, use the modules in the buckets: see [[Python/Modules]].

* '''They bring their own copies of software the cluster already provides.''' Ask for one package and you may also receive compilers, maths libraries, CUDA and MPI as dependencies. These sit alongside Anunna's own versions and can be picked up instead of them, which produces results that are hard to explain.

* '''Their MPI is unlikely to use Anunna's fast network.''' A program that spreads work across several nodes needs MPI built against the cluster's high-speed interconnect. The MPI that comes out of a conda environment is a generic build, so a multi-node job may fall back to ordinary networking and run far slower, or fail to start at all. If your work runs on more than one node, use the MPI modules.

* '''They assume system libraries live where they do on a laptop.''' Those assumptions often do not hold here, and the result is a job that fails part-way through with an error that points nowhere useful.

* '''They fill your home directory.''' A single environment is tens of thousands of small files. Your home directory is small, it is backed up, and it sits on a filesystem measured at roughly ninety times slower than Lustre for large reads and writes. See [[Quotas]] and [[Storage Systems Overview]].

* '''Installing them slows the cluster down for everybody.''' Working out and unpacking an environment writes thousands of small files at once. On a login node, which is shared by every user, that is felt by everyone rather than only by you.

* '''They change your shell without asking.''' The <code>conda init</code> command writes a block into your <code>~/.bashrc</code>. That block is then read again by job scripts and by interactive sessions, and it can leak into containers, causing failures that look nothing like a conda problem.

* '''Anaconda additionally has a licensing problem.''' Its 2024 terms require a paid licence for organisations of 200 or more people, and the free exemption covers teaching rather than research. Other HPC centres have already stopped providing it.

== What to use instead ==

In order — try the first before the second, and so on.

=== Check whether it is already a module ===

Most widely used scientific software is already installed on Anunna and compiled for our hardware. Search for it by name before installing anything yourself:

<syntaxhighlight lang="bash">
module key blast
</syntaxhighlight>

<code>module key</code> searches every bucket at once, so you do not need to load one first. See [[Environment Modules]].

=== A virtual environment on a Python module ===

If you only need Python packages, a virtual environment built on one of our Python modules is the right tool. It is smaller, faster and simpler than a conda environment, and it uses the Python we compiled for this machine.

[[Python/Modules#Virtual environments]] covers this in full. The one thing to remember here is that the environment belongs on Lustre — under <code>$myNobackup</code> — and not in your home directory.

For a faster alternative to <code>pip</code> when installing into such an environment, the <code>uv</code> module is available in the 2025 bucket.

=== A ready-made container ===

Almost every package on bioconda is also published as a prebuilt container image, so you can run the software without building an environment at all. This is often the quickest route for bioinformatics software.

<syntaxhighlight lang="bash">
module load utilities Apptainer
apptainer pull https://depot.galaxyproject.org/singularity/multiqc:1.29--pyhdfd78af_0
apptainer run multiqc_1.29--pyhdfd78af_0.sif multiqc --help
</syntaxhighlight>

Browse [https://depot.galaxyproject.org/singularity/ the image list] for the tool and version you want. See [[Apptainer]], and set <code>APPTAINER_CACHEDIR</code> to a location on Lustre first, as that page describes.

=== When none of these work ===

Some software is published only as a conda package and has no module, no Python-only equivalent and no container. That is a real situation, and the rest of this page is for it.

== If you still need conda ==

Everything below is aimed at avoiding the problems described above. The two that matter most are keeping conda out of your <code>~/.bashrc</code> and never building an environment on a login node.

=== Use Miniforge, not Anaconda ===

[https://github.com/conda-forge/miniforge Miniforge] is a small installer from the conda-forge community. It gives you both <code>conda</code> and <code>mamba</code>, it uses conda-forge instead of Anaconda's repository, and it carries no licence conditions.

=== Install it on Lustre, not in your home directory ===

Start an interactive session first — see below — then:

<syntaxhighlight lang="bash">
cd $myNobackup
curl -L -O https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p $myNobackup/miniforge3
</syntaxhighlight>

<code>-p</code> chooses where it goes. <code>-b</code> installs without asking questions '''and without touching your shell configuration''', which is what you want.

<code>$myNobackup</code> is your Lustre location without backup, which is the right tier here: an environment can always be rebuilt from its list of packages, so it does not need backing up. If you have not set that variable yet, see [[Installing Personal Software#Aliases and local variables]].

=== Do not let it change your shell ===

Never run <code>conda init</code>. Load conda explicitly instead, whenever you need it:

<syntaxhighlight lang="bash">
source $myNoBkp/miniforge3/etc/profile.d/conda.sh
conda activate myenv
</syntaxhighlight>

If you have run <code>conda init</code> in the past, open <code>~/.bashrc</code> and delete everything between the lines <code># >>> conda initialize >>></code> and <code># <<< conda initialize <<<</code>, including those two lines.

=== Keep the package cache off your home directory ===

Conda keeps a cache of downloaded packages and a record of your environments. Point both at Lustre by adding these lines to your <code>~/.bash_aliases</code>:

<syntaxhighlight lang="bash">
export CONDA_PKGS_DIRS=$myNoBkp/conda/pkgs
export CONDA_ENVS_DIRS=$myNoBkp/conda/envs
</syntaxhighlight>

=== Never build an environment on a login node ===

Creating an environment is exactly the kind of work that disturbs the shared filesystem for other users. Ask for an interactive session on a compute node and do it there:

<syntaxhighlight lang="bash">
sinteractive -c 4 --mem 16G --time 120
</syntaxhighlight>

Then create the environment. Listing the packages in a file makes it repeatable:

<syntaxhighlight lang="bash">
source $myNoBkp/miniforge3/etc/profile.d/conda.sh
mamba create -n myenv python=3.12 numpy pandas
</syntaxhighlight>

See [[Interactive Jobs]] for the options <code>sinteractive</code> accepts.

=== Using an environment in a job script ===

Load conda and activate the environment inside the script itself. Note the first line: use plain <code>#!/bin/bash</code>, not <code>#!/bin/bash -i</code>, so the script does not read your shell configuration.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

source $myNobackup/miniforge3/etc/profile.d/conda.sh
conda activate myenv

python myscript.py
</syntaxhighlight>

See [[Batch Jobs]].

=== Cleaning up ===

Environments are large and easy to forget. Check what you have from time to time, remove what you no longer use, and empty the download cache:

<syntaxhighlight lang="bash">
conda env list
conda env remove -n myenv
conda clean --all
</syntaxhighlight>

== See also ==

* [[Python]]
* [[Python/Modules]]
* [[Environment Modules]]
* [[Apptainer]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Installing Personal Software]]
* [[Quotas]]

== External links ==

* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]
* [https://bioconda.github.io/ bioconda]
* [https://depot.galaxyproject.org/singularity/ Prebuilt bioconda container images]
* [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's terms of service for academia and research]

Python/Conda

2026-08-17T13:42:12Z

Honfi001: Created page with "Conda is a tool for installing scientific software and keeping it in self-contained folders called environments. You will meet it most often in bioinformatics, data science and machine learning, where a great deal of software is published this way. This page explains what Anaconda and Mamba are, why we do not recommend them on Anunna, what to use instead, and — if you have no alternative — how to use conda here with the least trouble. == Introduction == === What i..."

Conda is a tool for installing scientific software and keeping it in self-contained folders called environments. You will meet it most often in bioinformatics, data science and machine learning, where a great deal of software is published this way.

This page explains what Anaconda and Mamba are, why we do not recommend them on Anunna, what to use instead, and — if you have no alternative — how to use conda here with the least trouble.

== Introduction ==

=== What is Anaconda ===

Anaconda is a Python distribution made by a company, Anaconda Inc. It arrives as one large download containing Python, the <code>conda</code> command and several hundred scientific packages, already installed and ready to use. It is several gigabytes in size.

It was designed for a personal computer: you install it once and nearly everything you might need is already there. That convenience is why it became so popular.

=== What is Mamba ===

Mamba is a free, open-source tool that does the same job as Anaconda's <code>conda</code> command. It is not a bundle — it installs only the packages you ask for, and nothing else.

Mamba comes from the [https://conda-forge.org/ conda-forge] community. You normally get it by installing [https://github.com/conda-forge/miniforge Miniforge], which provides both <code>conda</code> and <code>mamba</code>. It gives you access to conda-forge and to [https://bioconda.github.io/ bioconda], the repository where most bioinformatics software is published.

=== What they have in common ===

Both build '''environments''' — a self-contained folder with its own Python and its own libraries. You can keep several side by side, each holding different versions of the same software, without them interfering with one another.

Both also install much more than Python packages. <code>pip</code> installs Python code and nothing else, whereas conda and mamba can install the things that code depends on as well: R, compilers, C libraries and command-line programs. That is the main reason people choose them.

Underneath, they are the same system. They use the same package format and the same repositories, so an environment built by one can be used by the other, and the commands are identical — <code>create</code>, <code>install</code>, <code>activate</code>. Moving from one to the other is not a migration; nothing needs converting.

=== How they differ ===

{| class="wikitable"
! !! Anaconda !! Mamba
|-
| '''What you get''' || Python and several hundred packages, whether you wanted them or not. Several gigabytes. || Only the packages you ask for.
|-
| '''Where packages come from''' || Anaconda's own repository. || conda-forge and bioconda.
|-
| '''Cost''' || A paid licence is required for large organisations. || Free and open source.
|}

Speed used to be a clear difference too, as mamba worked out dependencies far faster than conda did. Since conda version 23.10 the two share the same machinery for this, so the gap is much smaller than it once was.

On licensing, Anaconda changed its terms in 2024 and they may not permit research use at an institution the size of WUR — see [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's own statement on academia and research].

== Why they are not recommended in the cluster ==

Both tools do run on Anunna. Neither is a good fit for it. The reasons below apply to Anaconda and Mamba alike, except the last one.

* '''Neither is built for our hardware.''' Conda packages — from Anaconda and from conda-forge equally — are general-purpose builds that have to work on any machine in the world. The software we provide as modules is compiled for the exact processors in each generation of Anunna's nodes. If you care about speed and efficiency, use the modules in the buckets: see [[Python/Modules]].

* '''They bring their own copies of software the cluster already provides.''' Ask for one package and you may also receive compilers, maths libraries, CUDA and MPI as dependencies. These sit alongside Anunna's own versions and can be picked up instead of them, which produces results that are hard to explain.

* '''Their MPI is unlikely to use Anunna's fast network.''' A program that spreads work across several nodes needs MPI built against the cluster's high-speed interconnect. The MPI that comes out of a conda environment is a generic build, so a multi-node job may fall back to ordinary networking and run far slower, or fail to start at all. If your work runs on more than one node, use the MPI modules.

* '''They assume system libraries live where they do on a laptop.''' Those assumptions often do not hold here, and the result is a job that fails part-way through with an error that points nowhere useful.

* '''They fill your home directory.''' A single environment is tens of thousands of small files. Your home directory is small, it is backed up, and it sits on a filesystem measured at roughly ninety times slower than Lustre for large reads and writes. See [[Quotas]] and [[Storage Systems Overview]].

* '''Installing them slows the cluster down for everybody.''' Working out and unpacking an environment writes thousands of small files at once. On a login node, which is shared by every user, that is felt by everyone rather than only by you.

* '''They change your shell without asking.''' The <code>conda init</code> command writes a block into your <code>~/.bashrc</code>. That block is then read again by job scripts and by interactive sessions, and it can leak into containers, causing failures that look nothing like a conda problem.

* '''Anaconda additionally has a licensing problem.''' Its 2024 terms require a paid licence for organisations of 200 or more people, and the free exemption covers teaching rather than research. Other HPC centres have already stopped providing it.

== What to use instead ==

In order — try the first before the second, and so on.

=== Check whether it is already a module ===

Most widely used scientific software is already installed on Anunna and compiled for our hardware. Search for it by name before installing anything yourself:

<syntaxhighlight lang="bash">
module key blast
</syntaxhighlight>

<code>module key</code> searches every bucket at once, so you do not need to load one first. See [[Environment Modules]].

=== A virtual environment on a Python module ===

If you only need Python packages, a virtual environment built on one of our Python modules is the right tool. It is smaller, faster and simpler than a conda environment, and it uses the Python we compiled for this machine.

[[Python/Modules#Virtual environments]] covers this in full. The one thing to remember here is that the environment belongs on Lustre — under <code>$myNobackup</code> — and not in your home directory.

For a faster alternative to <code>pip</code> when installing into such an environment, the <code>uv</code> module is available in the 2025 bucket.

=== A ready-made container ===

Almost every package on bioconda is also published as a prebuilt container image, so you can run the software without building an environment at all. This is often the quickest route for bioinformatics software.

<syntaxhighlight lang="bash">
module load utilities Apptainer
apptainer pull https://depot.galaxyproject.org/singularity/multiqc:1.29--pyhdfd78af_0
apptainer run multiqc_1.29--pyhdfd78af_0.sif multiqc --help
</syntaxhighlight>

Browse [https://depot.galaxyproject.org/singularity/ the image list] for the tool and version you want. See [[Apptainer]], and set <code>APPTAINER_CACHEDIR</code> to a location on Lustre first, as that page describes.

=== When none of these work ===

Some software is published only as a conda package and has no module, no Python-only equivalent and no container. That is a real situation, and the rest of this page is for it.

== If you still need conda ==

Everything below is aimed at avoiding the problems described above. The two that matter most are keeping conda out of your <code>~/.bashrc</code> and never building an environment on a login node.

=== Use Miniforge, not Anaconda ===

[https://github.com/conda-forge/miniforge Miniforge] is a small installer from the conda-forge community. It gives you both <code>conda</code> and <code>mamba</code>, it uses conda-forge instead of Anaconda's repository, and it carries no licence conditions.

=== Install it on Lustre, not in your home directory ===

Start an interactive session first — see below — then:

<syntaxhighlight lang="bash">
cd $myNobackup
curl -L -O https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p $myNobackup/miniforge3
</syntaxhighlight>

<code>-p</code> chooses where it goes. <code>-b</code> installs without asking questions '''and without touching your shell configuration''', which is what you want.

<code>$myNobackup</code> is your Lustre location without backup, which is the right tier here: an environment can always be rebuilt from its list of packages, so it does not need backing up. If you have not set that variable yet, see [[Installing Personal Software#Aliases and local variables]].

=== Do not let it change your shell ===

Never run <code>conda init</code>. Load conda explicitly instead, whenever you need it:

<syntaxhighlight lang="bash">
source $myNobackup/miniforge3/etc/profile.d/conda.sh
conda activate myenv
</syntaxhighlight>

If you have run <code>conda init</code> in the past, open <code>~/.bashrc</code> and delete everything between the lines <code># >>> conda initialize >>></code> and <code># <<< conda initialize <<<</code>, including those two lines.

=== Keep the package cache off your home directory ===

Conda keeps a cache of downloaded packages and a record of your environments. Point both at Lustre by adding these lines to your <code>~/.bash_aliases</code>:

<syntaxhighlight lang="bash">
export CONDA_PKGS_DIRS=$myNobackup/conda/pkgs
export CONDA_ENVS_DIRS=$myNobackup/conda/envs
</syntaxhighlight>

=== Never build an environment on a login node ===

Creating an environment is exactly the kind of work that disturbs the shared filesystem for other users. Ask for an interactive session on a compute node and do it there:

<syntaxhighlight lang="bash">
sinteractive -c 4 --mem 16G --time 120
</syntaxhighlight>

Then create the environment. Listing the packages in a file makes it repeatable:

<syntaxhighlight lang="bash">
source $myNobackup/miniforge3/etc/profile.d/conda.sh
mamba create -n myenv python=3.12 numpy pandas
</syntaxhighlight>

See [[Interactive Jobs]] for the options <code>sinteractive</code> accepts.

=== Using an environment in a job script ===

Load conda and activate the environment inside the script itself. Note the first line: use plain <code>#!/bin/bash</code>, not <code>#!/bin/bash -i</code>, so the script does not read your shell configuration.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

source $myNobackup/miniforge3/etc/profile.d/conda.sh
conda activate myenv

python myscript.py
</syntaxhighlight>

See [[Batch Jobs]].

=== Cleaning up ===

Environments are large and easy to forget. Check what you have from time to time, remove what you no longer use, and empty the download cache:

<syntaxhighlight lang="bash">
conda env list
conda env remove -n myenv
conda clean --all
</syntaxhighlight>

== See also ==

* [[Python]]
* [[Python/Modules]]
* [[Environment Modules]]
* [[Apptainer]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Installing Personal Software]]
* [[Quotas]]

== External links ==

* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]
* [https://bioconda.github.io/ bioconda]
* [https://depot.galaxyproject.org/singularity/ Prebuilt bioconda container images]
* [https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research Anaconda's terms of service for academia and research]

Python

2026-08-12T15:10:19Z

Honfi001:

Python

2026-08-12T15:09:04Z

Honfi001:

Python/Modules

2026-08-12T15:07:10Z

Honfi001:

= Modules =
The Python Environment Modules are the only officially supported Python distributions in the HPC. These are compiled for each specific architecture of the HPC and hence are likely going to be more performant than the versions obtained via Mamba or UV.

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

So, for instance, loading Python 3.12.3
<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Jupyter kernels ==

Once your virtual environment has been setup, you can can access it in jupyter, either via the apps or the notebooks page by setting up a kernel. A kernel is nothing more than a json file with details for jupyter on where to find your Python environment. First things first, with your Python modules loaded activate your virtual environment from the example before.

<syntaxhighlight lang="bash">source $myNobackup/PythonEnv/my_env/bin/activate</syntaxhighlight>The install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">(my_env) user001@login200:~$ pip install -U ipykernel</syntaxhighlight>This is the only dependency that Python needs for creating kernels, you can then run <syntaxhighlight lang="bash">(my_env) user001@login200:~$ python -m ipykernel install --user --name=myenv --display-name="Python (myenv)"</syntaxhighlight>This command will create a folder labelled <code>myenv</code> containing a <code>kernel.json</code> file inside of <code>$HOME/.local/share/jupyter/kernels/</code> , which jupyter watches. If your goal is just to create a kernel for yourself, then you are done. The kernel should be visible to you both inside of the jupyter instances, either the ones accessed via the [https://apps.anunna.wur.nl anuna apps portal] or the [https://notebook.anunna.wur.nl notebooks page]. Thus, the kernel will not be visible or accessible to other users.

The --name=myenv flag is entirely optional, it is the directory in which the <code>kernel.json</code> file is generated. If not used, python will simply label it <code>python3</code> . The use of this flag is recommended, specially when working with multiple kernels to avoid overwriting previously created kernels.

=== Shared Kernels ===
Whether for a course or collaboration, sometimes it is convenient to share a kernel with other users. This is possible and the procedure is a variation of the previous one, with just some additional requirements

* The virtual environment you create needs to be in a shared location , e.g. /lutre/shared
* The virtual environment folder needs to be visiible and accessible to other users i.e. it must have read and execute permissions, 755 is recommended

Like before load the bucket and its corresponding Python version and create your project folder and go there<syntaxhighlight lang="bash">module load 2025
module load Python/3.13.1
mkdir /lustre/shared/MyProject
cd /lustre/shared/MyProject

</syntaxhighlight>Then create your virtual environemnt, activate it and install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">python -m venv ./project_env
source /lustre/shared/MyProject/project_env/bin/activate
pip install -U ipykernel</syntaxhighlight>

Here is where things differ, inside the activate kernel run the command<syntaxhighlight lang="bash">
python -m ipykernel install --sys-prefix --name=myproject --display-name="Python (project_env)"
</syntaxhighlight>Note that now the command uses the flag --sys-prefix . This flag sets the installation path of the kernel '''inside''' the virtual environment's directory, in this case, the kernel.json file will be generate in <code>/lustre/shared/MyProject/project_env/share/jupyter/kernels/myproject</code>.

While the kernel has been created, jupyter still will not be able to see it. In order to be able to see and access the kernel you would need to either copy it to your folder<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
cp -r $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>or link it<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
ln -s $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>Any users that run this last step should gain access to the kernel from jupyter. Though note, that only users that have write permissions to virtual environment will be able to install and remove modules from it.

<div style="background-color: #e7f3fe; border-left: 6px solid #2196F3; padding: 10px; margin-bottom: 15px;">
'''Note:''' When using custom kernels do not run <code>!pip</code>, just run <code>pip</code> without the <code>!</code>
</div>

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

Python/Modules

2026-08-12T15:06:07Z

Honfi001: /* Jupyter kernels */

= Modules =
The Python Environment Modules are the only officially supported Python distributions in the HPC. These are compiled for each specific architecture of the HPC and hence are likely going to be more performant than the versions obtained via Mamba or UV.

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Jupyter kernels ==

Once your virtual environment has been setup, you can can access it in jupyter, either via the apps or the notebooks page by setting up a kernel. A kernel is nothing more than a json file with details for jupyter on where to find your Python environment. First things first, with your Python modules loaded activate your virtual environment from the example before.

<syntaxhighlight lang="bash">source $myNobackup/PythonEnv/my_env/bin/activate</syntaxhighlight>The install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">(my_env) user001@login200:~$ pip install -U ipykernel</syntaxhighlight>This is the only dependency that Python needs for creating kernels, you can then run <syntaxhighlight lang="bash">(my_env) user001@login200:~$ python -m ipykernel install --user --name=myenv --display-name="Python (myenv)"</syntaxhighlight>This command will create a folder labelled <code>myenv</code> containing a <code>kernel.json</code> file inside of <code>$HOME/.local/share/jupyter/kernels/</code> , which jupyter watches. If your goal is just to create a kernel for yourself, then you are done. The kernel should be visible to you both inside of the jupyter instances, either the ones accessed via the [https://apps.anunna.wur.nl anuna apps portal] or the [https://notebook.anunna.wur.nl notebooks page]. Thus, the kernel will not be visible or accessible to other users.

The --name=myenv flag is entirely optional, it is the directory in which the <code>kernel.json</code> file is generated. If not used, python will simply label it <code>python3</code> . The use of this flag is recommended, specially when working with multiple kernels to avoid overwriting previously created kernels.

=== Shared Kernels ===
Whether for a course or collaboration, sometimes it is convenient to share a kernel with other users. This is possible and the procedure is a variation of the previous one, with just some additional requirements

* The virtual environment you create needs to be in a shared location , e.g. /lutre/shared
* The virtual environment folder needs to be visiible and accessible to other users i.e. it must have read and execute permissions, 755 is recommended

Like before load the bucket and its corresponding Python version and create your project folder and go there<syntaxhighlight lang="bash">module load 2025
module load Python/3.13.1
mkdir /lustre/shared/MyProject
cd /lustre/shared/MyProject

</syntaxhighlight>Then create your virtual environemnt, activate it and install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">python -m venv ./project_env
source /lustre/shared/MyProject/project_env/bin/activate
pip install -U ipykernel</syntaxhighlight>

Here is where things differ, inside the activate kernel run the command<syntaxhighlight lang="bash">
python -m ipykernel install --sys-prefix --name=myproject --display-name="Python (project_env)"
</syntaxhighlight>Note that now the command uses the flag --sys-prefix . This flag sets the installation path of the kernel '''inside''' the virtual environment's directory, in this case, the kernel.json file will be generate in <code>/lustre/shared/MyProject/project_env/share/jupyter/kernels/myproject</code>.

While the kernel has been created, jupyter still will not be able to see it. In order to be able to see and access the kernel you would need to either copy it to your folder<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
cp -r $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>or link it<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
ln -s $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>Any users that run this last step should gain access to the kernel from jupyter. Though note, that only users that have write permissions to virtual environment will be able to install and remove modules from it.

<div style="background-color: #e7f3fe; border-left: 6px solid #2196F3; padding: 10px; margin-bottom: 15px;">
'''Note:''' When using custom kernels do not run <code>!pip</code>, just run <code>pip</code> without the <code>!</code>
</div>

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

Python/Modules

2026-08-12T14:56:26Z

Honfi001:

= Modules =
The Python Environment Modules are the only officially supported Python distributions in the HPC. These are compiled for each specific architecture of the HPC and hence are likely going to be more performant than the versions obtained via Mamba or UV.

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Jupyter kernels ==

Once your virtual environment has been setup, you can can access it in jupyter, either via the apps or the notebooks page by setting up a kernel. A kernel is nothing more than a json file with details for jupyter on where to find your Python environment. First things first, with your Python modules loaded activate your virtual environment from the example before.

<syntaxhighlight lang="bash">source $myNobackup/PythonEnv/my_env/bin/activate</syntaxhighlight>The install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">(my_env) user001@login200:~$ pip install -U ipykernel</syntaxhighlight>This is the only dependency that Python needs for creating kernels, you can then run <syntaxhighlight lang="bash">(my_env) user001@login200:~$ python -m ipykernel install --user --name=myenv --display-name="Python (myenv)"</syntaxhighlight>This command will create a folder labelled <code>myenv</code> containing a <code>kernel.json</code> file inside of <code>$HOME/.local/share/jupyter/kernels/</code> , which jupyter watches. If your goal is just to create a kernel for yourself, then you are done. The kernel should be visible to you both inside of the jupyter instances, either the ones accessed via the [https://apps.anunna.wur.nl anuna apps portal] or the [https://notebook.anunna.wur.nl notebooks page]. Thus, the kernel will not be visible or accessible to other users.

The --name=myenv flag is entirely optional, it is the directory in which the <code>kernel.json</code> file is generated. If not used, python will simply label it <code>python3</code> . The use of this flag is recommended, specially when working with multiple kernels to avoid overwriting previously created kernels.

=== Shared Kernels ===
Whether for a course or collaboration, sometimes it is convenient to share a kernel with other users. This is possible and the procedure is a variation of the previous one, with just some additional requirements

* The virtual environment you create needs to be in a shared location , e.g. /lutre/shared
* The virtual environment folder needs to be visiible and accessible to other users i.e. it must have read and execute permissions, 755 is recommended

Like before load the bucket and its corresponding Python version and create your project folder and go there<syntaxhighlight lang="bash">module load 2025
module load Python/3.13.1
mkdir /lustre/shared/MyProject
cd /lustre/shared/MyProject

</syntaxhighlight>Then create your virtual environemnt, activate it and install [https://pypi.org/project/ipykernel/ ipyKernel]<syntaxhighlight lang="bash">python -m venv ./project_env
source /lustre/shared/MyProject/project_env/bin/activate
pip install -U ipykernel</syntaxhighlight>

Here is where things differ, inside the activate kernel run the command<syntaxhighlight lang="bash">
python -m ipykernel install --sys-prefix --name=myproject --display-name="Python (project_env)"
</syntaxhighlight>Note that now the command uses the flag --sys-prefix . This flag sets the installation path of the kernel '''inside''' the virtual environment's directory, in this case, the kernel.json file will be generate in <code>/lustre/shared/MyProject/project_env/share/jupyter/kernels/myproject</code>.

While the kernel has been created, jupyter still will not be able to see it. In order to be able to see and access the kernel you would need to either copy it to your folder<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
cp -r $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>or link it<syntaxhighlight lang="bash">
MYPROJ=/lustre/shared/MyProject
ln -s $MYPROJ/project_env/share/jupyter/kernels/myproject $HOME/.local/share/jupyter/kernels/
</syntaxhighlight>Any users that run this last step should gain access to the kernel from jupyter. Though note, that only users that have write permissions to the kernel file will be able to modify

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

Python/Modules

2026-08-12T11:56:02Z

Honfi001:

= Modules =
The Python Environment Modules are the only officially supported Python distributions in the HPC. These are compiled for each specific architecture of the HPC and hence are likely going to be more performant than the versions obtained via Mamba or UV.

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Miniforge (conda / mamba) ==

[https://github.com/conda-forge/miniforge Miniforge] is a minimal installer that gives you the <code>conda</code> and <code>mamba</code> package managers preconfigured to use the community [https://conda-forge.org/ conda-forge] channel. <code>mamba</code> is a fast drop-in replacement for <code>conda</code>. This is the recommended way to use conda-style environments on Anunna; it avoids Anaconda's licensing restrictions.

Download and run the installer, pointing it at a location with room (your Lustre nobackup space, not your home directory):

<syntaxhighlight lang="bash">
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p $myNobackup/miniforge3
</syntaxhighlight>

'''Do not run <code>conda init</code>''' on Anunna — it writes startup code into your <code>~/.bashrc</code> that runs on every login and can interfere with the module system. Instead, activate Miniforge only when you need it:

<syntaxhighlight lang="bash">
source $myNobackup/miniforge3/bin/activate
</syntaxhighlight>

Then create and use environments with <code>mamba</code>:

<syntaxhighlight lang="bash">
mamba create -n myenv python=3.12 numpy pandas
mamba activate myenv
</syntaxhighlight>

== Jupyter kernels ==

To use one of your environments inside [[Jupyter]], register it as a kernel.

=== From a virtual environment ===

A <code>venv</code> kernel needs a wrapper script, because the kernel is launched without your normal shell environment and so cannot load modules by itself. First, with the environment active, install <code>ipykernel</code> and generate the kernel:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
source $myNobackup/PythonEnv/my_env/bin/activate
pip install ipykernel
python -m ipykernel install --user --name=my_env_kernel
</syntaxhighlight>

The kernel is written to <code>~/.local/share/jupyter/kernels/</code>, which Jupyter watches. On its own it will not work, because it cannot find the modules — so write a wrapper script that loads them. Save this as, for example, <code>$HOME/wrap.sh</code>:

<syntaxhighlight lang="bash">
#!/bin/bash -l

module reset
module load 2024
module load Python/3.12.3

exec $myNobackup/PythonEnv/my_env/bin/python -m ipykernel_launcher "$@"
</syntaxhighlight>

The <code>#!/bin/bash -l</code> line starts a login shell, which loads Lmod and sources your <code>~/.bash_aliases</code> (so <code>$myNobackup</code> is defined). Make the wrapper executable with <code>chmod +x $HOME/wrap.sh</code>.

Finally point the kernel at the wrapper by editing <code>~/.local/share/jupyter/kernels/my_env_kernel/kernel.json</code>:

<syntaxhighlight lang="json">
{
"argv": [
"/home/WUR/user001/wrap.sh",
"-f",
"{connection_file}"
],
"display_name": "Python my_env",
"language": "python",
"metadata": {
"debugger": true
}
}
</syntaxhighlight>

The only difference from a plain kernel file is that <code>argv</code> points at the wrapper script instead of the Python executable directly.

=== From a conda / mamba environment ===

A conda or mamba environment is simpler, because the environment is self-contained. With Miniforge active and your environment created, install <code>ipykernel</code> into it and register the kernel:

<syntaxhighlight lang="bash">
mamba create -y -n kernel_test python=3 ipykernel
mamba activate kernel_test
python -m ipykernel install --user --name kernel_test
</syntaxhighlight>

To remove the kernel and environment again:

<syntaxhighlight lang="bash">
jupyter kernelspec uninstall kernel_test
mamba deactivate
mamba remove -y -n kernel_test --all
</syntaxhighlight>

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

Python/Modules

2026-08-12T11:52:28Z

Honfi001: Created page with "= Modules = Anunna provides one Python version per module bucket, plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module: * '''2023''': <code>Python/3.11.3</code> * '''2024''': <code>Python/3.12.3</code> * '''2024''': <code>Python/3.13.1</code> <syntaxhighlight lang="bash"> module load 2024 module load Python/3.12.3 </syntaxhighlight> The bundle module <code>Python-bundle-PyPI</code> adds ma..."

= Modules =

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Miniforge (conda / mamba) ==

[https://github.com/conda-forge/miniforge Miniforge] is a minimal installer that gives you the <code>conda</code> and <code>mamba</code> package managers preconfigured to use the community [https://conda-forge.org/ conda-forge] channel. <code>mamba</code> is a fast drop-in replacement for <code>conda</code>. This is the recommended way to use conda-style environments on Anunna; it avoids Anaconda's licensing restrictions.

Download and run the installer, pointing it at a location with room (your Lustre nobackup space, not your home directory):

<syntaxhighlight lang="bash">
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p $myNobackup/miniforge3
</syntaxhighlight>

'''Do not run <code>conda init</code>''' on Anunna — it writes startup code into your <code>~/.bashrc</code> that runs on every login and can interfere with the module system. Instead, activate Miniforge only when you need it:

<syntaxhighlight lang="bash">
source $myNobackup/miniforge3/bin/activate
</syntaxhighlight>

Then create and use environments with <code>mamba</code>:

<syntaxhighlight lang="bash">
mamba create -n myenv python=3.12 numpy pandas
mamba activate myenv
</syntaxhighlight>

== Jupyter kernels ==

To use one of your environments inside [[Jupyter]], register it as a kernel.

=== From a virtual environment ===

A <code>venv</code> kernel needs a wrapper script, because the kernel is launched without your normal shell environment and so cannot load modules by itself. First, with the environment active, install <code>ipykernel</code> and generate the kernel:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
source $myNobackup/PythonEnv/my_env/bin/activate
pip install ipykernel
python -m ipykernel install --user --name=my_env_kernel
</syntaxhighlight>

The kernel is written to <code>~/.local/share/jupyter/kernels/</code>, which Jupyter watches. On its own it will not work, because it cannot find the modules — so write a wrapper script that loads them. Save this as, for example, <code>$HOME/wrap.sh</code>:

<syntaxhighlight lang="bash">
#!/bin/bash -l

module reset
module load 2024
module load Python/3.12.3

exec $myNobackup/PythonEnv/my_env/bin/python -m ipykernel_launcher "$@"
</syntaxhighlight>

The <code>#!/bin/bash -l</code> line starts a login shell, which loads Lmod and sources your <code>~/.bash_aliases</code> (so <code>$myNobackup</code> is defined). Make the wrapper executable with <code>chmod +x $HOME/wrap.sh</code>.

Finally point the kernel at the wrapper by editing <code>~/.local/share/jupyter/kernels/my_env_kernel/kernel.json</code>:

<syntaxhighlight lang="json">
{
"argv": [
"/home/WUR/user001/wrap.sh",
"-f",
"{connection_file}"
],
"display_name": "Python my_env",
"language": "python",
"metadata": {
"debugger": true
}
}
</syntaxhighlight>

The only difference from a plain kernel file is that <code>argv</code> points at the wrapper script instead of the Python executable directly.

=== From a conda / mamba environment ===

A conda or mamba environment is simpler, because the environment is self-contained. With Miniforge active and your environment created, install <code>ipykernel</code> into it and register the kernel:

<syntaxhighlight lang="bash">
mamba create -y -n kernel_test python=3 ipykernel
mamba activate kernel_test
python -m ipykernel install --user --name kernel_test
</syntaxhighlight>

To remove the kernel and environment again:

<syntaxhighlight lang="bash">
jupyter kernelspec uninstall kernel_test
mamba deactivate
mamba remove -y -n kernel_test --all
</syntaxhighlight>

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

Python

2026-08-12T11:51:11Z

Honfi001:

Python is a high-level, interpreted programming language popular in scientific computing for its readability and its enormous ecosystem of third-party libraries — NumPy, Pandas, scikit-learn, PyTorch, TensorFlow, and many more. This page describes how to use Python on Anunna: the provided modules, how to manage your own packages with virtual environments or Miniforge, and how to expose an environment as a Jupyter kernel.

== Modules ==

Anunna provides one Python version per [[Environment Modules | module bucket]], plus bundle modules that carry a curated set of common extensions. Load a bucket, then the Python module:

* '''2023''': <code>Python/3.11.3</code>
* '''2024''': <code>Python/3.12.3</code>
* '''2024''': <code>Python/3.13.1</code>

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

The bundle module <code>Python-bundle-PyPI</code> adds many frequently-used packages on top of the base interpreter. Use <code>module key <package></code> to find which bundle contains a package you need (see [[Environment Modules#Searching by keyword | searching modules by keyword]]).

For packages not in a module, the two recommended routes are a '''virtual environment''' built on a Python module (below), or '''Miniforge''' for a self-contained conda/mamba setup. The use of Anaconda is discouraged on Anunna — its default channels carry licensing restrictions and the full distribution is heavy; Miniforge is the lighter, unrestricted alternative.

== Virtual environments ==

A virtual environment is a self-contained directory holding a specific Python and its packages, so one project's dependencies cannot clash with another's. Python's built-in <code>venv</code> module is the simplest way to make one on top of a Python module.

First load the Python version you want:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
</syntaxhighlight>

Then create the environment in a location of your choosing. The example uses <code>$myNobackup/PythonEnv</code> — <code>$myNobackup</code> is your Lustre nobackup location, set in your <code>~/.bash_aliases</code> (see [[Installing Personal Software#Aliases and local variables | Aliases and local variables]]). Keeping environments on Lustre rather than your home directory avoids filling your home quota, and the nobackup tier is the right choice because an environment can always be recreated from scratch and so does not need backing up.

<syntaxhighlight lang="bash">
python -m venv $myNobackup/PythonEnv/my_env
</syntaxhighlight>

Activate it whenever you want to use it:

<syntaxhighlight lang="bash">
source $myNobackup/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Once active, the environment name appears as a prefix in your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages with <code>pip</code> while the environment is active; they go into the environment, not your home directory:

<syntaxhighlight lang="bash">
pip install -U numpy pandas matplotlib
</syntaxhighlight>

Leave the environment with <code>deactivate</code>.

== Miniforge (conda / mamba) ==

[https://github.com/conda-forge/miniforge Miniforge] is a minimal installer that gives you the <code>conda</code> and <code>mamba</code> package managers preconfigured to use the community [https://conda-forge.org/ conda-forge] channel. <code>mamba</code> is a fast drop-in replacement for <code>conda</code>. This is the recommended way to use conda-style environments on Anunna; it avoids Anaconda's licensing restrictions.

Download and run the installer, pointing it at a location with room (your Lustre nobackup space, not your home directory):

<syntaxhighlight lang="bash">
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
bash Miniforge3-Linux-x86_64.sh -b -p $myNobackup/miniforge3
</syntaxhighlight>

'''Do not run <code>conda init</code>''' on Anunna — it writes startup code into your <code>~/.bashrc</code> that runs on every login and can interfere with the module system. Instead, activate Miniforge only when you need it:

<syntaxhighlight lang="bash">
source $myNobackup/miniforge3/bin/activate
</syntaxhighlight>

Then create and use environments with <code>mamba</code>:

<syntaxhighlight lang="bash">
mamba create -n myenv python=3.12 numpy pandas
mamba activate myenv
</syntaxhighlight>

== Jupyter kernels ==

To use one of your environments inside [[Jupyter]], register it as a kernel.

=== From a virtual environment ===

A <code>venv</code> kernel needs a wrapper script, because the kernel is launched without your normal shell environment and so cannot load modules by itself. First, with the environment active, install <code>ipykernel</code> and generate the kernel:

<syntaxhighlight lang="bash">
module load 2024
module load Python/3.12.3
source $myNobackup/PythonEnv/my_env/bin/activate
pip install ipykernel
python -m ipykernel install --user --name=my_env_kernel
</syntaxhighlight>

The kernel is written to <code>~/.local/share/jupyter/kernels/</code>, which Jupyter watches. On its own it will not work, because it cannot find the modules — so write a wrapper script that loads them. Save this as, for example, <code>$HOME/wrap.sh</code>:

<syntaxhighlight lang="bash">
#!/bin/bash -l

module reset
module load 2024
module load Python/3.12.3

exec $myNobackup/PythonEnv/my_env/bin/python -m ipykernel_launcher "$@"
</syntaxhighlight>

The <code>#!/bin/bash -l</code> line starts a login shell, which loads Lmod and sources your <code>~/.bash_aliases</code> (so <code>$myNobackup</code> is defined). Make the wrapper executable with <code>chmod +x $HOME/wrap.sh</code>.

Finally point the kernel at the wrapper by editing <code>~/.local/share/jupyter/kernels/my_env_kernel/kernel.json</code>:

<syntaxhighlight lang="json">
{
"argv": [
"/home/WUR/user001/wrap.sh",
"-f",
"{connection_file}"
],
"display_name": "Python my_env",
"language": "python",
"metadata": {
"debugger": true
}
}
</syntaxhighlight>

The only difference from a plain kernel file is that <code>argv</code> points at the wrapper script instead of the Python executable directly.

=== From a conda / mamba environment ===

A conda or mamba environment is simpler, because the environment is self-contained. With Miniforge active and your environment created, install <code>ipykernel</code> into it and register the kernel:

<syntaxhighlight lang="bash">
mamba create -y -n kernel_test python=3 ipykernel
mamba activate kernel_test
python -m ipykernel install --user --name kernel_test
</syntaxhighlight>

To remove the kernel and environment again:

<syntaxhighlight lang="bash">
jupyter kernelspec uninstall kernel_test
mamba deactivate
mamba remove -y -n kernel_test --all
</syntaxhighlight>

== See also ==

* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Jupyter]]
* [[R]]
* [[Apptainer]]

== External links ==

* [https://docs.python.org/3/library/venv.html Python venv documentation]
* [https://github.com/conda-forge/miniforge Miniforge]
* [https://conda-forge.org/ conda-forge]

For PIs

2026-08-12T07:29:38Z

Honfi001:

Information for group leaders, PIs, and teachers running projects and courses on Anunna.

== In this section ==

* [[Dos and Don'ts]] — good and bad practice at a glance
* [[Project Creation]] — setting up a project / billing account
* [[Managing Group Members]] — working with groups
* [[Storage Requests]] — requesting more storage
* [[Resource Allocation Requests]] — requesting larger or dedicated allocations
* [[External Collaborator Access]] — giving access to collaborators outside WUR
* [[Reporting Usage]] — tracking your group's usage and costs
* [[Grant Support Documentation]] — facility descriptions and support for funding applications

== See also ==
* [[Steps for courses]] — running a course on Anunna

Julia

2026-08-11T12:51:17Z

Honfi001: Add table with Julia versions for every bucket

[https://julialang.org/ Julia] is a high-level, high-performance language for technical computing. This page covers using Julia on Anunna.

== Loading Julia ==

Julia is provided through the module system. Load a [[Environment Modules|bucket]] first, then Julia:

<syntaxhighlight lang="bash">
module load 2024
module load Julia/1.11.6
</syntaxhighlight>

To see which versions are available, use <code>module key Julia</code> (see [[Environment Modules]]).

There is at least one version of Julia installed in each bucket:
{| class="wikitable"
|+
!Bucket
!Julia Version
|-
|2023
|1.11.3
|-
|2024
|1.11.6
|-
|2025
|1.12.3
|}

== Packages ==

Julia manages its own packages with the built-in package manager (<code>Pkg</code>). By default packages install into your home directory under <code>~/.julia</code>. Because [[Home Directory|home]] is small and not meant for heavy I/O, point Julia's depot at a larger [[Compute Storage|Lustre]] location by setting <code>JULIA_DEPOT_PATH</code> before starting Julia:

<syntaxhighlight lang="bash">
export JULIA_DEPOT_PATH=/lustre/nobackup/<group>/<user>/julia_depot
</syntaxhighlight>

Then add packages from the Julia REPL:

<syntaxhighlight lang="julia">
import Pkg
Pkg.add("DataFrames")
</syntaxhighlight>

== Running Julia in a job ==

Run Julia scripts as batch jobs through the scheduler — see [[Batch Jobs]]. For interactive work, start an [[Interactive Jobs|interactive job]] first.

To use multiple cores, start Julia with the <code>-t</code> (threads) option, or use Julia's distributed/parallel facilities — see [[Performance Optimization/Multiple CPUs]]:

<syntaxhighlight lang="bash">
julia -t $SLURM_CPUS_PER_TASK myscript.jl
</syntaxhighlight>

== See also ==
* [[Software Overview]]
* [[Environment Modules]]
* [[Jupyter]]
* [[Installing Personal Software]]

Software

2026-08-11T12:18:03Z

Honfi001:

The software available on Anunna and how to use, install, and manage it.

== Organization ==

* [[Software Overview]] — the software landscape
* [[Environment Modules]] — loading software with modules and buckets
* [[Installing Personal Software]] — installing into your own space
* [[Licensed Software]] — software that needs a licence

== Scripting Languanges ==
* [[Python]]
* [[R]]
* [[Julia]]

== Containers ==
* [[Apptainer]] — containers

== MPI Implemenations ==
* [[OpenMPI]]
* [[IntelMPI]]

OpenMPI

2026-08-06T14:55:29Z

Honfi001:

'''OpenMPI''' is the MPI library Anunna is built around. MPI — the Message Passing Interface — is how a single program runs as many cooperating processes at once, each with its own private memory, passing messages between themselves as the calculation proceeds. It is what lets one job use more than one node.

This page is the practical side: which OpenMPI modules exist here, how to launch a job on one node and across several, and how to choose the network transport, which on Anunna needs saying out loud. If you are still deciding whether MPI is the right shape for your work at all, start at [[Workflows/Multi-Process|Multi-Process Workflows]] — that page covers when to use MPI and when a [[Workflows/Embarassinly-Parallel|job array]] or [[Workflows/Multi-threaded|threads]] would serve you better. For a worked example you can run yourself, see [[Tutorials/Parallelism-Estimating-Pi|Parallelism: Estimating π]].

One thing to be clear about before anything else: '''MPI has to be built into the program.''' You cannot add it from a job script. If your software's documentation never mentions MPI or ranks, this page is not the one you need.

== What is available on Anunna ==

Software here is built with [https://easybuild.io EasyBuild] and grouped into [[Environment Modules|buckets]]. A bucket has to be loaded before you can ''load'' anything from it — and before <code>module avail</code> will list its contents. Each bucket corresponds to an EasyBuild toolchain generation:

{| class="wikitable"
! Bucket !! Toolchain !! OpenMPI !! Compiler
|-
| <code>2023</code> || <code>foss/2023a</code>, <code>gompi/2023a</code> || <code>OpenMPI/4.1.5</code> || GCC 12.3.0
|-
| <code>2024</code> || <code>foss/2024a</code>, <code>gompi/2024a</code> || <code>OpenMPI/5.0.3</code> || GCC 13.3.0
|-
| <code>2025</code> || <code>foss/2025a</code>, <code>gompi/2025a</code> || <code>OpenMPI/5.0.7</code> || GCC 14.2.0
|}

Three ways in, and they differ only in how much they bring with them:

* '''<code>foss</code>''' — the full toolchain: GCC, OpenMPI, OpenBLAS, FFTW and ScaLAPACK. Load this if your program wants linear algebra or FFTs as well as MPI.
* '''<code>gompi</code>''' — GCC and OpenMPI, nothing else. The lean choice when MPI is all you need.
* '''<code>OpenMPI/<version></code>''' — the library on its own, with its compiler underneath.

You can list them yourself with <code>module key</code>, which searches module descriptions as well as names. It reads Lmod's cache rather than your current environment, so — unlike <code>module avail</code> — it needs '''no''' bucket loaded and reports matches from every bucket at once:

<syntaxhighlight lang="bash">
module key OpenMPI # no bucket needed; searches across all of them
</syntaxhighlight>

That is the quickest way to see which OpenMPI versions exist and which toolchains carry them. Once you have picked one, ''then'' load its bucket and the module.

=== Loading it ===

Two steps, always in this order: '''the bucket, then the module.''' Nothing from a bucket can be loaded until the bucket itself is.

To get the full toolchain from the 2024 bucket:

<syntaxhighlight lang="bash">module load 2024
module load 2024 OpenMPI/5.0.3 # OpenMPI alone</syntaxhighlight>

The short version number is enough — you do '''not''' need to type the compiler suffix. <code>OpenMPI/5.0.3</code> resolves to <code>OpenMPI/5.0.3-GCC-13.3.0</code>, which is the full name <code>module list</code> will show you afterwards. If you want to see everything a bucket offers, <code>module avail OpenMPI</code> lists it once that bucket is loaded.

Alternatively, swap the second line for modules that bundle OpenMPI with other additional modules:

<syntaxhighlight lang="bash">module load 2024 gompi/2024a # GCC + OpenMPI, no maths libraries
module load foss/2024a # GCC + OpenMPI + maths libraries</syntaxhighlight>

'''Check what you actually got.''' Two commands, both worth the habit

<syntaxhighlight lang="bash">
module list # everything now loaded, including pulled-in dependencies
mpirun --version # confirms which OpenMPI is on your PATH
</syntaxhighlight>

=== Which one to load ===

The rule that saves the most time: '''load the same MPI the program was built against.''' Building with one MPI and running under another produces failures that look like almost anything except the mismatch causing them — hangs, garbled output, crashes inside library calls.

In practice that means:

* '''Running software from the module system?''' Load '''its''' module and let it pull in the matching MPI as a dependency. You never have to choose, and you cannot get it wrong.
* '''Running something you built yourself?''' Load the same toolchain you built it with.
* '''Building it yourself?''' Use the compiler wrappers the MPI module provides — <code>mpicc</code>, <code>mpicxx</code>, <code>mpifort</code> — which call your ordinary compiler with the MPI headers and libraries already in place.

If none of those settles it, prefer the newest bucket your software is available in: the OpenMPI it carries is the one that has had the most attention here.

== Recommended Environment ==
<syntaxhighlight lang="bash">
export OMPI_MCA_pml=cm
export OMPI_MCA_mtl=ofi
export FI_PROVIDER=psm2
export OMPI_MCA_btl=self,sm #substitute sm with vader for v4
</syntaxhighlight>

== Running on a single node ==

Remember that nothing should run on the login nodes — they are shared, and are for editing, submitting and light housekeeping only. Describe the job in a script, keep the data on Lustre under <code>$myScratch</code>, and hand it to <code>sbatch</code>.

<syntaxhighlight lang="bash">#!/bin/bash
#SBATCH --job-name=mpi_single
#SBATCH --nodes=1
#SBATCH --ntasks=64 # 64 ranks, all on one machine
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G # per CORE, not per node
#SBATCH --time=01:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 foss/2024a

export OMPI_MCA_pml=cm
export OMPI_MCA_mtl=ofi
export FI_PROVIDER=psm2
export OMPI_MCA_btl=self,sm

mpirun -np ${SLURM_NTASKS} ./my_mpi_program</syntaxhighlight>

Two habits in there are worth keeping:

* '''<code>-np ${SLURM_NTASKS}</code>, never a hard-coded number.''' Slurm exports the task count it granted; passing that through means the request and the run cannot drift apart when you change one of them.
* '''<code>--mem-per-cpu</code>, not <code>--mem</code>.''' <code>--mem</code> is a ''per-node'' request shared out among all ranks on that node — at 64 or 128 ranks each one ends up with a few megabytes, less than an MPI process needs simply to start, and the job dies of an out-of-memory kill that points nowhere near the flag responsible.

On one node OpenMPI talks to itself through shared memory. There is nothing to configure and no network involved, which is why this case is simple and the next one is not.

== Running across several nodes ==

The moment a job spans two nodes, the ranks have to talk over the network — and on Anunna that path needs to be chosen explicitly.

Anunna's fabric is '''Omni-Path (OPA100)''', and inter-node MPI rides it through '''libfabric'''. libfabric offers more than one ''provider'' for this hardware, and on our cluster they are not equally healthy:

{| class="wikitable"
! Provider !! Status on Anunna !! Use it?
|-
| <code>psm2</code> || Validated at line rate (~97 Gbps), on both Intel and AMD nodes, under all three toolchains || '''Yes — this is the one'''
|-
| <code>opx</code> || Not currently operational || Not yet
|}

Because TCP can be picked up automatically, '''an inter-node MPI job should explicitly ask for <code>psm2</code>''' rather than trusting the default:

<syntaxhighlight lang="bash">
mpirun --mca pml cm --mca mtl ofi --mca mtl_ofi_provider_include psm2 \
-x FI_PROVIDER=psm2 \
-np ${SLURM_NTASKS} ./my_mpi_program --input data.nc
</syntaxhighlight>

A complete multi-node job script:

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=mpi_multi
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=64 # 256 ranks in total
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G
#SBATCH --time=02:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 foss/2024a

mpirun --mca pml cm --mca mtl ofi \
--mca mtl_ofi_provider_include psm2 \
--mca btl self,sm \
-x FI_PROVIDER=psm2 \
-np ${SLURM_NTASKS} ./my_mpi_program
</syntaxhighlight>Note some of the settings can be done by OpenMPI's own tags.

=== Exporting variables to the other nodes: the <code>-x</code> flag ===

<code>-x</code> hands a named environment variable to every rank, including those in other nodes :

<syntaxhighlight lang="bash">
mpirun -x FI_PROVIDER=psm2 -x MY_DATA_DIR -np ${SLURM_NTASKS} ./prog
</syntaxhighlight>

<code>-x VAR=value</code> sets it; <code>-x VAR</code> passes through whatever value it already has. Use one <code>-x</code> per variable.

'''With OpenMPI 5 inside a Slurm job, you usually do not need it.''' '''Though, do not read that as universal, it depends on the version.''' It was verified for OpenMPI 5.0.7, in the <code>2025</code> bucket. The <code>2023</code> bucket carries '''OpenMPI 4.1.5''', which may not propagate your environment automatically; historically variables did have to be passed by hand. And in a hostfile launch over <code>ssh</code>, rather than inside a Slurm allocation, remote shells start clean and nothing travels at all.

So use <code>-x</code> if you are on the <code>2023</code> bucket, if you are launching outside Slurm, or if you simply would rather not depend on any of the above. It costs nothing when it turns out to be redundant, and it is the one form that works in every case.

So treat <code>-x</code> as the '''explicit''' form rather than a workaround. It costs nothing, it states in the launch line what the run depends on, and it keeps working if the launch path changes underneath you. And if a job behaves differently across several nodes than it does on one, a variable that failed to travel is still the first thing worth suspecting.

=== A start-up gotcha ===

Do not use <code>sbatch --wrap</code> for MPI jobs. The wrapped command runs under <code>dash</code>, where Lmod's <code>module</code> function is not initialised, so module loads fail quietly and the job proceeds without the software it asked for. Write a real script starting <code>#!/bin/bash -l</code> instead.

== Advanced: pinning and rank placement ==

By default OpenMPI decides where each rank sits and which cores it may use. For most jobs that is fine. It stops being fine when a job is memory-bandwidth-bound, when ranks each run threads of their own, or when timings vary between runs for no visible reason — all of which come down to ''placement''.

Two ideas, easy to mix up:

* '''Mapping''' — which rank goes where. <code>--map-by <unit></code>, where the unit can be <code>core</code>, <code>l3cache</code>, <code>numa</code>, <code>package</code> (socket), or <code>node</code>.
* '''Binding''' — how tightly a rank is then pinned. <code>--bind-to core</code> is the usual choice; <code>--bind-to none</code> lets a rank roam.

Before tuning anything, look at what you are actually getting:

<syntaxhighlight lang="bash">
mpirun --report-bindings -np ${SLURM_NTASKS} ./my_mpi_program
</syntaxhighlight>

That prints one line per rank showing the cores it is bound to. It is the only way to know rather than assume, and defaults differ between OpenMPI versions.

=== Why multiples of eight suit our AMD nodes ===

Most of Anunna's compute is AMD — '''Zen 3''' (EPYC Milan) and '''Zen 5''' (EPYC Turin). Both build a socket out of ''chiplets'' of '''8 cores''', and each chiplet has its own slice of L3 cache. Cores inside a chiplet share that cache and reach memory together; cores in different chiplets do not.

The practical consequence: '''ask for ranks in multiples of 8''', so that chiplets are filled evenly and no chiplet is left holding a single stray rank while its cache and memory channels go mostly unused. 64, 128 and 256 are comfortable numbers here; 50 or 100 are not, and will spread unevenly no matter how you map them.

Mapping by cache makes the intent explicit — one rank per chiplet, each free to use that chiplet's cores:

<syntaxhighlight lang="bash">
mpirun --map-by l3cache --bind-to core -np ${SLURM_NTASKS} ./my_mpi_program
</syntaxhighlight>

A related point for memory-bound work: spreading ranks across chiplets and NUMA domains uses many memory controllers at once, while packing them close leaves most idle. On our AMD nodes that difference is large, not marginal.

=== Ranks with threads inside them ===

If each rank also runs OpenMP threads, mapping and binding stop being optional. Give each rank a block of cores with <code>PE=</code>, and bind inside it:

<syntaxhighlight lang="bash">
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
mpirun --map-by ppr:8:node:PE=${SLURM_CPUS_PER_TASK} --bind-to core \
-x OMP_NUM_THREADS \
-np ${SLURM_NTASKS} ./my_hybrid_program
</syntaxhighlight>

That reads as: place 8 ranks per node, give each one <code>SLURM_CPUS_PER_TASK</code> cores, and bind them there. '''Without the <code>PE=</code> part every thread of a rank can end up on one single core''' — the job runs, produces correct results, and is many times slower than it should be. It is a silent failure, so check it with <code>--report-bindings</code>.

Note also that '''MPI binding and OpenMP affinity are separate knobs.''' <code>--map-by</code> and <code>--bind-to</code> decide which cores a ''rank'' owns; <code>OMP_PROC_BIND</code> and <code>OMP_PLACES</code> decide how that rank's ''threads'' arrange themselves within those cores. Setting the OpenMP variables cannot rescue a rank that was only given one core to begin with — the threads have nowhere to spread to. Get the MPI side right first.

== See also ==

* [[Workflows/Multi-Process|Multi-Process Workflows]] — when MPI is the right choice, and what the job script looks like.
* [[Intel MPI]] — the other MPI here, its <code>I_MPI_*</code> pinning dialect, and MKL's behaviour on AMD nodes.
* [[Workflows/Multi-threaded|Multi-threaded Workflows]] — threads on one node, for programs that are not MPI.
* [[Tutorials/Parallelism-Estimating-Pi|Parallelism: Estimating π]] — the same calculation run serially, threaded, with MPI, and across nodes.
* [[Environment Modules]] — buckets, and how to find the module you need.
* [[Compute Hardware Overview]] — the nodes, their core counts and their memory.
* [[Batch Jobs]] — writing and submitting job scripts.
* [[Scheduler Overview (Slurm)]] — how the scheduler allocates what you ask for.

Software

2026-08-06T14:48:30Z

Honfi001:

The software available on Anunna and how to use, install, and manage it.

== In this section ==

* [[Software Overview]] — the software landscape
* [[Environment Modules]] — loading software with modules and buckets
* [[Installing Personal Software]] — installing into your own space
* [[Python]] — Python (modules, Miniforge, virtual environments)
* [[R]] — R-lang and libraries
* [[Julia]] — Julia
* [[Apptainer]] — containers
* [[OpenMPI]]
* [[IntelMPI]]
* [[Licensed Software]] — software that needs a licence

OpenMPI

2026-08-06T14:46:43Z

Honfi001: Add recommended environment section

'''OpenMPI''' is the MPI library Anunna is built around. MPI — the Message Passing Interface — is how a single program runs as many cooperating processes at once, each with its own private memory, passing messages between themselves as the calculation proceeds. It is what lets one job use more than one node.

This page is the practical side: which OpenMPI modules exist here, how to launch a job on one node and across several, and how to choose the network transport, which on Anunna needs saying out loud. If you are still deciding whether MPI is the right shape for your work at all, start at [[Workflows/Multi-Process|Multi-Process Workflows]] — that page covers when to use MPI and when a [[Workflows/Embarassinly-Parallel|job array]] or [[Workflows/Multi-threaded|threads]] would serve you better. For a worked example you can run yourself, see [[Tutorials/Parallelism-Estimating-Pi|Parallelism: Estimating π]].

One thing to be clear about before anything else: '''MPI has to be built into the program.''' You cannot add it from a job script. If your software's documentation never mentions MPI or ranks, this page is not the one you need.

== What is available on Anunna ==

Software here is built with [https://easybuild.io EasyBuild] and grouped into [[Environment Modules|buckets]]. A bucket has to be loaded before you can ''load'' anything from it — and before <code>module avail</code> will list its contents. Each bucket corresponds to an EasyBuild toolchain generation:

{| class="wikitable"
! Bucket !! Toolchain !! OpenMPI !! Compiler
|-
| <code>2023</code> || <code>foss/2023a</code>, <code>gompi/2023a</code> || <code>OpenMPI/4.1.5</code> || GCC 12.3.0
|-
| <code>2024</code> || <code>foss/2024a</code>, <code>gompi/2024a</code> || <code>OpenMPI/5.0.3</code> || GCC 13.3.0
|-
| <code>2025</code> || <code>foss/2025a</code>, <code>gompi/2025a</code> || <code>OpenMPI/5.0.7</code> || GCC 14.2.0
|}

Three ways in, and they differ only in how much they bring with them:

* '''<code>foss</code>''' — the full toolchain: GCC, OpenMPI, OpenBLAS, FFTW and ScaLAPACK. Load this if your program wants linear algebra or FFTs as well as MPI.
* '''<code>gompi</code>''' — GCC and OpenMPI, nothing else. The lean choice when MPI is all you need.
* '''<code>OpenMPI/<version></code>''' — the library on its own, with its compiler underneath.

You can list them yourself with <code>module key</code>, which searches module descriptions as well as names. It reads Lmod's cache rather than your current environment, so — unlike <code>module avail</code> — it needs '''no''' bucket loaded and reports matches from every bucket at once:

<syntaxhighlight lang="bash">
module key OpenMPI # no bucket needed; searches across all of them
</syntaxhighlight>

That is the quickest way to see which OpenMPI versions exist and which toolchains carry them. Once you have picked one, ''then'' load its bucket and the module.

=== Loading it ===

Two steps, always in this order: '''the bucket, then the module.''' Nothing from a bucket can be loaded until the bucket itself is.

To get the full toolchain from the 2024 bucket:

<syntaxhighlight lang="bash">module load 2024
module load 2024 OpenMPI/5.0.3 # OpenMPI alone</syntaxhighlight>

The short version number is enough — you do '''not''' need to type the compiler suffix. <code>OpenMPI/5.0.3</code> resolves to <code>OpenMPI/5.0.3-GCC-13.3.0</code>, which is the full name <code>module list</code> will show you afterwards. If you want to see everything a bucket offers, <code>module avail OpenMPI</code> lists it once that bucket is loaded.

Alternatively, swap the second line for modules that bundle OpenMPI with other additional modules:

<syntaxhighlight lang="bash">module load 2024 gompi/2024a # GCC + OpenMPI, no maths libraries
module load foss/2024a # GCC + OpenMPI + maths libraries</syntaxhighlight>

'''Check what you actually got.''' Two commands, both worth the habit

<syntaxhighlight lang="bash">
module list # everything now loaded, including pulled-in dependencies
mpirun --version # confirms which OpenMPI is on your PATH
</syntaxhighlight>

=== Which one to load ===

The rule that saves the most time: '''load the same MPI the program was built against.''' Building with one MPI and running under another produces failures that look like almost anything except the mismatch causing them — hangs, garbled output, crashes inside library calls.

In practice that means:

* '''Running software from the module system?''' Load '''its''' module and let it pull in the matching MPI as a dependency. You never have to choose, and you cannot get it wrong.
* '''Running something you built yourself?''' Load the same toolchain you built it with.
* '''Building it yourself?''' Use the compiler wrappers the MPI module provides — <code>mpicc</code>, <code>mpicxx</code>, <code>mpifort</code> — which call your ordinary compiler with the MPI headers and libraries already in place.

If none of those settles it, prefer the newest bucket your software is available in: the OpenMPI it carries is the one that has had the most attention here.

== Recommended Environment ==
<syntaxhighlight lang="bash">
export OMPI_MCA_pml=cm
export OMPI_MCA_mtl=ofi
export FI_PROVIDER=psm2
export OMPI_MCA_btl=self,sm #substitute sm with vader for v4
</syntaxhighlight>

== Running on a single node ==

Remember that nothing should run on the login nodes — they are shared, and are for editing, submitting and light housekeeping only. Describe the job in a script, keep the data on Lustre under <code>$myScratch</code>, and hand it to <code>sbatch</code>.

<syntaxhighlight lang="bash">#!/bin/bash
#SBATCH --job-name=mpi_single
#SBATCH --nodes=1
#SBATCH --ntasks=64 # 64 ranks, all on one machine
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G # per CORE, not per node
#SBATCH --time=01:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 foss/2024a

mpirun -np ${SLURM_NTASKS} ./my_mpi_program</syntaxhighlight>

Two habits in there are worth keeping:

* '''<code>-np ${SLURM_NTASKS}</code>, never a hard-coded number.''' Slurm exports the task count it granted; passing that through means the request and the run cannot drift apart when you change one of them.
* '''<code>--mem-per-cpu</code>, not <code>--mem</code>.''' <code>--mem</code> is a ''per-node'' request shared out among all ranks on that node — at 64 or 128 ranks each one ends up with a few megabytes, less than an MPI process needs simply to start, and the job dies of an out-of-memory kill that points nowhere near the flag responsible.

On one node OpenMPI talks to itself through shared memory. There is nothing to configure and no network involved, which is why this case is simple and the next one is not.

== Running across several nodes ==

The moment a job spans two nodes, the ranks have to talk over the network — and on Anunna that path needs to be chosen explicitly.

Anunna's fabric is '''Omni-Path (OPA100)''', and inter-node MPI rides it through '''libfabric'''. libfabric offers more than one ''provider'' for this hardware, and on our cluster they are not equally healthy:

{| class="wikitable"
! Provider !! Status on Anunna !! Use it?
|-
| <code>psm2</code> || Validated at line rate (~97 Gbps), on both Intel and AMD nodes, under all three toolchains || '''Yes — this is the one'''
|-
| <code>opx</code> || Not currently operational || Not yet
|}

Because TCP can be picked up automatically, '''an inter-node MPI job should explicitly ask for <code>psm2</code>''' rather than trusting the default:

<syntaxhighlight lang="bash">
mpirun --mca pml cm --mca mtl ofi --mca mtl_ofi_provider_include psm2 \
-x FI_PROVIDER=psm2 \
-np ${SLURM_NTASKS} ./my_mpi_program --input data.nc
</syntaxhighlight>

A complete multi-node job script:

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=mpi_multi
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=64 # 256 ranks in total
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G
#SBATCH --time=02:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 foss/2024a

mpirun --mca pml cm --mca mtl ofi --mca mtl_ofi_provider_include psm2 \
-x FI_PROVIDER=psm2 \
-np ${SLURM_NTASKS} ./my_mpi_program --input data.nc
</syntaxhighlight>

=== Exporting variables to the other nodes: the <code>-x</code> flag ===

<code>-x</code> hands a named environment variable to every rank:

<syntaxhighlight lang="bash">
mpirun -x FI_PROVIDER=psm2 -x MY_DATA_DIR -np ${SLURM_NTASKS} ./prog
</syntaxhighlight>

<code>-x VAR=value</code> sets it; <code>-x VAR</code> passes through whatever value it already has. Use one <code>-x</code> per variable.

'''With OpenMPI 5 inside a Slurm job, you usually do not need it.''' '''Though, do not read that as universal, it depends on the version.''' It was verified for OpenMPI 5.0.7, in the <code>2025</code> bucket. The <code>2023</code> bucket carries '''OpenMPI 4.1.5''', which predates the PRRTE launcher and has not been tested here; historically variables did have to be passed by hand. And in a hostfile launch over <code>ssh</code>, rather than inside a Slurm allocation, remote shells start clean and nothing travels at all.

So use <code>-x</code> if you are on the <code>2023</code> bucket, if you are launching outside Slurm, or if you simply would rather not depend on any of the above. It costs nothing when it turns out to be redundant, and it is the one form that works in every case.

So treat <code>-x</code> as the '''explicit''' form rather than a workaround. It costs nothing, it states in the launch line what the run depends on, and it keeps working if the launch path changes underneath you. And if a job behaves differently across several nodes than it does on one, a variable that failed to travel is still the first thing worth suspecting.

=== A start-up gotcha ===

Do not use <code>sbatch --wrap</code> for MPI jobs. The wrapped command runs under <code>dash</code>, where Lmod's <code>module</code> function is not initialised, so module loads fail quietly and the job proceeds without the software it asked for. Write a real script starting <code>#!/bin/bash -l</code> instead.

== Advanced: pinning and rank placement ==

By default OpenMPI decides where each rank sits and which cores it may use. For most jobs that is fine. It stops being fine when a job is memory-bandwidth-bound, when ranks each run threads of their own, or when timings vary between runs for no visible reason — all of which come down to ''placement''.

Two ideas, easy to mix up:

* '''Mapping''' — which rank goes where. <code>--map-by <unit></code>, where the unit can be <code>core</code>, <code>l3cache</code>, <code>numa</code>, <code>package</code> (socket), or <code>node</code>.
* '''Binding''' — how tightly a rank is then pinned. <code>--bind-to core</code> is the usual choice; <code>--bind-to none</code> lets a rank roam.

Before tuning anything, look at what you are actually getting:

<syntaxhighlight lang="bash">
mpirun --report-bindings -np ${SLURM_NTASKS} ./my_mpi_program
</syntaxhighlight>

That prints one line per rank showing the cores it is bound to. It is the only way to know rather than assume, and defaults differ between OpenMPI versions.

=== Why multiples of eight suit our AMD nodes ===

Most of Anunna's compute is AMD — '''Zen 3''' (EPYC Milan) and '''Zen 5''' (EPYC Turin). Both build a socket out of ''chiplets'' of '''8 cores''', and each chiplet has its own slice of L3 cache. Cores inside a chiplet share that cache and reach memory together; cores in different chiplets do not.

The practical consequence: '''ask for ranks in multiples of 8''', so that chiplets are filled evenly and no chiplet is left holding a single stray rank while its cache and memory channels go mostly unused. 64, 128 and 256 are comfortable numbers here; 50 or 100 are not, and will spread unevenly no matter how you map them.

Mapping by cache makes the intent explicit — one rank per chiplet, each free to use that chiplet's cores:

<syntaxhighlight lang="bash">
mpirun --map-by l3cache --bind-to core -np ${SLURM_NTASKS} ./my_mpi_program
</syntaxhighlight>

A related point for memory-bound work: spreading ranks across chiplets and NUMA domains uses many memory controllers at once, while packing them close leaves most idle. On our AMD nodes that difference is large, not marginal.

=== Ranks with threads inside them ===

If each rank also runs OpenMP threads, mapping and binding stop being optional. Give each rank a block of cores with <code>PE=</code>, and bind inside it:

<syntaxhighlight lang="bash">
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
mpirun --map-by ppr:8:node:PE=${SLURM_CPUS_PER_TASK} --bind-to core \
-x OMP_NUM_THREADS \
-np ${SLURM_NTASKS} ./my_hybrid_program
</syntaxhighlight>

That reads as: place 8 ranks per node, give each one <code>SLURM_CPUS_PER_TASK</code> cores, and bind them there. '''Without the <code>PE=</code> part every thread of a rank can end up on one single core''' — the job runs, produces correct results, and is many times slower than it should be. It is a silent failure, so check it with <code>--report-bindings</code>.

Note also that '''MPI binding and OpenMP affinity are separate knobs.''' <code>--map-by</code> and <code>--bind-to</code> decide which cores a ''rank'' owns; <code>OMP_PROC_BIND</code> and <code>OMP_PLACES</code> decide how that rank's ''threads'' arrange themselves within those cores. Setting the OpenMP variables cannot rescue a rank that was only given one core to begin with — the threads have nowhere to spread to. Get the MPI side right first.

== See also ==

* [[Workflows/Multi-Process|Multi-Process Workflows]] — when MPI is the right choice, and what the job script looks like.
* [[Intel MPI]] — the other MPI here, its <code>I_MPI_*</code> pinning dialect, and MKL's behaviour on AMD nodes.
* [[Workflows/Multi-threaded|Multi-threaded Workflows]] — threads on one node, for programs that are not MPI.
* [[Tutorials/Parallelism-Estimating-Pi|Parallelism: Estimating π]] — the same calculation run serially, threaded, with MPI, and across nodes.
* [[Environment Modules]] — buckets, and how to find the module you need.
* [[Compute Hardware Overview]] — the nodes, their core counts and their memory.
* [[Batch Jobs]] — writing and submitting job scripts.
* [[Scheduler Overview (Slurm)]] — how the scheduler allocates what you ask for.

IntelMPI

2026-08-05T14:46:12Z

Honfi001:

'''Intel MPI''' is the other MPI implementation available on Anunna, alongside [[OpenMPI]]. It arrives as part of the Intel toolchain, together with the Intel compilers and the Intel Math Kernel Library (MKL).

Everything general about MPI — that it runs one program as many cooperating processes, that it has to be built into the program, and when it is the right tool at all — is covered on [[Workflows/Multi-Process|Multi-Process Workflows]]. This page is the Intel-specific practice: which modules exist, how to launch, how the pinning controls differ from OpenMPI's, and one thing that catches people out on our AMD nodes.

'''Which should you use?''' Whichever your program was built with. That is not a dodge — mixing MPI implementations between build and run does not work, so in practice the software chooses for you. Where you genuinely have a choice, Intel MPI and MKL are the natural pairing for code that leans on Intel's maths libraries, and [[OpenMPI]] is the default for everything else here.

== What is available on Anunna ==

Software is built with [https://easybuild.io EasyBuild] and grouped into [[Environment Modules|buckets]]; a bucket has to be loaded before you can load anything from it. The Intel toolchain follows the same yearly generations as <code>foss</code>:

{| class="wikitable"
! Bucket !! Toolchain !! Contains
|-
| <code>2023</code> || <code>intel/2023a</code> || Intel compilers, Intel MPI, MKL
|-
| <code>2024</code> || <code>intel/2024a</code> || Intel compilers, Intel MPI, MKL
|-
| <code>2025</code> || <code>intel/2025a</code> || Intel compilers, Intel MPI, MKL
|}

<code>intel/<year>a</code> is the counterpart of <code>foss/<year>a</code>: one module that brings the whole stack — compilers, MPI, MKL and FFTW. There are leaner ways in, mirroring <code>gompi</code> on the <code>foss</code> side:

* '''<code>iimpi/<year>a</code>''' — Intel compilers and Intel MPI, without MKL. The lean choice when you only need MPI.
* '''<code>impi/<version></code>''' — the MPI library alone. In the 2025 bucket this is <code>impi/2021.15.0-intel-compilers-2025.1.1</code>; note that Intel MPI's own version numbering (2021.x) does not track the toolchain year.

To see what a bucket actually offers, ask the module system — <code>module key</code> reads Lmod's cache, so it needs no bucket loaded and searches every bucket at once:

<syntaxhighlight lang="bash">
module key intel # the toolchains
module key impi # the MPI library on its own
</syntaxhighlight>

=== Loading it ===

Bucket first, then the module — the same two steps as any other software here:

<syntaxhighlight lang="bash">
module load 2024 intel/2024a
</syntaxhighlight>

Then confirm what you got, which is worth the habit:

<syntaxhighlight lang="bash">
module list # everything loaded, dependencies included
mpirun -version # which Intel MPI is on your PATH
</syntaxhighlight>

Keep a job inside one bucket. Do not load <code>intel</code> and <code>foss</code> in the same job — they provide competing compilers, MPI libraries and BLAS, and the result is unpredictable rather than merely slow. If you want a clean starting point, <code>module reset</code> returns you to the system defaults; the <code>slurm</code> module is ''sticky'' and survives it either way.

== Running on a single node ==

Nothing runs on the login nodes. Put the work in a script, keep data on Lustre under <code>$myScratch</code>, and submit with <code>sbatch</code>.

<syntaxhighlight lang="bash">
#!/bin/bash -l
#SBATCH --job-name=impi_single
#SBATCH --nodes=1
#SBATCH --ntasks=64
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G # per CORE, not per node
#SBATCH --time=01:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 intel/2024a

mpirun -np ${SLURM_NTASKS} ./my_mpi_program --input data.nc
</syntaxhighlight>

Two things carried over from [[OpenMPI]], for the same reasons: pass <code>${SLURM_NTASKS}</code> rather than hard-coding the rank count, and use <code>--mem-per-cpu</code> rather than <code>--mem</code>, which is a per-''node'' request divided among all the ranks on that node.

Note the <code>#!/bin/bash -l</code>. A login shell is what initialises Lmod, so without it <code>module load</code> can fail quietly. For the same reason, never launch an MPI job with <code>sbatch --wrap</code> — that runs under <code>dash</code>, where <code>module</code> does not exist.

On one node the ranks talk through shared memory and there is nothing to configure.

== Running across several nodes ==

Anunna's fabric is '''Omni-Path (OPA100)''', and both MPI implementations reach it through '''libfabric'''. The provider that works reliably here is '''<code>psm2</code>'''; the <code>opx</code> provider is broken for inter-node traffic on this hardware. So an inter-node job should name the provider rather than trust auto-selection:

<syntaxhighlight lang="bash">
#!/bin/bash -l
#SBATCH --job-name=impi_multi
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=64
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G
#SBATCH --time=02:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 intel/2024a

mpirun -genv I_MPI_OFI_PROVIDER psm2 -genv FI_PROVIDER psm2 \
-np ${SLURM_NTASKS} ./my_mpi_program --input data.nc
</syntaxhighlight>

To see what was actually selected — provider, pinning, rank layout — turn up Intel MPI's own diagnostics:

<syntaxhighlight lang="bash">
export I_MPI_DEBUG=4
</syntaxhighlight>

That prints the chosen fabric and the rank-to-core map at start-up, and is the fastest way to confirm a setting took effect instead of assuming it did.

=== Getting variables to the other nodes ===

Ranks on remote nodes are started fresh, so any variable your program or the fabric depends on has to reach them. Intel MPI's controls are:

{| class="wikitable"
! Option !! Effect
|-
| <code>-genv <VAR> <value></code> || Set one variable for '''all''' ranks. The explicit, always-safe form.
|-
| <code>-genvall</code> || Pass the whole launching environment to all ranks. This is the default.
|-
| <code>-genvlist <a,b,c></code> || Pass only the named variables.
|-
| <code>-genvnone</code> || Pass nothing.
|}

In practice '''a plain <code>export</code> in your job script does reach every rank''' — tested on Anunna across two nodes.

It arrives whatever you do to stop it: restricting Intel MPI's own propagation with <code>-genvlist PATH</code>, which should have excluded the test variable, delivered it to both nodes anyway. [[OpenMPI]] 5.0.7 behaved identically — with <code>-x</code>, without it, and even with Slurm's own environment forwarding suppressed.

The practical reading is that '''inside a Slurm allocation both implementations get your environment to the ranks''', by way of the process-management layer (PMIx and the launcher's job description) rather than the batch environment. On the OpenMPI side that was pinned down: removing the variable from <code>mpirun</code>'s own environment is the only thing that stopped it arriving — see [[OpenMPI#Exporting variables to the other nodes: the -x flag|the OpenMPI page]] for the full result.

None of which is worth depending on in a job script. It held for the current toolchains and not necessarily for older ones, so say what your run needs explicitly with <code>-genv</code> and the question stops mattering.

'''<code>-genv</code> is still the habit worth keeping''' for anything the run depends on, as in the script above. It does not rely on which layer happens to be propagating, it states the intent in the launch line, it survives a site or script that restricts propagation later, and it is the first thing to try if a variable does mysteriously fail to arrive.

== Intel MPI, MKL, and AMD nodes ==

Most of Anunna's compute is AMD, and the Intel stack has a well-known wrinkle there. It is worth being precise about where it lives: '''the issue is in MKL, not in Intel MPI.''' But MKL arrives with the same <code>intel/<year>a</code> module, and "our Intel build is slow on the AMD nodes" is usually this, so it belongs on this page.

MKL checks the '''CPU vendor''' at run time. On a non-Intel CPU, kernels without specific Zen coverage can fall back to an '''SSE''' code path instead of using '''AVX2''', leaving much of the vector width unused. This is still the behaviour in MKL 2025.x — it has not been quietly fixed. The old <code>MKL_DEBUG_CPU_TYPE</code> workaround is gone, removed back in MKL 2020 Update 1, so anything you read recommending it is out of date.

The current mitigation is a small shim, '''<code>libfakeintel.so</code>''', <code>LD_PRELOAD</code>ed ahead of MKL. It overrides the vendor test (<code>mkl_serv_intel_cpu_true</code>) so that it returns true, and MKL then dispatches its AVX2 kernels:

<syntaxhighlight lang="bash">
export LD_PRELOAD=/path/to/libfakeintel.so
mpirun -np ${SLURM_NTASKS} ./my_mpi_program
</syntaxhighlight>

Before assuming this is your problem, two things are worth checking, because often it is not. The shim only ''widens arithmetic'', so it pays off when MKL is doing '''compute-bound work'''. It does nothing when MKL is only handling bandwidth-bound work, and nothing at all if your library does its own arithmetic e.g. PETSc.

<code>MKL_VERBOSE=1</code> is the quick way to find out: it prints every MKL call, so you can see whether MKL is on your hot path at all and what it is being asked to do. Keep it off any run you intend to time — at high call counts the logging is itself a real cost.

If MKL turns out to suit your workload poorly on the AMD nodes, the <code>foss</code> toolchain's OpenBLAS-based BLAS is worth benchmarking against.

== Advanced: pinning and rank placement ==

Intel MPI and OpenMPI express placement differently: OpenMPI takes command-line flags, Intel MPI reads <code>I_MPI_*</code> environment variables. The intents map across cleanly, so if you know one dialect this table gives you the other:

{| class="wikitable"
! Intent !! Intel MPI !! OpenMPI
|-
| One rank per physical core, no migration || <code>I_MPI_PIN=1</code>, <code>I_MPI_PIN_DOMAIN=core</code> || <code>--bind-to core</code>
|-
| Fill the node, neighbours adjacent || <code>I_MPI_PIN_ORDER=compact</code> || <code>--map-by core</code>
|-
| Under-subscribed, spread across all NUMA domains || <code>I_MPI_PIN_ORDER=scatter</code> || <code>--map-by numa --bind-to core</code>
|-
| One rank per chiplet / L3 slice || <code>I_MPI_PIN_DOMAIN=cache3</code> || <code>--map-by l3cache --bind-to core</code>
|-
| Print the map actually used || <code>I_MPI_DEBUG=4</code> || <code>--report-bindings</code>
|}

Three rules that matter more than the individual settings:

* '''Verify, do not assume.''' Print the map with <code>I_MPI_DEBUG=4</code> before and after any change. Placement bugs do not announce themselves; the job runs and is simply slower than it should be.
* '''One source of truth.''' Pin with the MPI launcher '''or''' with Slurm (<code>--cpu-bind</code>, <code>--distribution</code>), never both. Two pinners fighting each other produce a nonsense map.
* '''Spread suits memory-bound work.''' AMD sockets are built from 8-core chiplets, each with its own cache slice and share of the memory channels. A bandwidth-bound job saturates memory well before every core is busy, so spreading a reduced number of ranks across all NUMA domains keeps every memory channel active, while packing them together leaves most idle. Asking for ranks in multiples of 8 keeps chiplets evenly filled.

For the reasoning behind that last point, and the equivalent OpenMPI syntax, see [[OpenMPI#Advanced: pinning and rank placement|the OpenMPI page]].

== See also ==

* [[OpenMPI]] — the other MPI here, and the default for most software.
* [[Workflows/Multi-Process|Multi-Process Workflows]] — whether MPI is the right shape for your work.
* [[Environment Modules]] — buckets, and finding the module you need.
* [[Compute Hardware Overview]] — what the nodes actually are.
* [[Batch Jobs]] — writing and submitting job scripts.
* [[Scheduler Overview (Slurm)]] — how the scheduler allocates what you ask for.

IntelMPI

2026-08-05T14:40:48Z

Honfi001: Created page with "'''Intel MPI''' is the other MPI implementation available on Anunna, alongside OpenMPI. It arrives as part of the Intel toolchain, together with the Intel compilers and the Intel Math Kernel Library (MKL). Everything general about MPI — that it runs one program as many cooperating processes, that it has to be built into the program, and when it is the right tool at all — is covered on Multi-Process Workflows. This page is the Intel-sp..."

'''Intel MPI''' is the other MPI implementation available on Anunna, alongside [[OpenMPI]]. It arrives as part of the Intel toolchain, together with the Intel compilers and the Intel Math Kernel Library (MKL).

Everything general about MPI — that it runs one program as many cooperating processes, that it has to be built into the program, and when it is the right tool at all — is covered on [[Workflows/Multi-Process|Multi-Process Workflows]]. This page is the Intel-specific practice: which modules exist, how to launch, how the pinning controls differ from OpenMPI's, and one thing that catches people out on our AMD nodes.

'''Which should you use?''' Whichever your program was built with. That is not a dodge — mixing MPI implementations between build and run does not work, so in practice the software chooses for you. Where you genuinely have a choice, Intel MPI and MKL are the natural pairing for code that leans on Intel's maths libraries, and [[OpenMPI]] is the default for everything else here.

== What is available on Anunna ==

Software is built with [https://easybuild.io EasyBuild] and grouped into [[Environment Modules|buckets]]; a bucket has to be loaded before you can load anything from it. The Intel toolchain follows the same yearly generations as <code>foss</code>:

{| class="wikitable"
! Bucket !! Toolchain !! Contains
|-
| <code>2023</code> || <code>intel/2023a</code> || Intel compilers, Intel MPI, MKL
|-
| <code>2024</code> || <code>intel/2024a</code> || Intel compilers, Intel MPI, MKL
|-
| <code>2025</code> || <code>intel/2025a</code> || Intel compilers, Intel MPI, MKL
|}

<code>intel/<year>a</code> is the counterpart of <code>foss/<year>a</code>: one module that brings the whole stack — compilers, MPI, MKL and FFTW. There are leaner ways in, mirroring <code>gompi</code> on the <code>foss</code> side:

* '''<code>iimpi/<year>a</code>''' — Intel compilers and Intel MPI, without MKL. The lean choice when you only need MPI.
* '''<code>impi/<version></code>''' — the MPI library alone. In the 2025 bucket this is <code>impi/2021.15.0-intel-compilers-2025.1.1</code>; note that Intel MPI's own version numbering (2021.x) does not track the toolchain year.

To see what a bucket actually offers, ask the module system — <code>module key</code> reads Lmod's cache, so it needs no bucket loaded and searches every bucket at once:

<syntaxhighlight lang="bash">
module key intel # the toolchains
module key impi # the MPI library on its own
</syntaxhighlight>

=== Loading it ===

Bucket first, then the module — the same two steps as any other software here:

<syntaxhighlight lang="bash">
module load 2024 intel/2024a
</syntaxhighlight>

Then confirm what you got, which is worth the habit:

<syntaxhighlight lang="bash">
module list # everything loaded, dependencies included
mpirun -version # which Intel MPI is on your PATH
</syntaxhighlight>

If your software has its own module, load '''that''' instead and let it pull the matching MPI in as a dependency. A package built with this toolchain has a name ending in <code>-intel-2024a</code> or similar, which tells you what you are getting:

<syntaxhighlight lang="bash">
module load 2024
module load <YourSoftware>/<version> # Intel MPI arrives as a dependency
</syntaxhighlight>

Keep a job inside one bucket. Do not load <code>intel</code> and <code>foss</code> in the same job — they provide competing compilers, MPI libraries and BLAS, and the result is unpredictable rather than merely slow. If you want a clean starting point, <code>module reset</code> returns you to the system defaults; the <code>slurm</code> module is ''sticky'' and survives it either way.

== Running on a single node ==

Nothing runs on the login nodes. Put the work in a script, keep data on Lustre under <code>$myScratch</code>, and submit with <code>sbatch</code>.

<syntaxhighlight lang="bash">
#!/bin/bash -l
#SBATCH --job-name=impi_single
#SBATCH --nodes=1
#SBATCH --ntasks=64
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G # per CORE, not per node
#SBATCH --time=01:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 intel/2024a

mpirun -np ${SLURM_NTASKS} ./my_mpi_program --input data.nc
</syntaxhighlight>

Two things carried over from [[OpenMPI]], for the same reasons: pass <code>${SLURM_NTASKS}</code> rather than hard-coding the rank count, and use <code>--mem-per-cpu</code> rather than <code>--mem</code>, which is a per-''node'' request divided among all the ranks on that node.

Note the <code>#!/bin/bash -l</code>. A login shell is what initialises Lmod, so without it <code>module load</code> can fail quietly. For the same reason, never launch an MPI job with <code>sbatch --wrap</code> — that runs under <code>dash</code>, where <code>module</code> does not exist.

On one node the ranks talk through shared memory and there is nothing to configure.

== Running across several nodes ==

Anunna's fabric is '''Omni-Path (OPA100)''', and both MPI implementations reach it through '''libfabric'''. The provider that works reliably here is '''<code>psm2</code>'''; the <code>opx</code> provider is broken for inter-node traffic on this hardware. So an inter-node job should name the provider rather than trust auto-selection:

<syntaxhighlight lang="bash">
#!/bin/bash -l
#SBATCH --job-name=impi_multi
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=64
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=2G
#SBATCH --time=02:00:00
#SBATCH --output=%x-%j.out

cd "$myScratch/my_simulation"

module load 2024 intel/2024a

mpirun -genv I_MPI_OFI_PROVIDER psm2 -genv FI_PROVIDER psm2 \
-np ${SLURM_NTASKS} ./my_mpi_program --input data.nc
</syntaxhighlight>

To see what was actually selected — provider, pinning, rank layout — turn up Intel MPI's own diagnostics:

<syntaxhighlight lang="bash">
export I_MPI_DEBUG=4
</syntaxhighlight>

That prints the chosen fabric and the rank-to-core map at start-up, and is the fastest way to confirm a setting took effect instead of assuming it did.

=== Getting variables to the other nodes ===

Ranks on remote nodes are started fresh, so any variable your program or the fabric depends on has to reach them. Intel MPI's controls are:

{| class="wikitable"
! Option !! Effect
|-
| <code>-genv <VAR> <value></code> || Set one variable for '''all''' ranks. The explicit, always-safe form.
|-
| <code>-genvall</code> || Pass the whole launching environment to all ranks. This is the default.
|-
| <code>-genvlist <a,b,c></code> || Pass only the named variables.
|-
| <code>-genvnone</code> || Pass nothing.
|}

In practice '''a plain <code>export</code> in your job script does reach every rank''' — tested on Anunna across two nodes.

It arrives whatever you do to stop it: restricting Intel MPI's own propagation with <code>-genvlist PATH</code>, which should have excluded the test variable, delivered it to both nodes anyway. [[OpenMPI]] 5.0.7 behaved identically — with <code>-x</code>, without it, and even with Slurm's own environment forwarding suppressed.

The practical reading is that '''inside a Slurm allocation both implementations get your environment to the ranks''', by way of the process-management layer (PMIx and the launcher's job description) rather than the batch environment. On the OpenMPI side that was pinned down: removing the variable from <code>mpirun</code>'s own environment is the only thing that stopped it arriving — see [[OpenMPI#Exporting variables to the other nodes: the -x flag|the OpenMPI page]] for the full result.

None of which is worth depending on in a job script. It held for the current toolchains and not necessarily for older ones, so say what your run needs explicitly with <code>-genv</code> and the question stops mattering.

'''<code>-genv</code> is still the habit worth keeping''' for anything the run depends on, as in the script above. It does not rely on which layer happens to be propagating, it states the intent in the launch line, it survives a site or script that restricts propagation later, and it is the first thing to try if a variable does mysteriously fail to arrive.

== Intel MPI, MKL, and our AMD nodes ==

Most of Anunna's compute is AMD, and the Intel stack has a well-known wrinkle there. It is worth being precise about where it lives: '''the issue is in MKL, not in Intel MPI.''' But MKL arrives with the same <code>intel/<year>a</code> module, and "our Intel build is slow on the AMD nodes" is usually this, so it belongs on this page.

MKL checks the '''CPU vendor''' at run time. On a non-Intel CPU, kernels without specific Zen coverage can fall back to an '''SSE''' code path instead of using '''AVX2''', leaving much of the vector width unused. This is still the behaviour in MKL 2025.x — it has not been quietly fixed. The old <code>MKL_DEBUG_CPU_TYPE</code> workaround is gone, removed back in MKL 2020 Update 1, so anything you read recommending it is out of date.

The current mitigation is a small shim, '''<code>libfakeintel.so</code>''', <code>LD_PRELOAD</code>ed ahead of MKL. It overrides the vendor test (<code>mkl_serv_intel_cpu_true</code>) so that it returns true, and MKL then dispatches its AVX2 kernels:

<syntaxhighlight lang="bash">
export LD_PRELOAD=/path/to/libfakeintel.so
mpirun -np ${SLURM_NTASKS} ./my_mpi_program
</syntaxhighlight>

Before assuming this is your problem, two things are worth checking, because often it is not. The shim only ''widens arithmetic'', so it pays off when MKL is doing '''compute-bound Level-3 work''' such as <code>DGEMM</code>. It does nothing when MKL is only handling bandwidth-bound Level-1/2 vector kernels, and nothing at all if your library does its own arithmetic — PETSc, for instance, performs its sparse matrix–vector products itself, so MKL never sees the dominant cost of a Krylov solve.

<code>MKL_VERBOSE=1</code> is the quick way to find out: it prints every MKL call, so you can see whether MKL is on your hot path at all and what it is being asked to do. Keep it off any run you intend to time — at high call counts the logging is itself a real cost.

If MKL turns out to suit your workload poorly on the AMD nodes, the <code>foss</code> toolchain's OpenBLAS-based BLAS is worth benchmarking against.

== Advanced: pinning and rank placement ==

Intel MPI and OpenMPI express placement differently: OpenMPI takes command-line flags, Intel MPI reads <code>I_MPI_*</code> environment variables. The intents map across cleanly, so if you know one dialect this table gives you the other:

{| class="wikitable"
! Intent !! Intel MPI !! OpenMPI
|-
| One rank per physical core, no migration || <code>I_MPI_PIN=1</code>, <code>I_MPI_PIN_DOMAIN=core</code> || <code>--bind-to core</code>
|-
| Fill the node, neighbours adjacent || <code>I_MPI_PIN_ORDER=compact</code> || <code>--map-by core</code>
|-
| Under-subscribed, spread across all NUMA domains || <code>I_MPI_PIN_ORDER=scatter</code> || <code>--map-by numa --bind-to core</code>
|-
| One rank per chiplet / L3 slice || <code>I_MPI_PIN_DOMAIN=cache3</code> || <code>--map-by l3cache --bind-to core</code>
|-
| Print the map actually used || <code>I_MPI_DEBUG=4</code> || <code>--report-bindings</code>
|}

Three rules that matter more than the individual settings:

* '''Verify, do not assume.''' Print the map with <code>I_MPI_DEBUG=4</code> before and after any change. Placement bugs do not announce themselves; the job runs and is simply slower than it should be.
* '''One source of truth.''' Pin with the MPI launcher '''or''' with Slurm (<code>--cpu-bind</code>, <code>--distribution</code>), never both. Two pinners fighting each other produce a nonsense map.
* '''Spread suits memory-bound work.''' AMD sockets are built from 8-core chiplets, each with its own cache slice and share of the memory channels. A bandwidth-bound job saturates memory well before every core is busy, so spreading a reduced number of ranks across all NUMA domains keeps every memory channel active, while packing them together leaves most idle. Asking for ranks in multiples of 8 keeps chiplets evenly filled.

For the reasoning behind that last point, and the equivalent OpenMPI syntax, see [[OpenMPI#Advanced: pinning and rank placement|the OpenMPI page]].

== See also ==

* [[OpenMPI]] — the other MPI here, and the default for most software.
* [[Workflows/Multi-Process|Multi-Process Workflows]] — whether MPI is the right shape for your work.
* [[Environment Modules]] — buckets, and finding the module you need.
* [[Compute Hardware Overview]] — what the nodes actually are.
* [[Batch Jobs]] — writing and submitting job scripts.
* [[Scheduler Overview (Slurm)]] — how the scheduler allocates what you ask for.