HPCwiki - Recent changes [en]

Main Page

Haars0011 — Thu, 16 Apr 2026 07:43:12 GMT

← Older revision		Revision as of 09:43, 16 April 2026
Line 21:		Line 21:
	= Using Anunna =		= Using Anunna =
	* [[Tariffs \|Costs associated with resource usage]]		* [[Tariffs \|Costs associated with resource usage]]

			== Getting support ==
	* [[Support\|Get support with using Anunna (open a ticket with the HPC team)]]		* [[Support\|Get support with using Anunna (open a ticket with the HPC team)]]

Support

Prins0891 — Tue, 14 Apr 2026 14:15:08 GMT

← Older revision		Revision as of 16:15, 14 April 2026
(One intermediate revision by the same user not shown)
Line 1:		Line 1:
	This page explains how to request support from the HPC team for your work on Anunna.		This page explains how to request support from the HPC team for your work on Anunna.

	== How to ~~Request Support~~ ==		== How to request support ==

	=== WUR ~~Internal Users~~ ===		=== WUR internal users ===
	If you are a WUR user, please submit a support ticket via:		If you are a WUR user, please submit a support ticket via:

	* [https://support.wur.nl/esc WUR ~~Support Portal~~]		* [https://support.wur.nl/esc WUR support portal]
	* Select the 'service': '''HPC (Anunna)'''		* Select the 'service': '''HPC (Anunna)'''

	=== External ~~Users~~ ===		=== External users ===
	If you do not have a WUR account, please instead send an email to:		If you do not have a WUR account, please instead send an email to:

Line 15:		Line 15:
	* Include '''"HPC"''' in the subject line		* Include '''"HPC"''' in the subject line

	~~----~~		== What to include in your request ==

	== What to ~~Include~~ in ~~Your Request~~ ==
	To help us resolve your issue as quickly as possible, please provide a clear and complete description of your problem.		To help us resolve your issue as quickly as possible, please provide a clear and complete description of your problem.

Main Page

Prins0891 — Tue, 14 Apr 2026 12:50:58 GMT

Add link to support page

← Older revision		Revision as of 14:50, 14 April 2026
Line 20:		Line 20:

	= Using Anunna =		= Using Anunna =
	* [[Tariffs \| Costs associated with resource usage]]		* [[Tariffs \|Costs associated with resource usage]]
			* [[Support\|Get support with using Anunna (open a ticket with the HPC team)]]

	== Gaining access to Anunna==		== Gaining access to Anunna==

Support

Prins0891 — Tue, 14 Apr 2026 12:48:38 GMT

Support page written

← Older revision		Revision as of 14:48, 14 April 2026
Line 1:		Line 1:
	Support		This page explains how to request support from the HPC team for your work on Anunna.

			== How to Request Support ==

			=== WUR Internal Users ===
			If you are a WUR user, please submit a support ticket via:

			* [https://support.wur.nl/esc WUR Support Portal]
			* Select the 'service': '''HPC (Anunna)'''

			=== External Users ===
			If you do not have a WUR account, please instead send an email to:

			* '''[mailto:servicedesk.it@wur.nl?subject=HPC%20(Anunna) servicedesk.it@wur.nl]'''
			* Include '''"HPC"''' in the subject line

			----

			== What to Include in Your Request ==
			To help us resolve your issue as quickly as possible, please provide a clear and complete description of your problem.

			Include at least:

			* A '''concise description''' of the issue
			* The '''steps you performed''' before the problem occurred
			* Any '''error messages''' (copy/paste preferred)
			* Relevant '''job IDs''', filenames, or commands used
			* The '''time''' the issue occurred. Especially helpful for failed login attempts etc.

			The more precise your report, the faster we can assist you.

Support

Prins0891 — Tue, 14 Apr 2026 12:44:26 GMT

Created page with "Support"

New page

Support

SACCT

Honfi001 — Tue, 14 Apr 2026 05:33:28 GMT

← Older revision		Revision as of 07:33, 14 April 2026
(One intermediate revision by the same user not shown)
Line 1:		Line 1:
	The command [http://slurm.schedmd.com/sacct.html sacct] is meant to show your ~~costs from~~ Anunna.		The command [http://slurm.schedmd.com/sacct.html sacct] is meant to show your usage of Anunna.

	For instance at the login screen the command: sacct -a gives the following result:		For instance at the login screen the command: sacct -a gives the following result:

	~~'''[test001~~@~~nfs01~~ ~]$ sacct -a~~'''~~		<syntaxhighlight lang="bash">
			user001@login200:~$ sacct -a
	[[File:sacct capture.jpg]]		</syntaxhighlight>[[File:sacct capture.jpg]]

	It is a standard view to see all the users jobs of that day.		It is a standard view to see all the users jobs of that day.
	Better is to modify this command with special parameters to see what you are using from Anunna.		Better is to modify this command with special parameters to see what you are using from Anunna.<syntaxhighlight lang="bash">
			user001@login200:~$ sacct -P -X --delimiter=',' -S 2026-01-01 -E 2026-02-01 \
	~~'''[test001~~@~~nfs01~~ ~]$ sacct -X --format=comment%15,User,Partition%20,JobID,JobName,ncpus,nnodes,NodeList,Start,alloccpus,cputime%12,cputimeraw,state -~~S 03.01.14 -E 04.01.14 >report20140304~~.~~txt'''~~		--format=comment%15,User,Partition%20,JobID,JobName,ncpus,nnodes,NodeList,Start,alloccpus,cputime%12,cputimeraw,state > usage_report_$(date -I).csv

	The ~~txt~~-file can be uploaded in Excel. This gives you the next overview.		</syntaxhighlight>The csv-file can be uploaded in Excel. This gives you the next overview.
	Cputimeraw is the time of cpu-usage in seconds. The parameters "-S ~~03.~~01~~.14~~ -E ~~04.~~01~~.14~~" give you the start date and the end date of period you want to present.		Cputimeraw is the time of cpu-usage in seconds. The parameters "-S 2026-01-01 -E 2026-02-01" give you the start date and the end date of period you want to present.

	[[File:sacct uitgebreid.jpg]]		[[File:sacct uitgebreid.jpg]]

Maintenance and Management

Itter003 — Mon, 13 Apr 2026 12:57:07 GMT

← Older revision		Revision as of 14:57, 13 April 2026
Line 1:		Line 1:
	== Maintenance and Management ==		== Maintenance and Management ==

	The cluster is maintained for firmware and software updates on a six-monthly basis. Typically one downtime is scheduled in May, whilst the other is scheduled in October, in order to disrupt usage as little as possible.		The cluster is maintained for firmware and software updates on a six-monthly basis. Typically one downtime is scheduled in April/May in a week that is free of education, whilst the other is scheduled in October, in order to disrupt usage as little as possible.

	Any issues should be directed to us through [https://support.wur.nl/ WUR's Support portal]		Any issues should be directed to us through [https://support.wur.nl/ WUR's Support portal]

Linux desktop

Haars0011 — Fri, 03 Apr 2026 12:29:25 GMT

Chappie help

Show changes

File:Desktop extended form.png

Haars0011 — Fri, 27 Mar 2026 14:39:06 GMT

Haars0011 uploaded File:Desktop extended form.png

New page

desktop_extended_form

File:Minimal desktop form.png

Haars0011 — Fri, 27 Mar 2026 14:37:47 GMT

Haars0011 uploaded File:Minimal desktop form.png

New page

minimal_desktop_form

Using a hardware key for better security

Haars0011 — Wed, 25 Mar 2026 13:16:13 GMT

Show changes

Tutorials

Honfi001 — Wed, 25 Mar 2026 10:37:06 GMT

Apptainer

← Older revision		Revision as of 12:37, 25 March 2026
(One intermediate revision by the same user not shown)
Line 6:		Line 6:
	=== Apptainer ===		=== Apptainer ===

			* [[Tutorials/Apptainer-Introduction\|Introduction]]
	* [[Tutorials/Apptainer-PullingImages\|Pulling Images]]		* [[Tutorials/Apptainer-PullingImages\|Pulling Images]]
	* [[Tutorials/Apptainer-FakerootAndOverlays\|Fakeroot and overlays]]		* [[Tutorials/Apptainer-FakerootAndOverlays\|Fakeroot and overlays]]
	* [[Tutorials/Apptainer-FakerootAndSandbox\|Fakeroot and sandbox]]		* [[Tutorials/Apptainer-FakerootAndSandbox\|Fakeroot and sandbox]]
	* [[Tutorials/Apptainer-DefinitionFiles\|Definition Files]]		* [[Tutorials/Apptainer-DefinitionFiles\|Definition Files]]
			* [[Tutorials/Apptainer-GPUs\|GPUs]]

Tutorials/Apptainer-GPUs

Honfi001 — Wed, 25 Mar 2026 10:33:38 GMT

← Older revision		Revision as of 12:33, 25 March 2026
Line 6:		Line 6:

	* Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scratch space), not in your home directory.		* Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scratch space), not in your home directory.
	* Set your Apptainer cache to Lustre:		* Set your Apptainer cache to Lustre, since these images can get quite large:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
Line 40:		Line 40:
	Create a file called <code>whisper_nvidia.def</code>:		Create a file called <code>whisper_nvidia.def</code>:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">Bootstrap: docker
	Bootstrap: docker
	From: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel		From: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel

Line 54:		Line 53:

	%runscript		%runscript
	exec whisper "$@"		exec whisper --device cuda"$@"

	%help		%help
	Whisper speech recognition container (NVIDIA GPU).		Whisper speech recognition container (NVIDIA GPU).
	Usage: apptainer run --nv whisper_nvidia.sif <audio_file> [options]		Usage: apptainer run --nv whisper_nvidia.sif <audio_file> [options]</syntaxhighlight>
	</syntaxhighlight>

	A quick walkthrough of what is happening here:		A quick walkthrough of what is happening here:
Line 101:		Line 99:
	module reset		module reset
	module load utilities Apptainer		module load utilities Apptainer
	apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --~~device cuda~~		apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base
			apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model large

	</syntaxhighlight>		</syntaxhighlight>

Line 136:		Line 136:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	Bootstrap: docker		Bootstrap: docker
	From: rocm/pytorch:~~rocm6~~.~~2.4_ubuntu22~~.04_py3.~~10_pytorch_release_2~~.5.0		From: rocm/pytorch:rocm7.2_ubuntu24.04_py3.13_pytorch_release_2.10.0


	%post		%post
Line 148:		Line 149:

	%runscript		%runscript
	exec whisper "$@"		exec whisper --device cuda "$@"

	%help		%help
	Whisper speech recognition container (AMD ROCm GPU).		Whisper speech recognition container (AMD ROCm GPU).
	Usage: apptainer run --rocm whisper_amd.sif <audio_file> [options]		Usage: apptainer run --rocm whisper_amd.sif <audio_file> [options] --model <model_size>
	</syntaxhighlight>		</syntaxhighlight>

Line 175:		Line 176:
	Then load Apptainer and run:		Then load Apptainer and run:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">module reset
	module reset
	module load utilities Apptainer		module load utilities Apptainer
	apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --~~device cuda~~		apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base
	</syntaxhighlight>		apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model large </syntaxhighlight>

	Note that even though we are on an AMD GPU, the <code>--device cuda</code> flag is correct. PyTorch's ROCm backend uses the same <code>cuda</code> device name for compatibility.		Note that even though we are on an AMD GPU, the <code>--device cuda</code> flag is correct. PyTorch's ROCm backend uses the same <code>cuda</code> device name for compatibility.

Tutorials/Apptainer-GPUs

Honfi001 — Wed, 25 Mar 2026 09:42:06 GMT

Created page with "= Running Apptainer with GPUs = Apptainer can pass through GPU hardware from the host into a container, allowing you to run GPU-accelerated workloads (such as deep learning inference or training) inside a fully contained environment. This page covers how to use both NVIDIA and AMD GPUs on the Anunna cluster. '''Important:''' Before you begin, make sure the following are in place: * Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scrat..."

New page

= Running Apptainer with GPUs =

Apptainer can pass through GPU hardware from the host into a container, allowing you to run GPU-accelerated workloads (such as deep learning inference or training) inside a fully contained environment. This page covers how to use both NVIDIA and AMD GPUs on the Anunna cluster.

'''Important:''' Before you begin, make sure the following are in place:

* Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scratch space), not in your home directory.
* Set your Apptainer cache to Lustre:

<syntaxhighlight lang="bash">
export APPTAINER_CACHEDIR=$myScratch/Apptainer
</syntaxhighlight>

== How GPU Passthrough Works ==

Apptainer does not include GPU drivers inside the container. Instead, it '''binds''' the GPU drivers and libraries from the host system into the container at runtime. This means:

* The container must include software built for the correct GPU framework (CUDA for NVIDIA, ROCm for AMD).
* The host must have the matching GPU drivers installed (which Anunna already has on the GPU nodes).
* You tell Apptainer to enable GPU access using a flag: <code>--nv</code> for NVIDIA or <code>--rocm</code> for AMD.

== NVIDIA GPUs (<code>gpu</code> partition) ==

To use NVIDIA GPUs, you need to:

# Request a node on the <code>gpu</code> partition.
# Use the <code>--nv</code> flag when running your container.

The <code>--nv</code> flag tells Apptainer to:
* Make the <code>/dev/nvidiaX</code> device entries available inside the container.
* Locate and bind the CUDA libraries from the host into the container.
* Set <code>LD_LIBRARY_PATH</code> so the container uses the host's GPU libraries.

=== Example: Transcribing Audio with Whisper (NVIDIA) ===

[https://github.com/openai/whisper OpenAI Whisper] is a speech recognition model that benefits greatly from GPU acceleration. Let's build a container that runs Whisper on an NVIDIA GPU.

==== The Definition File ====

Create a file called <code>whisper_nvidia.def</code>:

<syntaxhighlight lang="bash">
Bootstrap: docker
From: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel

%post
apt-get update
apt-get install -y ffmpeg git
pip install openai-whisper
apt-get clean

%environment
export LC_ALL=C

%runscript
exec whisper "$@"

%help
Whisper speech recognition container (NVIDIA GPU).
Usage: apptainer run --nv whisper_nvidia.sif <audio_file> [options]
</syntaxhighlight>

A quick walkthrough of what is happening here:

{| class="wikitable"
|-
! Section !! Purpose
|-
| <code>Bootstrap / From</code> || Uses the official PyTorch Docker image, which already includes Python, PyTorch, CUDA, and cuDNN.
|-
| <code>%post</code> || Installs <code>ffmpeg</code> (required by Whisper for audio decoding), <code>git</code>, and the Whisper package itself.
|-
| <code>%environment</code> || Sets the locale to avoid encoding warnings.
|-
| <code>%runscript</code> || Makes <code>whisper</code> the default command, passing through any arguments.
|-
| <code>%help</code> || Provides usage information (accessible via <code>apptainer run-help</code>).
|}

==== Building the Image ====

Request a compute node and build:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer build whisper_nvidia.sif whisper_nvidia.def
</syntaxhighlight>

==== Running Whisper ====

Once built, run Whisper on the sample audio file. First, request a GPU node:

<syntaxhighlight lang="bash">
srun --partition=gpu --gres=gpu:1 --pty bash
</syntaxhighlight>

Then load Apptainer and run:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --device cuda
</syntaxhighlight>

The <code>--nv</code> flag is what makes the GPU visible to the container. Without it, PyTorch would not detect any CUDA devices and Whisper would fall back to CPU (much slower).

You can verify GPU access from inside the container with:

<syntaxhighlight lang="bash">
apptainer exec --nv whisper_nvidia.sif python -c "import torch; print(torch.cuda.is_available())"
</syntaxhighlight>

This should print <code>True</code>.

== AMD GPUs (<code>gpu_amd</code> partition) ==

To use AMD GPUs, you need to:

# Request a node on the <code>gpu_amd</code> partition.
# Use the <code>--rocm</code> flag when running your container.

The <code>--rocm</code> flag tells Apptainer to:
* Make the <code>/dev/dri/</code> and <code>/dev/kfd</code> device entries available inside the container.
* Locate and bind the ROCm libraries from the host into the container.
* Set <code>LD_LIBRARY_PATH</code> so the container uses the host's ROCm libraries.

=== Example: Transcribing Audio with Whisper (AMD) ===

The same Whisper workflow, but using an AMD GPU with the ROCm stack.

==== The Definition File ====

Create a file called <code>whisper_amd.def</code>:

<syntaxhighlight lang="bash">
Bootstrap: docker
From: rocm/pytorch:rocm6.2.4_ubuntu22.04_py3.10_pytorch_release_2.5.0

%post
apt-get update
apt-get install -y ffmpeg git
pip install openai-whisper
apt-get clean

%environment
export LC_ALL=C

%runscript
exec whisper "$@"

%help
Whisper speech recognition container (AMD ROCm GPU).
Usage: apptainer run --rocm whisper_amd.sif <audio_file> [options]
</syntaxhighlight>

The structure is identical to the NVIDIA version. The only difference is the base image: instead of <code>pytorch/pytorch</code> (which includes CUDA), we use <code>rocm/pytorch</code> (which includes ROCm). PyTorch's API is the same regardless of the backend — <code>torch.cuda.is_available()</code> returns <code>True</code> on ROCm as well, since ROCm maps onto the CUDA API.

==== Building the Image ====

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer build whisper_amd.sif whisper_amd.def
</syntaxhighlight>

==== Running Whisper ====

Request an AMD GPU node:

<syntaxhighlight lang="bash">
srun --partition=gpu_amd --gres=gpu:1 --pty bash
</syntaxhighlight>

Then load Apptainer and run:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --device cuda
</syntaxhighlight>

Note that even though we are on an AMD GPU, the <code>--device cuda</code> flag is correct. PyTorch's ROCm backend uses the same <code>cuda</code> device name for compatibility.

Verify GPU access:

<syntaxhighlight lang="bash">
apptainer exec --rocm whisper_amd.sif python -c "import torch; print(torch.cuda.is_available())"
</syntaxhighlight>

== Summary ==

{| class="wikitable"
|-
! !! NVIDIA !! AMD
|-
| '''Partition''' || <code>gpu</code> || <code>gpu_amd</code>
|-
| '''Apptainer flag''' || <code>--nv</code> || <code>--rocm</code>
|-
| '''Base image''' || <code>pytorch/pytorch</code> (includes CUDA) || <code>rocm/pytorch</code> (includes ROCm)
|-
| '''PyTorch device''' || <code>--device cuda</code> || <code>--device cuda</code> (same API)
|-
| '''Host devices bound''' || <code>/dev/nvidiaX</code> || <code>/dev/dri/</code>, <code>/dev/kfd</code>
|}

The key takeaway: the only things that change between NVIDIA and AMD are the '''base container image''', the '''Apptainer flag''', and the '''Slurm partition'''. Your application code stays the same.

Tutorials/Apptainer-Introduction

Honfi001 — Wed, 25 Mar 2026 09:23:54 GMT

apptainer pull

← Older revision		Revision as of 11:23, 25 March 2026
Line 28:		Line 28:
	'''Portability:''' An Apptainer container is a single <code>.sif</code> file. You can build it on your laptop, copy it to the cluster, share it with a collaborator, or archive it alongside a publication. It will run the same way everywhere Apptainer is installed.		'''Portability:''' An Apptainer container is a single <code>.sif</code> file. You can build it on your laptop, copy it to the cluster, share it with a collaborator, or archive it alongside a publication. It will run the same way everywhere Apptainer is installed.

	'''Reproducibility:''' A container captures the exact software environment used for an analysis — down to the specific library versions. This means an experiment can be repeated months or years later with the same ~~results~~, even if the host system has changed. When combined with a definition file (a text recipe that describes how the container was built), every step is documented and reproducible.		'''Reproducibility:''' A container captures the exact software environment used for an analysis — down to the specific library versions. This means an experiment can be repeated months or years later with the same environment, even if the host system has changed. When combined with a definition file (a text recipe that describes how the container was built), every step is documented and reproducible.

	== Loading Apptainer ==		== Loading Apptainer ==
Line 36:		Line 36:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	module reset		module reset
	module load utilities Apptainer		module load utilities
			module load Apptainer
	</syntaxhighlight>		</syntaxhighlight>

Line 66:		Line 67:
	=== Do Not Store Images in Your Home Directory ===		=== Do Not Store Images in Your Home Directory ===

	SIF files can easily be hundreds of megabytes or even several gigabytes in size. Store them on Lustre (e.g. in your scratch or project space) rather than in your home directory to avoid quota issues.		SIF files can easily be hundreds of megabytes or even several gigabytes in size. Store them on Lustre (e.g. in your scratch or project space) rather than in your home directory to avoid quota issues. Running images can be I/O intensive and this is not suitable for the home directory.

	== Basic Commands ==		== Basic Commands ==
Line 77:		Line 78:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	apptainer pull ubuntu.sif docker://ubuntu:24.04		apptainer pull ubuntu.sif docker://ubuntu:18.04
	</syntaxhighlight>		</syntaxhighlight>

Tutorials/Apptainer-Introduction

Honfi001 — Wed, 25 Mar 2026 09:18:12 GMT

Created page with "= Introduction to Apptainer = == What is Apptainer? == Apptainer (formerly known as Singularity) is a container platform designed for High Performance Computing (HPC) environments. If you have heard of Docker, Apptainer solves a similar problem — it lets you package an application together with all of its dependencies (libraries, tools, configuration) into a single portable unit called a '''container'''. The key difference is that Apptainer was built from the ground..."

New page

= Introduction to Apptainer =

== What is Apptainer? ==

Apptainer (formerly known as Singularity) is a container platform designed for High Performance Computing (HPC) environments. If you have heard of Docker, Apptainer solves a similar problem — it lets you package an application together with all of its dependencies (libraries, tools, configuration) into a single portable unit called a '''container'''. The key difference is that Apptainer was built from the ground up to run safely on shared multi-user systems like HPC clusters, where Docker cannot be used because it requires root privileges that would pose a security risk.

Some important differences between Apptainer and Docker:

{| class="wikitable"
|-
! !! Apptainer !! Docker
|-
| '''Privileges''' || Runs as a normal user — no root access needed || Requires root or membership in the <code>docker</code> group, which is effectively root
|-
| '''Image format''' || Single portable <code>.sif</code> file || Layered images stored in a daemon's internal storage
|-
| '''Integration with host''' || Seamless access to host filesystems, GPUs, and network by default || Isolated by default; explicit configuration needed to share resources
|-
| '''Designed for''' || HPC, scientific computing, shared multi-user systems || Cloud, microservices, development environments
|-
| '''Docker compatibility''' || Can pull and run Docker images directly || N/A
|}

=== Why Apptainer for Research? ===

Two features make Apptainer particularly valuable for scientific work:

'''Portability:''' An Apptainer container is a single <code>.sif</code> file. You can build it on your laptop, copy it to the cluster, share it with a collaborator, or archive it alongside a publication. It will run the same way everywhere Apptainer is installed.

'''Reproducibility:''' A container captures the exact software environment used for an analysis — down to the specific library versions. This means an experiment can be repeated months or years later with the same results, even if the host system has changed. When combined with a definition file (a text recipe that describes how the container was built), every step is documented and reproducible.

== Loading Apptainer ==

On the Anunna HPC cluster, Apptainer is available in the <code>utilities</code> module bucket. Load it with:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
</syntaxhighlight>

You can verify it is loaded by running:

<syntaxhighlight lang="bash">
apptainer --version
</syntaxhighlight>

== Best Practices ==

=== Set Your Cache Directory ===

Apptainer downloads container layers to a local cache before building SIF files. By default this cache lives under your home directory (<code>$HOME/.apptainer/cache</code>), which can quickly exceed your home quota — especially with large images.

Change the cache location to your scratch space by setting the <code>APPTAINER_CACHEDIR</code> variable. You will need to create the directory first:

<syntaxhighlight lang="bash">
mkdir -p $myScratch/Apptainer
export APPTAINER_CACHEDIR=$myScratch/Apptainer
</syntaxhighlight>

Consider adding the <code>export</code> line to your <code>~/.bashrc</code> so it is set automatically in every session.

=== Do Not Run Apptainer on the Login Nodes ===

Building and running containers can be resource-intensive. Always request a compute node (e.g. via <code>srun</code>) before running Apptainer commands. The login nodes are shared and should not be used for heavy workloads.

=== Do Not Store Images in Your Home Directory ===

SIF files can easily be hundreds of megabytes or even several gigabytes in size. Store them on Lustre (e.g. in your scratch or project space) rather than in your home directory to avoid quota issues.

== Basic Commands ==

Apptainer has five core commands you will use regularly:

=== <code>apptainer pull</code> ===

Downloads a container image from a remote registry and saves it as a <code>.sif</code> file.

<syntaxhighlight lang="bash">
apptainer pull ubuntu.sif docker://ubuntu:24.04
</syntaxhighlight>

=== <code>apptainer build</code> ===

The build command is used for converting images from one format to another. It can create a SIF from a definition file, convert a sandbox to a SIF, or convert between formats.

<syntaxhighlight lang="bash">
# Build from a definition file
apptainer build myimage.sif myrecipe.def

# Convert a sandbox directory to a SIF
apptainer build myimage.sif /path/to/sandbox/
</syntaxhighlight>

=== <code>apptainer exec</code> ===

Runs a single specific command inside a container and returns the output.

<syntaxhighlight lang="bash">
apptainer exec ubuntu.sif cat /etc/os-release
</syntaxhighlight>

=== <code>apptainer shell</code> ===

Opens an interactive shell inside the container, allowing you to explore and run commands as if you had logged into a different machine.

<syntaxhighlight lang="bash">
apptainer shell ubuntu.sif
</syntaxhighlight>

=== <code>apptainer run</code> ===

Executes the container's default action (its '''runscript'''). What this does depends on how the container was built.

<syntaxhighlight lang="bash">
apptainer run myimage.sif
</syntaxhighlight>

=== At a Glance ===

{| class="wikitable"
|-
! Command !! Purpose
|-
| <code>apptainer pull</code> || Download an image from a registry
|-
| <code>apptainer build</code> || Build an image from a definition file or sandbox
|-
| <code>apptainer exec</code> || Run a single command inside a container
|-
| <code>apptainer shell</code> || Open an interactive shell inside a container
|-
| <code>apptainer run</code> || Run the container's default action
|}

Main Page

Honfi001 — Wed, 25 Mar 2026 06:56:50 GMT

Add tutorials section and edited the heading of the courses section

← Older revision		Revision as of 08:56, 25 March 2026
Line 10:		Line 10:
	Access needs to be granted actively (by creation of an account on the cluster by FB-IT). Use of resources is limited by the scheduler. Note that the use of Anunna is not free of charge.		Access needs to be granted actively (by creation of an account on the cluster by FB-IT). Use of resources is limited by the scheduler. Note that the use of Anunna is not free of charge.

	= Our Courses =		= Our Courses and Tutorials =
	The Anunna team organizes HPC courses three times a year to strengthen basic & more advanced skills and enable users to make the most effective use of our facility.		The Anunna team organizes HPC courses three times a year to strengthen basic & more advanced skills and enable users to make the most effective use of our facility.

	== [[Linux Basic]] ==		* [[Linux Basic]]
			* [[HPC Basic]]
	== [[HPC Basic]] ==		* [[HPC Advanced]]
			* [[2026 Course dates]]
	== [[HPC Advanced]] ==		* [[Tutorials]]

	== [[2026 Course dates]] ==

	= Using Anunna =		= Using Anunna =

Tutorials/Apptainer-FakerootAndOverlays

Honfi001 — Wed, 25 Mar 2026 06:54:45 GMT

@@ Line 84: / Line 84: @@
 Once this completes, neofetch is installed — but only in the overlay. The original <code>ubuntu_24.04.sif</code> is still untouched.
-== Building a New Image from the Original + Overlay ==
+The takeaway tis that the an overlay enables a read-only image to be "modified" at runtime. This enables the use of applications that dowlooad data, e.g. databases at runtime.
 == Cleaning Up ==
@@ Line 125: / Line 107: @@
 |-
 | Install software || <code>apptainer exec --fakeroot --overlay $ov ubuntu_24.04.sif apt install -y neofetch</code>
 |-
 | Clean up overlay || <code>rm -rf $ov</code>
 |}

← Older revision		Revision as of 12:37, 25 March 2026
(One intermediate revision by the same user not shown)
Line 6:		Line 6:
	=== Apptainer ===		=== Apptainer ===

			* [[Tutorials/Apptainer-Introduction\|Introduction]]
	* [[Tutorials/Apptainer-PullingImages\|Pulling Images]]		* [[Tutorials/Apptainer-PullingImages\|Pulling Images]]
	* [[Tutorials/Apptainer-FakerootAndOverlays\|Fakeroot and overlays]]		* [[Tutorials/Apptainer-FakerootAndOverlays\|Fakeroot and overlays]]
	* [[Tutorials/Apptainer-FakerootAndSandbox\|Fakeroot and sandbox]]		* [[Tutorials/Apptainer-FakerootAndSandbox\|Fakeroot and sandbox]]
	* [[Tutorials/Apptainer-DefinitionFiles\|Definition Files]]		* [[Tutorials/Apptainer-DefinitionFiles\|Definition Files]]
			* [[Tutorials/Apptainer-GPUs\|GPUs]]