Honfi001 at 10:33, 25 March 2026

2026-03-25T10:33:38Z

← Older revision		Revision as of 12:33, 25 March 2026
Line 6:		Line 6:

	* Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scratch space), not in your home directory.		* Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scratch space), not in your home directory.
	* Set your Apptainer cache to Lustre:		* Set your Apptainer cache to Lustre, since these images can get quite large:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
Line 40:		Line 40:
	Create a file called <code>whisper_nvidia.def</code>:		Create a file called <code>whisper_nvidia.def</code>:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">Bootstrap: docker
	Bootstrap: docker
	From: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel		From: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel

Line 54:		Line 53:

	%runscript		%runscript
	exec whisper "$@"		exec whisper --device cuda"$@"

	%help		%help
	Whisper speech recognition container (NVIDIA GPU).		Whisper speech recognition container (NVIDIA GPU).
	Usage: apptainer run --nv whisper_nvidia.sif <audio_file> [options]		Usage: apptainer run --nv whisper_nvidia.sif <audio_file> [options]</syntaxhighlight>
	</syntaxhighlight>

	A quick walkthrough of what is happening here:		A quick walkthrough of what is happening here:
Line 101:		Line 99:
	module reset		module reset
	module load utilities Apptainer		module load utilities Apptainer
	apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --~~device cuda~~		apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base
			apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model large

	</syntaxhighlight>		</syntaxhighlight>

Line 136:		Line 136:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	Bootstrap: docker		Bootstrap: docker
	From: rocm/pytorch:~~rocm6~~.~~2.4_ubuntu22~~.04_py3.~~10_pytorch_release_2~~.5.0		From: rocm/pytorch:rocm7.2_ubuntu24.04_py3.13_pytorch_release_2.10.0


	%post		%post
Line 148:		Line 149:

	%runscript		%runscript
	exec whisper "$@"		exec whisper --device cuda "$@"

	%help		%help
	Whisper speech recognition container (AMD ROCm GPU).		Whisper speech recognition container (AMD ROCm GPU).
	Usage: apptainer run --rocm whisper_amd.sif <audio_file> [options]		Usage: apptainer run --rocm whisper_amd.sif <audio_file> [options] --model <model_size>
	</syntaxhighlight>		</syntaxhighlight>

Line 175:		Line 176:
	Then load Apptainer and run:		Then load Apptainer and run:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">module reset
	module reset
	module load utilities Apptainer		module load utilities Apptainer
	apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --~~device cuda~~		apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base
	</syntaxhighlight>		apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model large </syntaxhighlight>

	Note that even though we are on an AMD GPU, the <code>--device cuda</code> flag is correct. PyTorch's ROCm backend uses the same <code>cuda</code> device name for compatibility.		Note that even though we are on an AMD GPU, the <code>--device cuda</code> flag is correct. PyTorch's ROCm backend uses the same <code>cuda</code> device name for compatibility.

Honfi001: Created page with "= Running Apptainer with GPUs = Apptainer can pass through GPU hardware from the host into a container, allowing you to run GPU-accelerated workloads (such as deep learning inference or training) inside a fully contained environment. This page covers how to use both NVIDIA and AMD GPUs on the Anunna cluster. '''Important:''' Before you begin, make sure the following are in place: * Your `.sif` image files should be stored on '''Lustre''' (e.g. in your scrat..."

2026-03-25T09:42:06Z

Created page with "= Running Apptainer with GPUs = Apptainer can pass through GPU hardware from the host into a container, allowing you to run GPU-accelerated workloads (such as deep learning inference or training) inside a fully contained environment. This page covers how to use both NVIDIA and AMD GPUs on the Anunna cluster. '''Important:''' Before you begin, make sure the following are in place: * Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scrat..."

New page

= Running Apptainer with GPUs =

Apptainer can pass through GPU hardware from the host into a container, allowing you to run GPU-accelerated workloads (such as deep learning inference or training) inside a fully contained environment. This page covers how to use both NVIDIA and AMD GPUs on the Anunna cluster.

'''Important:''' Before you begin, make sure the following are in place:

* Your <code>.sif</code> image files should be stored on '''Lustre''' (e.g. in your scratch space), not in your home directory.
* Set your Apptainer cache to Lustre:

<syntaxhighlight lang="bash">
export APPTAINER_CACHEDIR=$myScratch/Apptainer
</syntaxhighlight>

== How GPU Passthrough Works ==

Apptainer does not include GPU drivers inside the container. Instead, it '''binds''' the GPU drivers and libraries from the host system into the container at runtime. This means:

* The container must include software built for the correct GPU framework (CUDA for NVIDIA, ROCm for AMD).
* The host must have the matching GPU drivers installed (which Anunna already has on the GPU nodes).
* You tell Apptainer to enable GPU access using a flag: <code>--nv</code> for NVIDIA or <code>--rocm</code> for AMD.

== NVIDIA GPUs (<code>gpu</code> partition) ==

To use NVIDIA GPUs, you need to:

# Request a node on the <code>gpu</code> partition.
# Use the <code>--nv</code> flag when running your container.

The <code>--nv</code> flag tells Apptainer to:
* Make the <code>/dev/nvidiaX</code> device entries available inside the container.
* Locate and bind the CUDA libraries from the host into the container.
* Set <code>LD_LIBRARY_PATH</code> so the container uses the host's GPU libraries.

=== Example: Transcribing Audio with Whisper (NVIDIA) ===

[https://github.com/openai/whisper OpenAI Whisper] is a speech recognition model that benefits greatly from GPU acceleration. Let's build a container that runs Whisper on an NVIDIA GPU.

==== The Definition File ====

Create a file called <code>whisper_nvidia.def</code>:

<syntaxhighlight lang="bash">
Bootstrap: docker
From: pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel

%post
apt-get update
apt-get install -y ffmpeg git
pip install openai-whisper
apt-get clean

%environment
export LC_ALL=C

%runscript
exec whisper "$@"

%help
Whisper speech recognition container (NVIDIA GPU).
Usage: apptainer run --nv whisper_nvidia.sif <audio_file> [options]
</syntaxhighlight>

A quick walkthrough of what is happening here:

{| class="wikitable"
|-
! Section !! Purpose
|-
| <code>Bootstrap / From</code> || Uses the official PyTorch Docker image, which already includes Python, PyTorch, CUDA, and cuDNN.
|-
| <code>%post</code> || Installs <code>ffmpeg</code> (required by Whisper for audio decoding), <code>git</code>, and the Whisper package itself.
|-
| <code>%environment</code> || Sets the locale to avoid encoding warnings.
|-
| <code>%runscript</code> || Makes <code>whisper</code> the default command, passing through any arguments.
|-
| <code>%help</code> || Provides usage information (accessible via <code>apptainer run-help</code>).
|}

==== Building the Image ====

Request a compute node and build:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer build whisper_nvidia.sif whisper_nvidia.def
</syntaxhighlight>

==== Running Whisper ====

Once built, run Whisper on the sample audio file. First, request a GPU node:

<syntaxhighlight lang="bash">
srun --partition=gpu --gres=gpu:1 --pty bash
</syntaxhighlight>

Then load Apptainer and run:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer run --nv whisper_nvidia.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --device cuda
</syntaxhighlight>

The <code>--nv</code> flag is what makes the GPU visible to the container. Without it, PyTorch would not detect any CUDA devices and Whisper would fall back to CPU (much slower).

You can verify GPU access from inside the container with:

<syntaxhighlight lang="bash">
apptainer exec --nv whisper_nvidia.sif python -c "import torch; print(torch.cuda.is_available())"
</syntaxhighlight>

This should print <code>True</code>.

== AMD GPUs (<code>gpu_amd</code> partition) ==

To use AMD GPUs, you need to:

# Request a node on the <code>gpu_amd</code> partition.
# Use the <code>--rocm</code> flag when running your container.

The <code>--rocm</code> flag tells Apptainer to:
* Make the <code>/dev/dri/</code> and <code>/dev/kfd</code> device entries available inside the container.
* Locate and bind the ROCm libraries from the host into the container.
* Set <code>LD_LIBRARY_PATH</code> so the container uses the host's ROCm libraries.

=== Example: Transcribing Audio with Whisper (AMD) ===

The same Whisper workflow, but using an AMD GPU with the ROCm stack.

==== The Definition File ====

Create a file called <code>whisper_amd.def</code>:

<syntaxhighlight lang="bash">
Bootstrap: docker
From: rocm/pytorch:rocm6.2.4_ubuntu22.04_py3.10_pytorch_release_2.5.0

%post
apt-get update
apt-get install -y ffmpeg git
pip install openai-whisper
apt-get clean

%environment
export LC_ALL=C

%runscript
exec whisper "$@"

%help
Whisper speech recognition container (AMD ROCm GPU).
Usage: apptainer run --rocm whisper_amd.sif <audio_file> [options]
</syntaxhighlight>

The structure is identical to the NVIDIA version. The only difference is the base image: instead of <code>pytorch/pytorch</code> (which includes CUDA), we use <code>rocm/pytorch</code> (which includes ROCm). PyTorch's API is the same regardless of the backend — <code>torch.cuda.is_available()</code> returns <code>True</code> on ROCm as well, since ROCm maps onto the CUDA API.

==== Building the Image ====

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer build whisper_amd.sif whisper_amd.def
</syntaxhighlight>

==== Running Whisper ====

Request an AMD GPU node:

<syntaxhighlight lang="bash">
srun --partition=gpu_amd --gres=gpu:1 --pty bash
</syntaxhighlight>

Then load Apptainer and run:

<syntaxhighlight lang="bash">
module reset
module load utilities Apptainer
apptainer run --rocm whisper_amd.sif /lustre/shared/hpcCourses/Whisper/audio.mp3 --model base --device cuda
</syntaxhighlight>

Note that even though we are on an AMD GPU, the <code>--device cuda</code> flag is correct. PyTorch's ROCm backend uses the same <code>cuda</code> device name for compatibility.

Verify GPU access:

<syntaxhighlight lang="bash">
apptainer exec --rocm whisper_amd.sif python -c "import torch; print(torch.cuda.is_available())"
</syntaxhighlight>

== Summary ==

{| class="wikitable"
|-
! !! NVIDIA !! AMD
|-
| '''Partition''' || <code>gpu</code> || <code>gpu_amd</code>
|-
| '''Apptainer flag''' || <code>--nv</code> || <code>--rocm</code>
|-
| '''Base image''' || <code>pytorch/pytorch</code> (includes CUDA) || <code>rocm/pytorch</code> (includes ROCm)
|-
| '''PyTorch device''' || <code>--device cuda</code> || <code>--device cuda</code> (same API)
|-
| '''Host devices bound''' || <code>/dev/nvidiaX</code> || <code>/dev/dri/</code>, <code>/dev/kfd</code>
|}

The key takeaway: the only things that change between NVIDIA and AMD are the '''base container image''', the '''Apptainer flag''', and the '''Slurm partition'''. Your application code stays the same.

Tutorials/Apptainer-GPUs - Revision history

Honfi001 at 10:33, 25 March 2026