Tutorials/Apptainer-Introduction

From HPCwiki
Jump to navigation Jump to search

Introduction to Apptainer

What is Apptainer?

Apptainer (formerly known as Singularity) is a container platform designed for High Performance Computing (HPC) environments. If you have heard of Docker, Apptainer solves a similar problem — it lets you package an application together with all of its dependencies (libraries, tools, configuration) into a single portable unit called a container. The key difference is that Apptainer was built from the ground up to run safely on shared multi-user systems like HPC clusters, where Docker cannot be used because it requires root privileges that would pose a security risk.

Some important differences between Apptainer and Docker:

Apptainer Docker
Privileges Runs as a normal user — no root access needed Requires root or membership in the docker group, which is effectively root
Image format Single portable .sif file Layered images stored in a daemon's internal storage
Integration with host Seamless access to host filesystems, GPUs, and network by default Isolated by default; explicit configuration needed to share resources
Designed for HPC, scientific computing, shared multi-user systems Cloud, microservices, development environments
Docker compatibility Can pull and run Docker images directly N/A

Why Apptainer for Research?

Two features make Apptainer particularly valuable for scientific work:

Portability: An Apptainer container is a single .sif file. You can build it on your laptop, copy it to the cluster, share it with a collaborator, or archive it alongside a publication. It will run the same way everywhere Apptainer is installed.

Reproducibility: A container captures the exact software environment used for an analysis — down to the specific library versions. This means an experiment can be repeated months or years later with the same environment, even if the host system has changed. When combined with a definition file (a text recipe that describes how the container was built), every step is documented and reproducible.

Loading Apptainer

On the Anunna HPC cluster, Apptainer is available in the utilities module bucket. Load it with:

module reset
module load utilities
module load Apptainer

You can verify it is loaded by running:

apptainer --version

Best Practices

Set Your Cache Directory

Apptainer downloads container layers to a local cache before building SIF files. By default this cache lives under your home directory ($HOME/.apptainer/cache), which can quickly exceed your home quota — especially with large images.

Change the cache location to your scratch space by setting the APPTAINER_CACHEDIR variable. You will need to create the directory first:

mkdir -p $myScratch/Apptainer
export APPTAINER_CACHEDIR=$myScratch/Apptainer

Consider adding the export line to your ~/.bashrc so it is set automatically in every session.

Do Not Run Apptainer on the Login Nodes

Building and running containers can be resource-intensive. Always request a compute node (e.g. via srun) before running Apptainer commands. The login nodes are shared and should not be used for heavy workloads.

Do Not Store Images in Your Home Directory

SIF files can easily be hundreds of megabytes or even several gigabytes in size. Store them on Lustre (e.g. in your scratch or project space) rather than in your home directory to avoid quota issues. Running images can be I/O intensive and this is not suitable for the home directory.

Basic Commands

Apptainer has five core commands you will use regularly:

apptainer pull

Downloads a container image from a remote registry and saves it as a .sif file.

apptainer pull ubuntu.sif docker://ubuntu:18.04

apptainer build

The build command is used for converting images from one format to another. It can create a SIF from a definition file, convert a sandbox to a SIF, or convert between formats.

# Build from a definition file
apptainer build myimage.sif myrecipe.def

# Convert a sandbox directory to a SIF
apptainer build myimage.sif /path/to/sandbox/

apptainer exec

Runs a single specific command inside a container and returns the output.

apptainer exec ubuntu.sif cat /etc/os-release

apptainer shell

Opens an interactive shell inside the container, allowing you to explore and run commands as if you had logged into a different machine.

apptainer shell ubuntu.sif

apptainer run

Executes the container's default action (its runscript). What this does depends on how the container was built.

apptainer run myimage.sif

At a Glance

Command Purpose
apptainer pull Download an image from a registry
apptainer build Build an image from a definition file or sandbox
apptainer exec Run a single command inside a container
apptainer shell Open an interactive shell inside a container
apptainer run Run the container's default action