Compute Hardware Overview
Anunna's compute nodes can be categorized into CPU compute nodes and GPU compute node, which are segregated into different partitions, according to the hardware they provide.
This page summarises provides an overview of the hardware available at each node type and provides information on how to select the appropriate hardware for your job.
CPU Compute Nodes
Standard compute nodes for jobs that do not need a GPU. These make up the main partition and handle the bulk of the cluster's work.
At the moment, there are three generations of nodes in the cluster. Each generation corresponds to a specific CPU architecture and it is recommended you do not mix them.
In order to select a specific CPU model, constrain a job to a specific model with, for example, --constraint='gen3'. (see Choosing a node (constraints)
| Nodes | Generation | Number of Nodes | CPU Maker | CPU Architecture | Sockets | Cores per Socket | Cores per Node | RAM (GB) | contraint |
|---|---|---|---|---|---|---|---|---|---|
| node1[00 - 28] | 2 | 29 | Intel | Skylake | 2 | 16 | 32 | 375 | gen2 |
| node2[00 - 72] | 3 | 73 | AMD | Zen3 Milan | 2 | 64 | 128 | 1000 | gen3 |
| node30[0, 2, 3] | 4 | 3 | AMD | Zen5 Turin | 2 | 96 | 192 | 2250 | gen4 |
GPU Compute Nodes
Nodes equipped with GPUs, for accelerated for workloads such as rendering, AI and deep learning.
The GPUs are distributed along two different partition, gpu and gpu_amd for Nvidia e AMD gpus respectively.
In order to allocate a GPU for a job, first the appropriate partition needs to be selected. Afterwards, at least one gpu needs to be allocated by using the command
--gres=gpu:<N>
Where <N> corresponds to the number of GPUs allocated for the job. If more than one GPU is desired, please make sure that the number of nodes in your SLURM script or srun command is set to 1. Otherwise SLURM may try to allocate GPUs in different nodes which will slow down you job considerably.
NVIDIA GPU Nodes
Available in the gpu partition. The cluster has several NVIDIA GPU models e.g. A100, L40s, A6000, and V100. The scheduler attempts to hand out the L40s first, then A100s, then A6000s, then V100s; the price per GPU-hour is the same for all of them.
In order to select a specific GPU model, constrain a job to a specific model with, for example, --constraint='A100'. (see Choosing a node (constraints)
AMD GPU Nodes
Available in the gpu_amd partition. Requested the same way as NVIDIA GPUs, using the gpu_amd partition. Since there is only one AMD GPU model, constraints are not necessary.
| node | CPU Maker | Partition | CPU Architecture | Sockets | Cores per Socket | Cores per Node | RAM (GB) | GPU Maker | GPU Type | Number of GPUs | VRAM (GB) | VRAM Type | constraint |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| gpu100 | Intel | gpu | Skylake | 2 | 16 | 32 | 375 | Nvidia | V100 | 4 | 16 | HBM2 | V100 |
| gpun20[0 - 3] | Intel | gpu | Skylake | 2 | 16 | 32 | 500 | Nvidia | A100 | 4 | 80 | HBM2e | A100 |
| gpuxn200 | Intel | gpu | Sapphire Rapids | 2 | 32 | 64 | 500 | Nvidia | RTX A6000 | 2 | 48 | GDDR6 | A6000 |
| node301 | AMD | gpu | Zen5 Turin | 2 | 96 | 192 | 2250 | Nvidia | L40s | 4 | 48 | GDDR6 | L40S |
| gpua201 | Intel | gpu_amd | Skylake | 2 | 16 | 32 | 375 | AMD | MI210 | 4 | 64 | HBM2e | MI210 |
Available GPU Models
Anunna has a variety of GPU models and depending on the type of workflow some GPUs are going to be better suited then others. Here is a brief description of each GPU:
- NVIDIA V100 (16 GB HBM2) – Strong general-purpose training GPU for FP32/FP16 workloads, well-suited for classic deep learning training and HPC compute.
- NVIDIA A6000 (48 GB GDDR6) – High end workstation GPU optimized for for high-throughput inference and small- to mid-scale training, rendering and AI workloads.
- NVIDIA A100 (80 GB HBM2e) – Top-tier large-scale AI GPU with excellent mixed-precision performance, ideal for massive model training and multi-GPU scaling.
- NVIDIA L40S (48 GB GDDR6) – Optimized for high-throughput inference and small- to mid-scale training, great performance-per-watt for production AI workloads.
- AMD MI210 (64 GB HBM2e) – Strong HPC-leaning training GPU with high FP64/FP32 and solid BF16/FP16 training performance, best when ROCm fits the software stack.