Workflows/Multi-threaded

Serial ran one program on one core. This page takes that same job and lets the program spread its work across many cores on the same machine — still one process, but now with several threads inside it, all sharing the same memory. It is the gentlest real step up the ladder, because for most tools someone else already did the hard part: the program already knows how to use extra cores. Your job is only to give it some, and to tell it how many.

If you have read the serial page you already have the whole script. This one is that script with two lines touched — and seeing exactly which two is the point.

When this is you

You are looking at a multi-threaded workflow when the program itself knows how to use more than one core, and says so:

• Its manual offers a flag like --threads 8, -p 4, or -@ 16 — some way to hand it a number of cores.
• Or its documentation mentions OpenMP, and it responds to an OMP_NUM_THREADS setting.
• It runs as a single program — not several cooperating copies, and not across more than one node.

Plenty of everyday tools are built this way: read aligners, samtools, and many of the numerical libraries underneath R and Python. If a tool has a threads option and you have been leaving it at its default of 1, this page is how you turn it up correctly on Anunna. If the program has no such option, it is serial — more cores will not help it, and you should go back one rung.

What it looks like on Anunna

Everything from the serial job still holds: you never run the analysis on the login node, your work lives under $myScratch on Lustre rather than in your home directory, and you describe the job in a script handed to sbatch. Only two lines change.

Start from the serial script, raise the core count, and keep one task:

#!/bin/bash
#SBATCH --job-name=my_analysis
#SBATCH --ntasks=1            # still ONE process ...
#SBATCH --cpus-per-task=16   # ... but now sixteen cores for it
#SBATCH --mem=4G             # unchanged: the threads share this memory
#SBATCH --time=01:00:00      # wall-clock limit; the job is killed past this
#SBATCH --output=%x-%j.out   # stdout + stderr, named by job name and id

cd "$myScratch/my_analysis"

module load 2024 Python/3.12.3       # replace with your actual module

# Hand the core count Slurm gave you to the program:
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
python myScript.py --input data.csv --output results.csv

Against the serial script the difference is exactly two lines. --cpus-per-task went from 1 to 16, and one new line was added just before the program runs. Three things did not change, and each is deliberate:

• --ntasks stayed at 1. This is one process that grows more arms, not many processes — the distinction that separates this page from Multi-Process.
• --mem stayed the same. The threads all read and write the same memory, so asking for more cores does not multiply your memory need the way separate processes would.
• Everything else — the module, the command, the scratch directory — is untouched.

The one new line depends on how your tool takes its core count. An OpenMP program, like the one above, reads the OMP_NUM_THREADS environment variable. A tool with its own flag takes the same number directly:

samtools sort --threads ${SLURM_CPUS_PER_TASK} -o sorted.bam input.bam

Either way the rule is identical: whatever number you put in --cpus-per-task, feed that same number to the program. Write ${SLURM_CPUS_PER_TASK} rather than hard-coding 16 in two places — then changing the request is a one-line edit and the two can never drift apart.

The line that does the real work

It is tempting to read --cpus-per-task=16 as the line that parallelised the job. It is not. That line only reserves sixteen cores; by itself it does exactly what it did for the serial program — fifteen of them sit idle with your name on them.

The line that actually uses them is the one that passes the count to the program. Slurm sets the cores aside and exports the number as SLURM_CPUS_PER_TASK, but it never tells your software they exist — you do. This is the single most common mistake on this rung: raise --cpus-per-task, forget the hand-off line, and you get a serial run that merely wasted fifteen cores and a longer queue wait to boot.

The π tutorial makes you do this by hand: its OpenMP job ships with OMP_NUM_THREADS=FIXME, and Step 3 is filling in ${SLURM_CPUS_PER_TASK}. If you want the muscle memory for this page, that is where to get it.

What it buys you

The tutorial runs the same ten-billion-dart π estimate at several thread counts on one node. The shape of the result is the whole lesson:

variant   cores    wall_s
serial        1    26.008
omp           1    26.499
omp          16     1.663
omp         128     0.235

One thread costs the serial time, as it should — same work, one core, plus a hair of overhead. Sixteen cores bring it down roughly sixteenfold; 128 cores by over a hundredfold. For this deliberately friendly problem the threads scale almost perfectly.

Real tools rarely scale this cleanly. Some part of every program refuses to be split, and past a point extra threads stop helping — that is Amdahl's law, sketched as intuition on the overview and shown with the actual curves in the tutorial's discussion. The practical habit that follows: try a few thread counts on your data and watch where the speedup flattens, rather than requesting a whole node by reflex.

The honest limitation

Threads share memory, and shared memory means one node. Every thread must reach the same physical RAM, so a multi-threaded job cannot spill onto a second machine — its hard ceiling is the core count of a single node, and not one core beyond. Anunna's nodes range from tens to hundreds of cores depending on generation (see Compute Hardware Overview); whichever node you land on, that number is your absolute maximum here.

When a problem needs more than one node's worth of cores, threads alone cannot take you there. That is the boundary where multi-process (MPI) begins — many processes across many nodes — and where hybrid workflows combine the two. Do not reach for them early, though: one node of modern cores is a great deal of compute, and staying on a single node keeps your job script simple and your queue wait short. Climb to the next rung only once you have measured yourself hitting this one's ceiling.

See also

• Parallel Computing — the overview, and the map to the other workflow types.
• Serial Workflows — the baseline this page starts from.
• Multi-Process Workflows — the next rung, for when one node is not enough.
• Parallelism: Estimating π — its OpenMP rung is the worked example for this page.
• Compute Hardware Overview — how many cores a single node actually has.