<?xml version="1.0"?>
<feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en">
	<id>https://wiki.anunna.wur.nl/index.php?action=history&amp;feed=atom&amp;title=OpenFOAM</id>
	<title>OpenFOAM - Revision history</title>
	<link rel="self" type="application/atom+xml" href="https://wiki.anunna.wur.nl/index.php?action=history&amp;feed=atom&amp;title=OpenFOAM"/>
	<link rel="alternate" type="text/html" href="https://wiki.anunna.wur.nl/index.php?title=OpenFOAM&amp;action=history"/>
	<updated>2026-11-18T19:30:14Z</updated>
	<subtitle>Revision history for this page on the wiki</subtitle>
	<generator>MediaWiki 1.45.3</generator>
	<entry>
		<id>https://wiki.anunna.wur.nl/index.php?title=OpenFOAM&amp;diff=3263&amp;oldid=prev</id>
		<title>Honfi001: Created page with &quot;&#039;&#039;&#039;OpenFOAM&#039;&#039;&#039; is a free, open-source toolbox for computational fluid dynamics (CFD). It simulates how fluids flow, mix, heat up and react, and it also covers related problems such as solid stress and electromagnetics. It is not a single program but a large collection of them: &#039;&#039;solvers&#039;&#039; that run the simulation, and &#039;&#039;utilities&#039;&#039; that prepare a case beforehand (meshing, setting initial conditions, splitting it across processors) and process the results afterwards.  This...&quot;</title>
		<link rel="alternate" type="text/html" href="https://wiki.anunna.wur.nl/index.php?title=OpenFOAM&amp;diff=3263&amp;oldid=prev"/>
		<updated>2026-11-16T11:46:53Z</updated>

		<summary type="html">&lt;p&gt;Created page with &amp;quot;&amp;#039;&amp;#039;&amp;#039;OpenFOAM&amp;#039;&amp;#039;&amp;#039; is a free, open-source toolbox for computational fluid dynamics (CFD). It simulates how fluids flow, mix, heat up and react, and it also covers related problems such as solid stress and electromagnetics. It is not a single program but a large collection of them: &amp;#039;&amp;#039;solvers&amp;#039;&amp;#039; that run the simulation, and &amp;#039;&amp;#039;utilities&amp;#039;&amp;#039; that prepare a case beforehand (meshing, setting initial conditions, splitting it across processors) and process the results afterwards.  This...&amp;quot;&lt;/p&gt;
&lt;p&gt;&lt;b&gt;New page&lt;/b&gt;&lt;/p&gt;&lt;div&gt;&amp;#039;&amp;#039;&amp;#039;OpenFOAM&amp;#039;&amp;#039;&amp;#039; is a free, open-source toolbox for computational fluid dynamics (CFD). It simulates how fluids flow, mix, heat up and react, and it also covers related problems such as solid stress and electromagnetics. It is not a single program but a large collection of them: &amp;#039;&amp;#039;solvers&amp;#039;&amp;#039; that run the simulation, and &amp;#039;&amp;#039;utilities&amp;#039;&amp;#039; that prepare a case beforehand (meshing, setting initial conditions, splitting it across processors) and process the results afterwards.&lt;br /&gt;
&lt;br /&gt;
This page covers the practical side of using OpenFOAM on Anunna: which versions are installed, how to load them, and how to set up your cases so they don&amp;#039;t create an enormous number of files. That last point matters more for OpenFOAM than for most software, and a few settings make a large difference. How to actually run a case as a Slurm job will be covered in a separate tutorial.&lt;br /&gt;
&lt;br /&gt;
== Two families of OpenFOAM ==&lt;br /&gt;
&lt;br /&gt;
Before anything else, it helps to know that there are &amp;#039;&amp;#039;&amp;#039;two separate OpenFOAM projects&amp;#039;&amp;#039;&amp;#039; sharing the same name and the same history. They split many years ago and have developed independently since:&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! !! OpenFOAM Foundation !! ESI-OpenCFD&lt;br /&gt;
|-&lt;br /&gt;
| &amp;#039;&amp;#039;&amp;#039;Website&amp;#039;&amp;#039;&amp;#039; || [https://openfoam.org openfoam.org] || [https://www.openfoam.com openfoam.com]&lt;br /&gt;
|-&lt;br /&gt;
| &amp;#039;&amp;#039;&amp;#039;Version numbers&amp;#039;&amp;#039;&amp;#039; || Whole numbers: 6, 11, 12 … || Year and month: v2312, v2406 …&lt;br /&gt;
|-&lt;br /&gt;
| &amp;#039;&amp;#039;&amp;#039;On Anunna&amp;#039;&amp;#039;&amp;#039; || &amp;lt;code&amp;gt;12&amp;lt;/code&amp;gt; || &amp;lt;code&amp;gt;v2406&amp;lt;/code&amp;gt;&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
The two are &amp;#039;&amp;#039;&amp;#039;not interchangeable&amp;#039;&amp;#039;&amp;#039;. Solver names, dictionary keywords and the layout of the tutorials differ between them, and they keep drifting further apart. For example, from version 11 onwards the Foundation release runs most simulations through a single &amp;lt;code&amp;gt;foamRun&amp;lt;/code&amp;gt; program with a solver module (&amp;lt;code&amp;gt;foamRun -solver incompressibleFluid&amp;lt;/code&amp;gt;), where older releases and the ESI line use separate programs such as &amp;lt;code&amp;gt;simpleFoam&amp;lt;/code&amp;gt;. A case set up for one family will usually not run on the other without changes, and sometimes not even on a different version of the same family.&lt;br /&gt;
&lt;br /&gt;
The practical rule: &amp;#039;&amp;#039;&amp;#039;find out which family and version your case, tutorial or colleague uses, and load that one.&amp;#039;&amp;#039;&amp;#039; When you start something new, begin from the tutorials that ship with the version you loaded. Once OpenFOAM is loaded (see [[#Loading OpenFOAM|Loading OpenFOAM]]) they are in &amp;lt;code&amp;gt;$FOAM_TUTORIALS&amp;lt;/code&amp;gt;, so you can copy one into your own scratch space and work from there.&lt;br /&gt;
&lt;br /&gt;
=== What a case looks like ===&lt;br /&gt;
&lt;br /&gt;
OpenFOAM keeps everything about a simulation, which it calls a &amp;#039;&amp;#039;&amp;#039;case&amp;#039;&amp;#039;&amp;#039;, in an ordinary directory of plain text files. A freshly set-up case has three folders:&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! Folder !! What it holds&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;0/&amp;lt;/code&amp;gt; || The starting state: one file per field (velocity &amp;lt;code&amp;gt;U&amp;lt;/code&amp;gt;, pressure &amp;lt;code&amp;gt;p&amp;lt;/code&amp;gt;, temperature &amp;lt;code&amp;gt;T&amp;lt;/code&amp;gt; …) with its initial and boundary conditions&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;constant/&amp;lt;/code&amp;gt; || Things that do not change during the run: the mesh (in &amp;lt;code&amp;gt;constant/polyMesh/&amp;lt;/code&amp;gt;) and physical properties&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;system/&amp;lt;/code&amp;gt; || How to run: time stepping and output in &amp;lt;code&amp;gt;controlDict&amp;lt;/code&amp;gt;, numerical schemes and solver settings, and how to split the case across processors&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
As the simulation runs, OpenFOAM writes a &amp;#039;&amp;#039;&amp;#039;new directory for every time it saves results&amp;#039;&amp;#039;&amp;#039; (&amp;lt;code&amp;gt;0.1/&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;0.2/&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;0.3/&amp;lt;/code&amp;gt; …), each with its own file for every field. Run the case in parallel and, by default, each processor gets its own &amp;lt;code&amp;gt;processor0/&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;processor1/&amp;lt;/code&amp;gt; … directory containing its own copy of all of that.&lt;br /&gt;
&lt;br /&gt;
Each file is small, but there are a great many of them, and the number multiplies quickly. A case that started with a dozen files can end with tens of thousands. This is the reason for the [[#Keeping file counts down|Keeping file counts down]] section further down this page.&lt;br /&gt;
&lt;br /&gt;
== Available versions ==&lt;br /&gt;
&lt;br /&gt;
Software on Anunna is grouped into [[Environment Modules|buckets]], and all the OpenFOAM versions live in the &amp;lt;code&amp;gt;2023&amp;lt;/code&amp;gt; bucket:&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! Module !! Family !! Released !! When to choose it&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;OpenFOAM/12-foss-2023a&amp;lt;/code&amp;gt; || Foundation || July 2024 || The usual choice for new work on the openfoam.org side&lt;br /&gt;
|-&lt;br /&gt;
| &amp;lt;code&amp;gt;OpenFOAM/v2406-foss-2023a&amp;lt;/code&amp;gt; || ESI-OpenCFD || June 2024 || Your cases, tutorials or colleagues use openfoam.com&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
Both were built with the same toolchain, &amp;lt;code&amp;gt;foss/2023a&amp;lt;/code&amp;gt;: GCC 12.3.0 as the compiler and OpenMPI 4.1.5 for running in parallel. Loading OpenFOAM loads these for you. If you want to know more about the MPI side, see [[OpenMPI]].&lt;br /&gt;
&lt;br /&gt;
&amp;#039;&amp;#039;&amp;#039;ParaView&amp;#039;&amp;#039;&amp;#039;, the program most people use to look at OpenFOAM results, comes along automatically: loading either version also loads &amp;lt;code&amp;gt;ParaView/5.11.2-foss-2023a&amp;lt;/code&amp;gt;.&lt;br /&gt;
&lt;br /&gt;
This table is accurate at the time of writing. To see what is installed right now, ask the module system directly. &amp;lt;code&amp;gt;module spider&amp;lt;/code&amp;gt; searches every bucket, so you do not need to load one first:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
module spider OpenFOAM&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
=== Need a different version? ===&lt;br /&gt;
&lt;br /&gt;
The &amp;lt;code&amp;gt;2023&amp;lt;/code&amp;gt; bucket is in maintenance mode: what is there stays, but no new software is added to it. If you need a version of OpenFOAM that is not listed above, submit a software request at [https://ideas.anunna.wur.nl ideas.anunna.wur.nl]. A new version would be installed in a newer bucket, not in &amp;lt;code&amp;gt;2023&amp;lt;/code&amp;gt;.&lt;br /&gt;
&lt;br /&gt;
== Loading OpenFOAM ==&lt;br /&gt;
&lt;br /&gt;
Getting OpenFOAM ready takes three lines, always in this order. The examples on this page assume the bash shell, which is the default on Anunna:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
module load 2023                     # 1. the bucket&lt;br /&gt;
module load OpenFOAM/12-foss-2023a   # 2. the version you want&lt;br /&gt;
source $FOAM_BASH                    # 3. OpenFOAM&amp;#039;s own environment&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
# &amp;#039;&amp;#039;&amp;#039;The bucket.&amp;#039;&amp;#039;&amp;#039; Nothing in a bucket can be loaded until the bucket itself is.&lt;br /&gt;
# &amp;#039;&amp;#039;&amp;#039;The module.&amp;#039;&amp;#039;&amp;#039; This puts OpenFOAM&amp;#039;s files on the system and loads what it depends on (the compiler, OpenMPI and ParaView). Swap in &amp;lt;code&amp;gt;v2406-foss-2023a&amp;lt;/code&amp;gt; for the ESI version.&lt;br /&gt;
# &amp;#039;&amp;#039;&amp;#039;OpenFOAM&amp;#039;s environment.&amp;#039;&amp;#039;&amp;#039; OpenFOAM comes with its own setup script, and the module tells you where it is in the &amp;lt;code&amp;gt;$FOAM_BASH&amp;lt;/code&amp;gt; variable. Sourcing it sets up the paths to the solvers and utilities and variables such as &amp;lt;code&amp;gt;$FOAM_TUTORIALS&amp;lt;/code&amp;gt;. &amp;#039;&amp;#039;&amp;#039;Do not skip this step.&amp;#039;&amp;#039;&amp;#039; The module alone is not enough, and without it commands like &amp;lt;code&amp;gt;blockMesh&amp;lt;/code&amp;gt; are simply not found.&lt;br /&gt;
&lt;br /&gt;
=== Checking it worked ===&lt;br /&gt;
&lt;br /&gt;
Two quick commands confirm that you have the version you meant to load and that its programs are found:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
echo $WM_PROJECT_VERSION    # prints the version, for example 12&lt;br /&gt;
which blockMesh             # prints a path ending in .../bin/blockMesh&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If &amp;lt;code&amp;gt;which&amp;lt;/code&amp;gt; prints nothing at all, the &amp;lt;code&amp;gt;source $FOAM_BASH&amp;lt;/code&amp;gt; line has not been run in this shell.&lt;br /&gt;
&lt;br /&gt;
=== Where to keep your cases ===&lt;br /&gt;
&lt;br /&gt;
Once loaded, OpenFOAM suggests a working directory for you, &amp;lt;code&amp;gt;$FOAM_RUN&amp;lt;/code&amp;gt;, and offers a &amp;lt;code&amp;gt;run&amp;lt;/code&amp;gt; shortcut to go there. &amp;#039;&amp;#039;&amp;#039;Do not use it.&amp;#039;&amp;#039;&amp;#039; It points inside your home directory (&amp;lt;code&amp;gt;~/OpenFOAM/&amp;lt;user&amp;gt;-&amp;lt;version&amp;gt;/run&amp;lt;/code&amp;gt;), and your home directory is small, backed up, and not meant for simulation data. See [[Shared Storage]] for what each filesystem is for.&lt;br /&gt;
&lt;br /&gt;
Keep your cases on Lustre instead, under &amp;lt;code&amp;gt;$myScratch&amp;lt;/code&amp;gt;. If you have not set up that variable yet, Part 1 of [[HPC Advanced/Preparation Exercise]] shows you how. For example, to make a folder for your OpenFOAM work and copy in a tutorial case to start from:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
mkdir -p $myScratch/openfoam&lt;br /&gt;
cp -r $FOAM_TUTORIALS/incompressibleFluid/pitzDaily $myScratch/openfoam/&lt;br /&gt;
cd $myScratch/openfoam/pitzDaily&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
That tutorial path is the one for version 12. In v2406 the same case is at &amp;lt;code&amp;gt;$FOAM_TUTORIALS/incompressible/simpleFoam/pitzDaily&amp;lt;/code&amp;gt;, another small sign of how the two families differ.&lt;br /&gt;
&lt;br /&gt;
Setting up and copying cases on the login node is fine. Running the solvers is not: that belongs in a Slurm job on a compute node, which the upcoming OpenFOAM tutorial will cover. And whichever directory you work in, the advice in [[#Keeping file counts down|Keeping file counts down]] applies.&lt;br /&gt;
&lt;br /&gt;
=== Keep it out of your .bashrc ===&lt;br /&gt;
&lt;br /&gt;
It is tempting to put these three lines in your &amp;lt;code&amp;gt;~/.bashrc&amp;lt;/code&amp;gt; so OpenFOAM is always ready. Please don&amp;#039;t. They would then run on every login, slow down every new shell, and interfere when you switch OpenFOAM versions or use other software built with a different toolchain. Run them at the start of the session where you need OpenFOAM, and put them at the top of every job script that uses it.&lt;br /&gt;
&lt;br /&gt;
== Keeping file counts down ==&lt;br /&gt;
&lt;br /&gt;
This is the most important section on this page. OpenFOAM&amp;#039;s default settings can turn one simulation into hundreds of thousands of small files, and a few changes to your case keep that number small without losing anything you need.&lt;br /&gt;
&lt;br /&gt;
=== Why file counts matter ===&lt;br /&gt;
&lt;br /&gt;
Every file and every directory on Lustre uses one &amp;#039;&amp;#039;&amp;#039;inode&amp;#039;&amp;#039;&amp;#039;, the entry the filesystem keeps to record that the file exists. It doesn&amp;#039;t matter how small the file is: a 1 KB file costs as much bookkeeping as a 1 GB one. All of these entries are kept on a shared &amp;#039;&amp;#039;&amp;#039;metadata server&amp;#039;&amp;#039;&amp;#039;, which has a limited number of them and handles every file lookup for every user on Anunna. Millions of tiny files make the whole filesystem slower for everyone, not just for you.&lt;br /&gt;
&lt;br /&gt;
OpenFOAM is especially prone to this because of how it writes results (see [[#What a case looks like|What a case looks like]]). By default, every processor writes its own file for every field at every saved time. A worked example:&lt;br /&gt;
&lt;br /&gt;
{| class=&amp;quot;wikitable&amp;quot;&lt;br /&gt;
! !! Default (uncollated) !! Collated&lt;br /&gt;
|-&lt;br /&gt;
| Parallel run on 128 processors, 100 saved times, 6 fields || 128 × 100 × 6 ≈ &amp;#039;&amp;#039;&amp;#039;77,000 files&amp;#039;&amp;#039;&amp;#039; || 100 × 6 ≈ &amp;#039;&amp;#039;&amp;#039;600 files&amp;#039;&amp;#039;&amp;#039;&lt;br /&gt;
|}&lt;br /&gt;
&lt;br /&gt;
Same simulation, same results, over a hundred times fewer files.&lt;br /&gt;
&lt;br /&gt;
Anunna does not yet limit how many files each user may have on Lustre, but per-user quotas are planned. Getting into these habits now keeps the filesystem fast for everyone, and means nothing breaks when the limits arrive. See [[Quotas]] for the current state.&lt;br /&gt;
&lt;br /&gt;
=== Write results less often ===&lt;br /&gt;
&lt;br /&gt;
How often OpenFOAM saves results is set in your case&amp;#039;s &amp;lt;code&amp;gt;system/controlDict&amp;lt;/code&amp;gt;. The tutorials ship with settings meant for learning, not for production runs, so it is worth looking at these lines in every case you run:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;cpp&amp;quot;&amp;gt;&lt;br /&gt;
writeInterval    0.01;     // how often to save (meaning depends on writeControl)&lt;br /&gt;
purgeWrite       0;        // 0 = keep every saved time; N = keep only the last N&lt;br /&gt;
writeFormat      ascii;    // ascii or binary&lt;br /&gt;
writeCompression off;      // off or on&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
* &amp;#039;&amp;#039;&amp;#039;&amp;lt;code&amp;gt;writeInterval&amp;lt;/code&amp;gt;&amp;#039;&amp;#039;&amp;#039; controls how often results are saved. Choose it for what you will actually look at afterwards, not &amp;quot;as often as possible, just in case&amp;quot;. Saving every 10 time units instead of every 1 means ten times fewer files. What the number means (seconds of simulated time, or number of time steps) depends on the &amp;lt;code&amp;gt;writeControl&amp;lt;/code&amp;gt; line just above it.&lt;br /&gt;
* &amp;#039;&amp;#039;&amp;#039;&amp;lt;code&amp;gt;purgeWrite&amp;lt;/code&amp;gt;&amp;#039;&amp;#039;&amp;#039; keeps only the most recent saved times and deletes older ones as the run goes on. For a steady-state run, where only the final answer matters, &amp;lt;code&amp;gt;purgeWrite 2;&amp;lt;/code&amp;gt; keeps the last two saves, which is enough to restart from. For a transient run where you need the history, leave it at 0 and use function objects (see below) to record the history instead.&lt;br /&gt;
* &amp;#039;&amp;#039;&amp;#039;&amp;lt;code&amp;gt;writeFormat binary;&amp;lt;/code&amp;gt;&amp;#039;&amp;#039;&amp;#039; and &amp;#039;&amp;#039;&amp;#039;&amp;lt;code&amp;gt;writeCompression on;&amp;lt;/code&amp;gt;&amp;#039;&amp;#039;&amp;#039; make each file smaller and faster to write. To be clear: they reduce the &amp;#039;&amp;#039;size&amp;#039;&amp;#039; of your data, not the &amp;#039;&amp;#039;number&amp;#039;&amp;#039; of files. They are still worth turning on, but they are not a substitute for the other settings in this section.&lt;br /&gt;
&lt;br /&gt;
=== Collate parallel output ===&lt;br /&gt;
&lt;br /&gt;
&amp;#039;&amp;#039;&amp;#039;For every parallel run on Anunna, use the collated file handler.&amp;#039;&amp;#039;&amp;#039; Instead of one &amp;lt;code&amp;gt;processor0/&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;processor1/&amp;lt;/code&amp;gt; … directory per processor, each with its own set of files, the collated handler gathers everything into a single &amp;lt;code&amp;gt;processors128/&amp;lt;/code&amp;gt; directory (the number is how many processors you used), with one file per field per saved time. This is where the 77,000 versus 600 in the table above comes from.&lt;br /&gt;
&lt;br /&gt;
Both versions on Anunna support it. The simplest way to switch it on is to add this to your case&amp;#039;s &amp;lt;code&amp;gt;system/controlDict&amp;lt;/code&amp;gt;, so that every OpenFOAM command run on that case uses it automatically:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;cpp&amp;quot;&amp;gt;&lt;br /&gt;
OptimisationSwitches&lt;br /&gt;
{&lt;br /&gt;
    fileHandler collated;&lt;br /&gt;
}&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
Alternatively, add &amp;lt;code&amp;gt;-fileHandler collated&amp;lt;/code&amp;gt; to each command, for example &amp;lt;code&amp;gt;decomposePar -fileHandler collated&amp;lt;/code&amp;gt;. That is easy to forget on one command and end up with a mix, so the &amp;lt;code&amp;gt;controlDict&amp;lt;/code&amp;gt; setting is the better habit.&lt;br /&gt;
&lt;br /&gt;
For very large runs, writing everything through a single process can become a bottleneck. You can then let a few processes share the writing, for example one per node, by setting the &amp;lt;code&amp;gt;FOAM_IORANKS&amp;lt;/code&amp;gt; variable in your job script to the list of processes that should write:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
export FOAM_IORANKS=&amp;#039;(0 64)&amp;#039;   # example: processes 0 and 64 each write for their group&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
The output is then split into a few &amp;lt;code&amp;gt;processors…&amp;lt;/code&amp;gt; directories, one per writing group: still a handful of files rather than thousands. For most runs you do not need this.&lt;br /&gt;
&lt;br /&gt;
If you already have a case decomposed the old way, you can convert it to collated with &amp;lt;code&amp;gt;foamFormatConvert&amp;lt;/code&amp;gt;, run in parallel inside a job:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
mpirun -np ${SLURM_NTASKS} foamFormatConvert -parallel -fileHandler collated&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
This writes the new &amp;lt;code&amp;gt;processors…&amp;lt;/code&amp;gt; directory but does &amp;#039;&amp;#039;&amp;#039;not&amp;#039;&amp;#039;&amp;#039; remove the old &amp;lt;code&amp;gt;processor0/&amp;lt;/code&amp;gt;, &amp;lt;code&amp;gt;processor1/&amp;lt;/code&amp;gt; … directories, so for a moment you have both. Once you have checked that the collated copy works (for example with &amp;lt;code&amp;gt;checkMesh -parallel&amp;lt;/code&amp;gt;), delete the old ones with &amp;lt;code&amp;gt;rm -r processor[0-9]*&amp;lt;/code&amp;gt;. Note that the pattern has a digit after &amp;lt;code&amp;gt;processor&amp;lt;/code&amp;gt;, which keeps the new &amp;lt;code&amp;gt;processors…&amp;lt;/code&amp;gt; directory safe.&lt;br /&gt;
&amp;lt;!-- Verified 2026-11-16 on Anunna (jobs 44143376, 44145943) for 12 and v2406: conversion + checkMesh on the collated copy alone, and the controlDict switch with a parallel solver run writing into processors4/. Re-test new versions with easyconfigs tests/openfoam/collated_suite.sbatch before adding them here. --&amp;gt;&lt;br /&gt;
&lt;br /&gt;
=== Save only what you need ===&lt;br /&gt;
&lt;br /&gt;
Often you don&amp;#039;t need the whole flow field at every saved time. You need a few numbers: the force on a wall, the pressure at a point, the flow rate through an outlet. OpenFOAM&amp;#039;s &amp;#039;&amp;#039;&amp;#039;function objects&amp;#039;&amp;#039;&amp;#039; calculate these &amp;#039;&amp;#039;while the simulation runs&amp;#039;&amp;#039; and append them to one small file per quantity, instead of you saving complete fields and working the numbers out afterwards.&lt;br /&gt;
&lt;br /&gt;
Every version ships ready-made function objects, including &amp;lt;code&amp;gt;probes&amp;lt;/code&amp;gt; (values at chosen points), &amp;lt;code&amp;gt;forces&amp;lt;/code&amp;gt; (forces and moments on surfaces) and &amp;lt;code&amp;gt;surfaceFieldValue&amp;lt;/code&amp;gt; (averages, totals or flow rates over a surface). To see the full list for your version:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
foamPostProcess -list    # version 12&lt;br /&gt;
postProcess -list        # version v2406&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
The syntax for adding them to a case differs between the two OpenFOAM families, so follow the tutorials and documentation for your version. Combined with a large &amp;lt;code&amp;gt;writeInterval&amp;lt;/code&amp;gt;, function objects let you keep the full time history of what you care about while saving complete fields only occasionally.&lt;br /&gt;
&lt;br /&gt;
=== Clean up when a run is finished ===&lt;br /&gt;
&lt;br /&gt;
A finished case usually contains a lot it no longer needs.&lt;br /&gt;
&lt;br /&gt;
* &amp;#039;&amp;#039;&amp;#039;Processor directories.&amp;#039;&amp;#039;&amp;#039; If you reconstruct the results into a single case with &amp;lt;code&amp;gt;reconstructPar&amp;lt;/code&amp;gt; (or &amp;lt;code&amp;gt;reconstructPar -latestTime&amp;lt;/code&amp;gt; for only the final time), the per-processor data is a complete duplicate. Once you have checked the reconstructed results, delete it (&amp;lt;code&amp;gt;rm -r processor*&amp;lt;/code&amp;gt;). With the collated handler there is only one &amp;lt;code&amp;gt;processors…&amp;lt;/code&amp;gt; directory to remove.&lt;br /&gt;
* &amp;#039;&amp;#039;&amp;#039;Saved times you don&amp;#039;t need.&amp;#039;&amp;#039;&amp;#039; &amp;lt;code&amp;gt;foamListTimes&amp;lt;/code&amp;gt; lists the saved times in a case, and with &amp;lt;code&amp;gt;-rm&amp;lt;/code&amp;gt; it deletes them. &amp;#039;&amp;#039;&amp;#039;It deletes every time it lists&amp;#039;&amp;#039;&amp;#039;, so always run it without &amp;lt;code&amp;gt;-rm&amp;lt;/code&amp;gt; first and narrow it down with &amp;lt;code&amp;gt;-time&amp;lt;/code&amp;gt;:&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
foamListTimes -time &amp;#039;0.1:0.9&amp;#039;        # 1. see which times would be affected&lt;br /&gt;
foamListTimes -time &amp;#039;0.1:0.9&amp;#039; -rm    # 2. then delete exactly those&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
:Add &amp;lt;code&amp;gt;-processor&amp;lt;/code&amp;gt; to do the same inside an uncollated case&amp;#039;s &amp;lt;code&amp;gt;processor*&amp;lt;/code&amp;gt; directories.&lt;br /&gt;
* &amp;#039;&amp;#039;&amp;#039;Finished cases you want to keep.&amp;#039;&amp;#039;&amp;#039; Pack the whole case into a single archive file, which takes one inode instead of thousands, check that the archive is complete, and only then remove the directory:&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
tar -czf pitzDaily.tar.gz pitzDaily&lt;br /&gt;
tar -tzf pitzDaily.tar.gz &amp;gt; /dev/null &amp;amp;&amp;amp; rm -r pitzDaily&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
=== Check your own file count ===&lt;br /&gt;
&lt;br /&gt;
To count the files and directories in a single case:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
find $myScratch/openfoam/pitzDaily | wc -l&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
To count everything you own in your scratch folder, use Lustre&amp;#039;s own &amp;lt;code&amp;gt;lfs find&amp;lt;/code&amp;gt;, which is gentler on the metadata server than plain &amp;lt;code&amp;gt;find&amp;lt;/code&amp;gt;. It can still take a while if you have a lot of files:&lt;br /&gt;
&lt;br /&gt;
&amp;lt;syntaxhighlight lang=&amp;quot;bash&amp;quot;&amp;gt;&lt;br /&gt;
lfs find $myScratch --user $USER | wc -l&lt;br /&gt;
&amp;lt;/syntaxhighlight&amp;gt;&lt;br /&gt;
&lt;br /&gt;
If either number runs into the hundreds of thousands, the sections above are the place to start. Once per-user limits are introduced, &amp;lt;code&amp;gt;lfs quota&amp;lt;/code&amp;gt; will show your usage directly; see [[Quotas]].&lt;br /&gt;
&lt;br /&gt;
== See also ==&lt;br /&gt;
&lt;br /&gt;
On this wiki:&lt;br /&gt;
&lt;br /&gt;
* [[OpenMPI]]: the MPI library OpenFOAM uses to run in parallel, and how MPI jobs are launched on Anunna&lt;br /&gt;
* [[Environment Modules]]: how buckets and modules work&lt;br /&gt;
* [[Shared Storage]]: what each filesystem is for, and why simulation data belongs on Lustre&lt;br /&gt;
* [[Quotas]]: how much space you have on each filesystem&lt;br /&gt;
* [[Best-Practices]]: the general do&amp;#039;s and don&amp;#039;ts for working on Anunna&lt;br /&gt;
* [[HPC Advanced/Preparation Exercise]]: setting up &amp;lt;code&amp;gt;$myScratch&amp;lt;/code&amp;gt; and the other Lustre shortcuts&lt;br /&gt;
&lt;br /&gt;
Need software that is not installed? Submit a request at [https://ideas.anunna.wur.nl ideas.anunna.wur.nl].&lt;br /&gt;
&lt;br /&gt;
OpenFOAM&amp;#039;s own documentation:&lt;br /&gt;
&lt;br /&gt;
* [https://doc.cfd.direct/openfoam/user-guide-v12/ OpenFOAM 12 User Guide]: the Foundation release (openfoam.org), written by CFD Direct&lt;br /&gt;
* [https://openfoam.org/version/12/ OpenFOAM 12 release notes]: what is new in version 12&lt;br /&gt;
* [https://www.openfoam.com/documentation/user-guide OpenFOAM User Guide (openfoam.com)]: the ESI-OpenCFD release, which covers &amp;lt;code&amp;gt;v2406&amp;lt;/code&amp;gt;&lt;/div&gt;</summary>
		<author><name>Honfi001</name></author>
	</entry>
</feed>