Installing Personal Software

This page covers the personal touches that make Anunna comfortable to work on: shell configuration, aliases, environment variables, a custom temporary directory, and a workaround for running older binaries. None of this is required to use the cluster, but most regular users set up at least some of it.

For installing actual software, the recommended routes are language- or tool-specific:

• Python — virtual environments and Jupyter kernels: see Python.
• R — local package libraries: see R.
• Containers — anything else, packaged reproducibly: see Apptainer.
• Cluster-wide modules — request a shared install: see Environment Modules.

Shell configuration files

Your shell reads a couple of files when it starts:

• ~/.bashrc — read by interactive non-login shells. This is the usual place for aliases and most environment variables.
• ~/.bash_profile — read by login shells. Often it just sources ~/.bashrc so both kinds of shell behave the same.

Keeping your aliases and variables in a separate ~/.bash_aliases file (sourced from ~/.bashrc) keeps things tidy. Add this to ~/.bashrc so the file is picked up:

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

Aliases and local variables

An alias is a short name for a longer command; a local variable stores a value (such as a path) you reuse often. Defining a handful of both saves a lot of typing.

The example below is a complete ~/.bash_aliases. It defines convenience aliases plus three variables that point at your personal directories on the Lustre filesystem — $myScratch, $myBackup, and $myNobackup. Other pages (for example Apptainer) assume these variables exist, so this is the recommended place to set them.

#!/bin/bash

# Preferred editor
editor='nano'   # or vim

# Edit / source the bash config files
alias ebc="${editor} ${HOME}/.bashrc"
alias sbc="source ${HOME}/.bashrc"
alias eba="${editor} ${HOME}/.bash_aliases"
alias sba="source ${HOME}/.bash_aliases"

# ls shortcuts
alias ll="ls -alth"

# Personal Lustre locations.
# The 'affiliation' depends on your group:
#   WUR researchers in a commitment group:  affiliation=WUR/<group>
#   External users:                         affiliation=<organisation>
# Set this before the exports below — leaving it empty will break the paths.
affiliation=

export myScratch=/lustre/scratch/$affiliation/$USER
export myBackup=/lustre/backup/$affiliation/$USER
export myNobackup=/lustre/nobackup/$affiliation/$USER

# Jump to those locations
alias cds="cd $myScratch"
alias cdb="cd $myBackup"
alias cdn="cd $myNobackup"

# Keep the Apptainer cache off your home quota (see Apptainer page)
export APPTAINER_CACHEDIR=$myScratch/.apptainer-cache

After editing the file, either log in again or run source ~/.bash_aliases to apply the changes to your current shell.

Environment variables

Beyond the variables above, two common tweaks go in ~/.bashrc.

Add a personal bin directory to your PATH so your own scripts run by name:

export PATH=$PATH:$HOME/bin

Keep more shell history:

# Lines kept in the history file on disk
export HISTFILESIZE=100000
# Lines kept in memory for the current session
export HISTSIZE=100000

Custom temporary directory (TMPDIR)

Many programs write intermediate or temporary files, sometimes without telling you — sort, for instance, needs a lot of scratch space for large inputs. By default they use /tmp. Every compute node has a large local /tmp (around 400 GB), but it is shared, and a job that fills it can cause other users' jobs to fail with anything from a crash to silently wrong output.

To send temporary files somewhere with more room, set the TMPDIR environment variable to a directory in your scratch space. Scratch is a good choice because it is periodically tidied up, so stray temporary files do not linger.

First create the directory:

mkdir -p $myScratch/tmp

Then set TMPDIR in ~/.bash_profile or ~/.bashrc:

export TMPDIR=$myScratch/tmp

It takes effect on your next login; to apply it to the current shell immediately, source the file you edited.

Java applications

Some compiled applications — Java programs in particular, and some C/C++ binaries — ignore TMPDIR. For Java, set the temporary directory through _JAVA_OPTIONS instead:

export _JAVA_OPTIONS=-Djava.io.tmpdir=$myScratch/tmp

This is not strictly a temporary-directory variable; it passes a runtime option to the JVM that happens to set the temporary directory.

Running older binaries

Most Linux binaries depend on shared libraries loaded by the operating system at runtime. As the cluster's OS is upgraded, some of those libraries change or disappear, and an older binary may then fail with errors about glibc or missing symbols.

The libraries from previous OS builds are kept on the shared filesystem so that older binaries can still find them. The LD_LIBRARY_PATH environment variable tells the linker where to look for libraries, and a module — sl7-libs — sets it up for you. It is already a prerequisite of most older module installs, so usually you do not need to do anything.

If you have compiled something yourself that is not a module and it fails this way, load the module by hand:

module load legacy sl7-libs/main

Be aware that this bad practice, if you really need to keep old software running, you are better off putting it inside a container. See the Apptainer page

Older binaries in a Jupyter kernel

A custom Jupyter kernel cannot load a module, because the kernel is started from a kernel.json file rather than a shell. Instead, set LD_LIBRARY_PATH directly in the kernel's environment block:

{
 "language": "python",
 "argv": [
  "/path/to/my/venv/bin/python",
  "-m",
  "ipykernel",
  "-f",
  "{connection_file}"
 ],
 "display_name": "myvenv",
 "env": {
  "LD_LIBRARY_PATH": "/usr/lib:/usr/lib64:/usr/lib/x86_64-linux-gnu:/shared/legacyapps/sl7-libs/lib:/shared/legacyapps/sl7-libs/lib64"
 }
}

When editing JSON, remember that every entry in an object is separated by a comma and only double quotes are allowed. Check that the file is valid by piping it through jq:

cat ~/.local/share/jupyter/kernels/mykernel/kernel.json | jq

If jq prints your JSON back, it parses correctly.

See also

• Environment Modules
• Locale settings
• Python
• R
• Apptainer
• Filesystems