Installing Personal Software: Difference between revisions
No edit summary |
No edit summary |
||
| Line 27: | Line 27: | ||
An '''alias''' is a short name for a longer command; a '''local variable''' stores a value (such as a path) you reuse often. Defining a handful of both saves a lot of typing. | An '''alias''' is a short name for a longer command; a '''local variable''' stores a value (such as a path) you reuse often. Defining a handful of both saves a lot of typing. | ||
The example below is a complete <code>~/.bash_aliases</code>. It defines convenience aliases plus three variables that point at your personal directories on the [[Filesystems | Lustre filesystem]] — <code>$myScratch</code>, <code>$ | The example below is a complete <code>~/.bash_aliases</code>. It defines convenience aliases plus three variables that point at your personal directories on the [[Filesystems | Lustre filesystem]] — <code>$myScratch</code>, <code>$myBkp</code>, and <code>$ myNoBkp</code> . Other pages (for example [[Apptainer]]) assume these variables exist, so this is the recommended place to set them. | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash">#!/bin/bash | ||
#!/bin/bash | |||
# Preferred editor | # Preferred editor | ||
| Line 57: | Line 56: | ||
# Jump to those locations | # Jump to those locations | ||
alias cds="cd $myScratch" | alias cds="cd $myScratch" | ||
alias cdb="cd $ | alias cdb="cd $myBkp" | ||
alias cdn="cd $ | alias cdn="cd $myNoBkp" | ||
# Keep the Apptainer cache off your home quota (see Apptainer page) | # Keep the Apptainer cache off your home quota (see Apptainer page) | ||
export APPTAINER_CACHEDIR=$myScratch/.apptainer-cache | export APPTAINER_CACHEDIR=$myScratch/.apptainer-cache</syntaxhighlight> | ||
</syntaxhighlight> | |||
After editing the file, either log in again or run <code>source ~/.bash_aliases</code> to apply the changes to your current shell. | After editing the file, either log in again or run <code>source ~/.bash_aliases</code> to apply the changes to your current shell. | ||
Latest revision as of 13:20, 6 November 2026
This page covers the personal touches that make Anunna comfortable to work on: shell configuration, aliases, environment variables, a custom temporary directory, and a workaround for running older binaries. None of this is required to use the cluster, but most regular users set up at least some of it.
For installing actual software, the recommended routes are language- or tool-specific:
- Python — virtual environments and Jupyter kernels: see Python.
- R — local package libraries: see R.
- Containers — anything else, packaged reproducibly: see Apptainer.
- Cluster-wide modules — request a shared install: see Environment Modules.
Shell configuration files
Your shell reads a couple of files when it starts:
~/.bashrc— read by interactive non-login shells. This is the usual place for aliases and most environment variables.~/.bash_profile— read by login shells. Often it just sources~/.bashrcso both kinds of shell behave the same.
Keeping your aliases and variables in a separate ~/.bash_aliases file (sourced from ~/.bashrc) keeps things tidy. Add this to ~/.bashrc so the file is picked up:
if [ -f ~/.bash_aliases ]; then
. ~/.bash_aliases
fi
Aliases and local variables
An alias is a short name for a longer command; a local variable stores a value (such as a path) you reuse often. Defining a handful of both saves a lot of typing.
The example below is a complete ~/.bash_aliases. It defines convenience aliases plus three variables that point at your personal directories on the Lustre filesystem — $myScratch, $myBkp, and $ myNoBkp . Other pages (for example Apptainer) assume these variables exist, so this is the recommended place to set them.
#!/bin/bash
# Preferred editor
editor='nano' # or vim
# Edit / source the bash config files
alias ebc="${editor} ${HOME}/.bashrc"
alias sbc="source ${HOME}/.bashrc"
alias eba="${editor} ${HOME}/.bash_aliases"
alias sba="source ${HOME}/.bash_aliases"
# ls shortcuts
alias ll="ls -alth"
# Personal Lustre locations.
# The 'affiliation' depends on your group:
# WUR researchers in a commitment group: affiliation=WUR/<group>
# External users: affiliation=<organisation>
# Set this before the exports below — leaving it empty will break the paths.
affiliation=
export myScratch=/lustre/scratch/$affiliation/$USER
export myBkp=/lustre/backup/$affiliation/$USER
export myNoBkp=/lustre/nobackup/$affiliation/$USER
# Jump to those locations
alias cds="cd $myScratch"
alias cdb="cd $myBkp"
alias cdn="cd $myNoBkp"
# Keep the Apptainer cache off your home quota (see Apptainer page)
export APPTAINER_CACHEDIR=$myScratch/.apptainer-cache
After editing the file, either log in again or run source ~/.bash_aliases to apply the changes to your current shell.
Environment variables
Beyond the variables above, two common tweaks go in ~/.bashrc.
Add a personal bin directory to your PATH so your own scripts run by name:
export PATH=$PATH:$HOME/bin
Keep more shell history:
# Lines kept in the history file on disk
export HISTFILESIZE=100000
# Lines kept in memory for the current session
export HISTSIZE=100000
Custom temporary directory (TMPDIR)
Many programs write intermediate or temporary files, sometimes without telling you — sort, for instance, needs a lot of scratch space for large inputs. By default they use /tmp. Every compute node has a large local /tmp (around 400 GB), but it is shared, and a job that fills it can cause other users' jobs to fail with anything from a crash to silently wrong output.
To send temporary files somewhere with more room (but with lower speed), set the TMPDIR environment variable to a directory in your scratch space. Scratch is a good choice because it is periodically tidied up, so stray temporary files do not linger.
First create the directory:
mkdir -p $myScratch/tmp
Then set TMPDIR in ~/.bash_profile or ~/.bashrc:
export TMPDIR=$myScratch/tmp
It takes effect on your next login; to apply it to the current shell immediately, source the file you edited.
Java applications
Some compiled applications — Java programs in particular, and some C/C++ binaries — ignore TMPDIR. For Java, set the temporary directory through _JAVA_OPTIONS instead:
export _JAVA_OPTIONS=-Djava.io.tmpdir=$myScratch/tmp
This is not strictly a temporary-directory variable; it passes a runtime option to the JVM that happens to set the temporary directory.
Running older binaries
Most Linux binaries depend on shared libraries loaded by the underlying operating system at runtime. As the cluster's OS is upgraded, some of those libraries change or disappear, and an older binary may then fail with errors about glibc or missing symbols.
If you really need to keep old software running, you will need to put it inside a container, using the old OS as a start. See the Apptainer page