Uv: Difference between revisions
Created page with "{{DISPLAYTITLE:uv}} uv is a tool for installing Python packages and for managing the environments they live in. It does the same work as <code>pip</code> and <code>venv</code>, and it does it a great deal faster. This page explains what uv is, how to load it on Anunna, where it keeps your files, and how to use it for everyday work. It also explains what uv is '''not''' the right tool for: the packages it installs are not built for our hardware, so when the speed of you..." |
mNo edit summary |
||
| Line 1: | Line 1: | ||
{{DISPLAYTITLE:uv}} | {{DISPLAYTITLE:uv}} | ||
uv is a tool for installing Python packages and for managing the environments they live in. It does the same work as <code>pip</code> and <code>venv</code>, and it | uv is a tool for installing Python packages and for managing the environments they live in. It does the same work as <code>pip</code> and <code>venv</code>, and it works it a great deal faster. | ||
This page explains what uv is, how to load it on Anunna, where it keeps your files, and how to use it for everyday work. It also explains what uv is '''not''' the right tool for: the packages it installs are not built for our hardware, so when the speed of your calculation matters, the Python modules remain the better choice. | This page explains what uv is, how to load it on Anunna, where it keeps your files, and how to use it for everyday work. It also explains what uv is '''not''' the right tool for: the packages it installs are not built for our hardware, so when the speed of your calculation matters, the Python modules remain the better choice. | ||
Latest revision as of 09:15, 28 August 2026
uv is a tool for installing Python packages and for managing the environments they live in. It does the same work as pip and venv, and it works it a great deal faster.
This page explains what uv is, how to load it on Anunna, where it keeps your files, and how to use it for everyday work. It also explains what uv is not the right tool for: the packages it installs are not built for our hardware, so when the speed of your calculation matters, the Python modules remain the better choice.
Introduction
What uv is
uv is a single program that covers work you would otherwise need several separate tools for. On Anunna it is available as a module, already set up for the cluster.
What uv does
| uv can | Otherwise you would use |
|---|---|
| Install Python packages | pip
|
| Create and manage environments | venv
|
| Download other versions of Python | a Python module, if one exists |
What it is faster at
Installing packages, and building the environments that hold them. uv produces the same result as pip — the very same packages, from the very same place — in a fraction of the time.
That is worth more here than it would be on your own computer. Installing an environment means writing tens of thousands of small files onto a filesystem that everybody on the cluster shares, so finishing sooner is better for you and better for everyone else.
What it is not faster at
Running your code.
The packages uv installs are general-purpose builds, made to work on any machine in the world. They are exactly what pip would have given you. The software we provide as modules is compiled for the precise processors in each generation of Anunna's nodes, and that is where the speed of a calculation comes from. If your work is heavy, use the modules — see Python/Modules.
The same applies to Python itself. uv is able to download a Python for you, but it is a general-purpose build too. The module is set up to use the Python you have loaded rather than fetching one.
What we have already set up
Left alone, uv keeps its downloads and environments in your home directory, which is small, backed up, and slower than Lustre. The Anunna module points all of it at Lustre instead, and there is nothing you need to do to make that happen. The next two sections cover loading the module and where your files end up.
Loading uv
uv lives in the 2025 bucket:
module load 2025 uv
Loading uv also loads Python 3.13.1, so you do not need to load a Python module first.
Software from different buckets should not be mixed, so anything you use alongside uv should come from the 2025 bucket as well. See Environment Modules#Software buckets.
To check that it worked:
uv --version
A second command, uvx, arrives together with uv. It is described in #Other things uv can do.
Where uv keeps your files
uv accumulates three kinds of thing as you use it: the packages it has downloaded, any Python versions it has fetched, and the tools you have installed with it. On Anunna the module places all of them on Lustre, under a folder called uv in your nobackup location — $myNoBkp/uv. Nobackup is the right tier because everything in there can be downloaded again.
The module does this by setting a handful of variables for you:
| Variable | What it points at |
|---|---|
UV_DIR |
The folder that holds all of the others. This is the only one you might want to change. |
UV_CACHE_DIR |
Packages uv has downloaded. Normally the largest of them. |
UV_PYTHON_INSTALL_DIR |
Python versions uv has downloaded. |
UV_TOOL_DIR |
Environments for tools installed with uv tool install.
|
UV_TOOL_BIN_DIR |
The programs those tools provide. The module has already added this to your PATH.
|
UV_PYTHON_BIN_DIR |
Shortcuts to downloaded Python versions. |
Putting them somewhere else
Every folder in that table sits inside UV_DIR, so setting that one variable moves all of them. Set it before loading the module:
export UV_DIR=$myScratch/uv
module load 2025 uv
To see where uv is actually working, ask:
echo $UV_DIR
Use that rather than $myNoBkp, which is only set once you have added it to your shell yourself — see Installing Personal Software#Aliases and local variables.
Keeping it from growing
The cache is meant to grow; that is part of what makes uv fast. It still counts against your Lustre quota, so clear it out from time to time:
uv cache prune # remove entries nothing is using
uv cache clean # remove everything
Creating an environment and installing packages
Never build an environment on a login node
Building an environment writes thousands of small files at once. A login node is shared by every user on the cluster, so that work is felt by all of them rather than only by you. Ask for an interactive session on a compute node and work there:
sinteractive -c 4 --mem 16G --time 120
See Interactive Jobs for the options sinteractive accepts.
Creating an environment
module load 2025 uv
uv venv $myNoBkp/PythonEnv/my_env
Environments belong on Lustre rather than in your home directory, and nobackup is the right tier because an environment can always be built again from the list of packages in it.
If you run uv venv without giving a path, uv creates a folder named .venv in whatever directory you happen to be in. Giving the path yourself is clearer, and it keeps your environments together in one place.
Using it
Activate the environment whenever you want it:
source $myNoBkp/PythonEnv/my_env/bin/activate
Its name then appears at the front of your prompt:
(my_env) user001@login200:~$
Install packages into it:
uv pip install numpy pandas
Leave it again with deactivate.
What uv builds is an ordinary virtual environment. pip still works inside it if you prefer, and it can be turned into a Jupyter kernel exactly as Python/Modules#Jupyter kernels describes.
The commands you already know
| If you know this | Use this |
|---|---|
python -m venv my_env |
uv venv my_env
|
source my_env/bin/activate |
unchanged |
pip install numpy |
uv pip install numpy
|
pip list |
uv pip list
|
Other things uv can do
Running a tool without installing it
uvx fetches a command-line program, runs it, and does not leave an environment behind:
uvx ruff check .
Installing a tool you use often
uv tool install ruff
The program lands in your uv folder, which the module has already added to your PATH, so you can run it straight away and in any later session.
Using a different version of Python
If you need a version of Python we do not provide as a module, uv can fetch one:
uv python install 3.12
uv venv --python 3.12 $myNoBkp/PythonEnv/py312
Remember that these are general-purpose builds, like the packages. They are convenient, and they are the right choice when you need a specific version for compatibility — but if the speed of your calculation matters, a Python module is the better starting point.
Do not run uv python update-shell. It writes a fixed path into your ~/.bashrc and ~/.bash_profile. That applies in every shell you ever open, whether or not the module is loaded, and it stops being correct the moment your uv folder moves.
Using uv in a job script
Build the environment before you submit the job, not inside it. Create it once in an interactive session, as above, and let the job simply use it. An environment built inside the job is rebuilt on every run, and when many jobs start at the same time they all write to the filesystem at once.
#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G
module load 2025 uv
source $myNoBkp/PythonEnv/my_env/bin/activate
python myscript.py
Both the module load and the source have to be inside the script. A job does not inherit the shell you submitted it from, so anything you loaded by hand beforehand is not there when the job runs. This is the most common reason a script that worked interactively fails under sbatch.
See Batch Jobs for what else can go in the #SBATCH lines.
FAQ
module load uvsays there is no such module.- The bucket has to be loaded first. Use
module load 2025 uv. See Environment Modules#Software buckets.
- Where did uv put my files?
echo $UV_DIRgives the folder they are all under. To hear it from uv itself,uv cache diranduv tool dir.
- uv warned that it could not find a Lustre directory.
- The module could not work out where your Lustre folders are, so uv has fallen back to your home directory — which will eventually fill up. Choose a location yourself with
export UV_DIR=$myNoBkp/uvand then load the module again.
- My home directory is still filling up, even though uv uses Lustre.
- If you used uv before this module existed, an old cache is probably still sitting in
~/.cache/uv. uv no longer looks there, so its own cleaning commands cannot reach it. Check its size withdu -sh ~/.cache/uvand delete it by hand if you no longer need it.
- Can I still use
pipinside a uv environment? - Yes. It is an ordinary virtual environment and
pipbehaves normally in it.
See also
- Python
- Python/Modules
- Python/Conda
- Environment Modules
- Installing Personal Software
- Interactive Jobs
- Batch Jobs
- Quotas