View source for Python/uv

{{DISPLAYTITLE:uv}}

uv is a tool for installing Python packages and for managing the environments they live in. It does the same work as <code>pip</code> and <code>venv</code>, and it does it a great deal faster.

This page explains what uv is, how to load it on Anunna, where it keeps your files, and how to use it for everyday work. It also explains what uv is '''not''' the right tool for: the packages it installs are not built for our hardware, so when the speed of your calculation matters, the Python modules remain the better choice.

== Introduction ==

=== What uv is ===

uv is a single program that covers work you would otherwise need several separate tools for. On Anunna it is available as a module, already set up for the cluster.

=== What uv does ===

{| class="wikitable"
! uv can !! Otherwise you would use
|-
| Install Python packages || <code>pip</code>
|-
| Create and manage environments || <code>venv</code>
|-
| Download other versions of Python || a Python module, if one exists
|}

=== What it is faster at ===

Installing packages, and building the environments that hold them. uv produces the same result as <code>pip</code> — the very same packages, from the very same place — in a fraction of the time.

That is worth more here than it would be on your own computer. Installing an environment means writing tens of thousands of small files onto a filesystem that everybody on the cluster shares, so finishing sooner is better for you and better for everyone else.

=== What it is not faster at ===

Running your code.

The packages uv installs are general-purpose builds, made to work on any machine in the world. They are exactly what <code>pip</code> would have given you. The software we provide as modules is compiled for the precise processors in each generation of Anunna's nodes, and that is where the speed of a calculation comes from. If your work is heavy, use the modules — see [[Python/Modules]].

The same applies to Python itself. uv is able to download a Python for you, but it is a general-purpose build too. The module is set up to use the Python you have loaded rather than fetching one.

=== What we have already set up ===

Left alone, uv keeps its downloads and environments in your home directory, which is small, backed up, and slower than Lustre. The Anunna module points all of it at Lustre instead, and there is nothing you need to do to make that happen. The next two sections cover loading the module and where your files end up.

== Loading uv ==

uv lives in the '''2025''' bucket:

<syntaxhighlight lang="bash">
module load 2025 uv
</syntaxhighlight>

Loading uv also loads Python 3.13.1, so you do not need to load a Python module first.

Software from different buckets should not be mixed, so anything you use alongside uv should come from the 2025 bucket as well. See [[Environment Modules#Software buckets]].

To check that it worked:

<syntaxhighlight lang="bash">
uv --version
</syntaxhighlight>

A second command, <code>uvx</code>, arrives together with <code>uv</code>. It is described in [[#Other things uv can do]].

== Where uv keeps your files ==

uv accumulates three kinds of thing as you use it: the packages it has downloaded, any Python versions it has fetched, and the tools you have installed with it. On Anunna the module places all of them on Lustre, under a folder called <code>uv</code> in your nobackup location — <code>$myNoBkp/uv</code>. Nobackup is the right tier because everything in there can be downloaded again.

The module does this by setting a handful of variables for you:

{| class="wikitable"
! Variable !! What it points at
|-
| <code>UV_DIR</code> || The folder that holds all of the others. This is the only one you might want to change.
|-
| <code>UV_CACHE_DIR</code> || Packages uv has downloaded. Normally the largest of them.
|-
| <code>UV_PYTHON_INSTALL_DIR</code> || Python versions uv has downloaded.
|-
| <code>UV_TOOL_DIR</code> || Environments for tools installed with <code>uv tool install</code>.
|-
| <code>UV_TOOL_BIN_DIR</code> || The programs those tools provide. The module has already added this to your <code>PATH</code>.
|-
| <code>UV_PYTHON_BIN_DIR</code> || Shortcuts to downloaded Python versions.
|}

=== Putting them somewhere else ===

Every folder in that table sits inside <code>UV_DIR</code>, so setting that one variable moves all of them. Set it '''before''' loading the module:

<syntaxhighlight lang="bash">
export UV_DIR=$myScratch/uv
module load 2025 uv
</syntaxhighlight>

To see where uv is actually working, ask:

<syntaxhighlight lang="bash">
echo $UV_DIR
</syntaxhighlight>

Use that rather than <code>$myNoBkp</code>, which is only set once you have added it to your shell yourself — see [[Installing Personal Software#Aliases and local variables]].

=== Keeping it from growing ===

The cache is meant to grow; that is part of what makes uv fast. It still counts against your Lustre quota, so clear it out from time to time:

<syntaxhighlight lang="bash">
uv cache prune      # remove entries nothing is using
uv cache clean      # remove everything
</syntaxhighlight>

== Creating an environment and installing packages ==

=== Never build an environment on a login node ===

Building an environment writes thousands of small files at once. A login node is shared by every user on the cluster, so that work is felt by all of them rather than only by you. Ask for an interactive session on a compute node and work there:

<syntaxhighlight lang="bash">
sinteractive -c 4 --mem 16G --time 120
</syntaxhighlight>

See [[Interactive Jobs]] for the options <code>sinteractive</code> accepts.

=== Creating an environment ===

<syntaxhighlight lang="bash">
module load 2025 uv
uv venv $myNoBkp/PythonEnv/my_env
</syntaxhighlight>

Environments belong on Lustre rather than in your home directory, and nobackup is the right tier because an environment can always be built again from the list of packages in it.

If you run <code>uv venv</code> without giving a path, uv creates a folder named <code>.venv</code> in whatever directory you happen to be in. Giving the path yourself is clearer, and it keeps your environments together in one place.

=== Using it ===

Activate the environment whenever you want it:

<syntaxhighlight lang="bash">
source $myNoBkp/PythonEnv/my_env/bin/activate
</syntaxhighlight>

Its name then appears at the front of your prompt:

<syntaxhighlight lang="text">
(my_env) user001@login200:~$
</syntaxhighlight>

Install packages into it:

<syntaxhighlight lang="bash">
uv pip install numpy pandas
</syntaxhighlight>

Leave it again with <code>deactivate</code>.

What uv builds is an ordinary virtual environment. <code>pip</code> still works inside it if you prefer, and it can be turned into a Jupyter kernel exactly as [[Python/Modules#Jupyter kernels]] describes.

=== The commands you already know ===

{| class="wikitable"
! If you know this !! Use this
|-
| <code>python -m venv my_env</code> || <code>uv venv my_env</code>
|-
| <code>source my_env/bin/activate</code> || unchanged
|-
| <code>pip install numpy</code> || <code>uv pip install numpy</code>
|-
| <code>pip list</code> || <code>uv pip list</code>
|}

== Other things uv can do ==

=== Running a tool without installing it ===

<code>uvx</code> fetches a command-line program, runs it, and does not leave an environment behind:

<syntaxhighlight lang="bash">
uvx ruff check .
</syntaxhighlight>

=== Installing a tool you use often ===

<syntaxhighlight lang="bash">
uv tool install ruff
</syntaxhighlight>

The program lands in your uv folder, which the module has already added to your <code>PATH</code>, so you can run it straight away and in any later session.

=== Using a different version of Python ===

If you need a version of Python we do not provide as a module, uv can fetch one:

<syntaxhighlight lang="bash">
uv python install 3.12
uv venv --python 3.12 $myNoBkp/PythonEnv/py312
</syntaxhighlight>

Remember that these are general-purpose builds, like the packages. They are convenient, and they are the right choice when you need a specific version for compatibility — but if the speed of your calculation matters, a Python module is the better starting point.

'''Do not run <code>uv python update-shell</code>.''' It writes a fixed path into your <code>~/.bashrc</code> and <code>~/.bash_profile</code>. That applies in every shell you ever open, whether or not the module is loaded, and it stops being correct the moment your uv folder moves.

== Using uv in a job script ==

Build the environment '''before''' you submit the job, not inside it. Create it once in an interactive session, as above, and let the job simply use it. An environment built inside the job is rebuilt on every run, and when many jobs start at the same time they all write to the filesystem at once.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=myjob
#SBATCH --time=1:00:00
#SBATCH --cpus-per-task=4
#SBATCH --mem=16G

module load 2025 uv
source $myNoBkp/PythonEnv/my_env/bin/activate

python myscript.py
</syntaxhighlight>

Both the <code>module load</code> and the <code>source</code> have to be inside the script. A job does not inherit the shell you submitted it from, so anything you loaded by hand beforehand is not there when the job runs. This is the most common reason a script that worked interactively fails under <code>sbatch</code>.

See [[Batch Jobs]] for what else can go in the <code>#SBATCH</code> lines.

== FAQ ==

; <code>module load uv</code> says there is no such module.
: The bucket has to be loaded first. Use <code>module load 2025 uv</code>. See [[Environment Modules#Software buckets]].

; Where did uv put my files?
: <code>echo $UV_DIR</code> gives the folder they are all under. To hear it from uv itself, <code>uv cache dir</code> and <code>uv tool dir</code>.

; uv warned that it could not find a Lustre directory.
: The module could not work out where your Lustre folders are, so uv has fallen back to your home directory — which will eventually fill up. Choose a location yourself with <code>export UV_DIR=$myNoBkp/uv</code> and then load the module again.

; My home directory is still filling up, even though uv uses Lustre.
: If you used uv before this module existed, an old cache is probably still sitting in <code>~/.cache/uv</code>. uv no longer looks there, so its own cleaning commands cannot reach it. Check its size with <code>du -sh ~/.cache/uv</code> and delete it by hand if you no longer need it.

; Can I still use <code>pip</code> inside a uv environment?
: Yes. It is an ordinary virtual environment and <code>pip</code> behaves normally in it.

== See also ==

* [[Python]]
* [[Python/Modules]]
* [[Python/Conda]]
* [[Environment Modules]]
* [[Installing Personal Software]]
* [[Interactive Jobs]]
* [[Batch Jobs]]
* [[Quotas]]

== External links ==

* [https://docs.astral.sh/uv/ uv documentation]
* [https://github.com/astral-sh/uv uv on GitHub]