Use uv venv and uv pip

[maurycy]

This is a conversation starter now: should we gradually start moving towards uv that handles custom builds, venvs and pinned runs more efficiently?

The code already works, confirmed with:

python dev.py run

Tests are 2x faster:

• last main run took 13m57s
• 3a402551d306ba7b4e80c1b308c426f5f2a08e30, with enable-cache in the workflow, took 7m14s
• 4b5f47a1292e94802af93873709b688cc0c43de1, without caching in the action, took 8m20s

I didn't want to do too many changes at once, but in my opinion the best direction would be iterating over each benchmark with uv. pyperformance is just a benchmark suite after all, and the whole venv, custom build etc. management makes it unnecessarily complex.

Notably, uv run has --project, --isolated (now it's isolated if the requirements cannot be met), --with-requirements and --python (supporting custom builds), eg:

8:41:50.613123804PM CEST maurycy@gunnbjorn ~/pyperformance (uv) % uv run --isolated --project=benchmarks/bm_sympy --python=/home/maurycy/cpython/python benchmarks/bm_sympy/run_benchmark.py
Installed 4 packages in 12ms
.....................
sympy_expand: Mean +- std dev: 281 ms +- 1 ms
.....................
sympy_integrate: Mean +- std dev: 12.8 ms +- 0.1 ms
.....................
sympy_sum: Mean +- std dev: 92.8 ms +- 0.8 ms
.....................
sympy_str: Mean +- std dev: 161 ms +- 1 ms

This is similar to what happens now, just merging temporary outputs.

From what I measured venv management and dependencies only take ~5% of the run time, so no dramatic speed up should be expected, but the simplification might be worth it. Important to keep in mind that many features are never ever used, if a large chunk of benchmarks stays broken for so long and some obvious issues lurked for a while.

Another elephant in the room is compile that could also be outsourced.

I'm more than open to any suggestions!

[corona10]

Since pyperformance users include not only CPython but also alternative implementations, I don’t think relying on uv would be a good approach for us to take. cc @hugovk

[maurycy]

Since pyperformance users include not only CPython but also alternative implementations, I don’t think relying on uv would be a good approach for us to take. cc @hugovk

uv supports CPython, PyPy, GraalPy and Pyodidie

--python accepts binaries, eg custom PyPy3 build:

10:36:09.653252000AM CET maurycy@gimel /Users/maurycy/src/pyperformance % uv run --isolated --project=benchmarks/bm_sympy --python=`which pypy3` benchmarks/bm_sympy/run_benchmark.py
      Built psutil==7.1.2
Installed 4 packages in 68ms
.......
sympy_expand: Mean +- std dev: 73.9 ms +- 6.0 ms
.......
sympy_integrate: Mean +- std dev: 22.5 ms +- 3.8 ms
.......
sympy_sum: Mean +- std dev: 46.7 ms +- 6.6 ms
.......
sympy_str: Mean +- std dev: 51.1 ms +- 11.6 ms

If I understand correctly, the only requirement is that -I, -B and -C needs to be supported by an interpreter. I might be totaly wrong here, though. uv seems to check sys.implementation.name and match it

ref astral-sh/uv#2096

[corona10]

No there are a lot of implementations over you listed. e.g RustPython Jython gpython. Let s not make a big huddle to them.

[corona10]

Unless you have a strong motivation to maintain this project based on uv, let's keep it as is based on standard(pip).
I love uv but this should consider other factors.