migration from poetry to uv

[king-11]

make use of standard keys for project and dependeny specification

• provide sources to run uv build so that it can refer local packages
• using hatchling for build as is stock build option
• use optional-dependencies.dev for dev-dependencies
• add hatch targets for packages and includes where unclear
• update Makefiles to use uv when running commands that access python dependencies

#7665

Important

25.05 FREEZE MAY 12TH: Non-bugfix PRs not ready by this date will wait for 25.08.

RC1 is scheduled on May 23rd, RC2 on May 26th, ...

The final release is on MAY 29TH.

Checklist

Before submitting the PR, ensure the following tasks are completed. If an item is not applicable to your PR, please mark it as checked:

• The changelog has been updated in the relevant commit(s) according to the guidelines.
• Tests have been added or modified to reflect the changes.
• Documentation has been reviewed and updated as needed.
• Related issues have been listed and linked, including any that this PR closes.

[king-11]

pending changes:

• ci.yaml
• macos.yaml
• setup.sh
• reckless

[vincenzopalazzo]

What are the benefit of uv over poetry?

[cdecker]

It's much faster (5-10 seconds compared to minutes using poetry). While that doesn't sound too different, it means that we don't create long lived venvs that may drift from the metadata, causing fewer "it worked on my machine" kind of errors. It can also be used as a shebang, automatically wrapping for example a plugin, downloading dependencies on first invocation, creating the venv and only then passing control to python. This means we can make plugins self-contained and single file if we wanted to.

The speed, which unlocks all of these use cases, is done via caching, and interesting speedup ideas, that are quite fun to read up on. And finally, you don't need a system-wide python installation, since uv can also download and manage python versions (prevents you from accidentally polluting the system python and potentially break your OS).

[cdecker]

Great initiative @king-11 , I had a branch with some initial steps, but you are much further along, so I'd rather help you than double-tracking

[king-11]

Thanks @cdecker i will get the last few things in, just raised a draft PR given I saw some interest around it and wanted to have a pre-ready change look available.

[king-11]

@cdecker do you have any suggestions I am currently encountering an issue where running tests shows pyln path not found.

After these changes I am setting things up as

uv venv --python 3.12
source .venv/bin/activate
uv sync --all-extras
uv run pytest tests/test_renepay.py

I ran the debug command

uv pip list

it shows pyln-testing is installed the only hunch I have here is maybe I need to set that as editable 🤔

[cdecker]

Yeah, workspace-related options take a bit of getting used to.

For packages that we'd like uv to pull in from the repo, rather than the PyPI repository, we need to tell uv about the dependency:

[project]
...
dependencies = [
  "pyln-testing",
  ...
]

Then we need to tell uv that there is a member gl-testing in the workspace:

[tool.uv.workspace]
members = [
  "pyln-testing",
  ...
]

And then finally we need to tell uv to prefer the local copy from the repo, rather than the PyPI repo:

[tool.uv.sources]
pyln-testing = { workspace = true }

At least these are the three steps that would work for me whenever I ran into this issue.

[king-11]

Got it thanks

[king-11]

Thanks for the workspace tip @cdecker that helped but one more thing was needed that is __init__.py files with PATH override for namespace sharing of pyln took some help from our ai friends going to get working on ci now I think

[king-11]

The flake8 fails for auto generated files not sure why but I have ran locally and tests are working fine now. Can you use some guidance on how to resolve the flake9 issue