-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(registry): adds reference to python-nix-template #424
base: main
Are you sure you want to change the base?
Conversation
1fd848e
to
88dc2cd
Compare
Also, is it possible to have a minimal template in the style of https://github.com/srid/haskell-template ? |
direnv fails because the template instantiation doesn't preserve the permissions of the files in the scripts/ folder ( need to also double check that git has it properly checked in ) but I can update the template envrc to avoid using if its not executable...this is used downstream for local testing of container image builds in a manner that's consistent with how it occurs in github actions It would be better if both were done purely rather than using env vars but this is how it's done upstream in flocken. The next problem that arises after See the ASCII cast there. It also shows the file permission preservation issue, but I've now separated that from #425 into |
I will look more closely at the current differences, but which aspects would you have in mind to trim out of the current draft of the python template? I am open to it, especially if we just have multiple templates, but the intent is for one of them to be fairly opinionated. |
Ah okay, I haven't looked at your other issues yet.
How about creating a new template that's basically "Hello world" and nothing else? It would provides a devShell that is aware of dependencies specified in project, and a An idea behind |
Thanks for pointing out the problem.
Yes it makes sense to at least have a default template with almost no source code so that anyone can start with a blank canvas and a second (or any number of additional) template(s) that require(s) specifying you'd like to adopt a few more conventions beyond nix alone.
Definitely. Many thanks for omnix and your input. |
Updated #424 (comment) noting workarounds for referenced issues. |
b4430c3
to
ea10a14
Compare
Actually, testing om init github:sciexp/python-nix-template/acac2f858 -o pnt-new in which the package is renamed but other values are left as defaults reveals the workaround for #425 in sciexp/python-nix-template#19 (comment) only covers the source code but still suffers from pytest error due to failure to replace strings associated to the package name in files below a folder that is also renamed
❯ pytest
==================================================================================== test session starts =====================================================================================
platform darwin -- Python 3.12.9, pytest-8.3.5, pluggy-1.5.0
rootdir: ~/projects/pnt-new
configfile: pyproject.toml
testpaths: src/pnt_new/tests
plugins: cov-6.0.0, hypothesis-6.127.6, xdoctest-1.2.0
collected 0 items / 1 error
=========================================================================================== ERRORS ===========================================================================================
______________________________________________________________________ ERROR collecting src/pnt_new/tests/test_main.py _______________________________________________________________________
ImportError while importing test module '~/projects/pnt-new/src/pnt_new/tests/test_main.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/nix/store/wwqdmdr2f5wrjnsjs64bny8df471rh9b-python3-3.12.9/lib/python3.12/importlib/__init__.py:90: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
src/pnt_new/tests/test_main.py:1: in <module>
from python_nix_template import main
E ModuleNotFoundError: No module named 'python_nix_template'
---------- coverage: platform darwin, python 3.12.9-final-0 ----------
Name Stmts Miss Cover Missing
--------------------------------------------------------------
src/pnt_new/__init__.py 2 1 50% 2
src/pnt_new/tests/__init__.py 0 0 100%
src/pnt_new/tests/test_main.py 8 7 12% 4-14
--------------------------------------------------------------
TOTAL 10 8 20%
================================================================================== short test summary info ===================================================================================
ERROR src/pnt_new/tests/test_main.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
====================================================================================== 1 error in 0.43s ====================================================================================== for the tests. The Perhaps somewhere the string replacement should be organized in such a manner that it performs a pass restricted to targeting files before a pass that is allowed to target directory names. The behavior suggests this is somehow happening in the opposite order and if a directory name has been replaced all the contents of all the files below it are ignored as candidates for string replacement. For now we workaround this using relative imports in the simple "hello world" package test. |
ea10a14
to
fdd4b27
Compare
This is at
in the omnix-registry lockfile fdd4b27 I believe omnix uses a cache if you've previously executed and do not pin up a new version of a template e.g. |
I suppose nix --accept-flake-config run github:cameronraysmith/omnix/registry-python -- \
init -o pnt-new is a better test for this PR's particular lockfile update, though it rebuilds omnix from scratch since I imagine it isn't able to hit a relevant cache given the changes ❯ nix --accept-flake-config run github:cameronraysmith/omnix/registry-python -- \
init -o pnt-new
❄️ nix -j auto eval --json '/nix/store/5bfmxzdd2f5flikf548h05fwpnank3i7-om-init-registry#registry' --quiet --quiet️
? Select a flake
rust-nix-template
> python-nix-template
haskell-flake
nixos-unified-template
haskell-template
[↑↓ to move, enter to select, type to filter]
> Select a flake python-nix-template
❄️ nix -j auto eval --json '/nix/store/d3v03rxgf691rbpqkclyawm29j2zvb54-source#om' --quiet --quiet️
Automatically choosing the one template available: python-nix-template
? Name of the Python package (kebab-case)
... Otherwise it seems to work as expected; however, How do you select a registry template non-interactively? It fails if I just pass |
Converting back to draft as it will be more natural to use a flat than hierarchical monorepo layout so that the template CI workflows will better function in either case of including just a single vs multiple packages in the monorepo. |
fdd4b27
to
15acc19
Compare
An approximation to python-nix-template test
rm -fr pnt-new && \
nix --accept-flake-config run github:juspay/omnix -- init github:sciexp/python-nix-template/main -o pnt-new --non-interactive --params '{
"package-name-kebab-case": "pnt-new",
"package-name-snake-case": "pnt_new",
"include-functional-package": false,
"git-org": "pnt-new",
"author": "PntNew",
"author-email": "[email protected]",
"include-vscode": true,
"include-github-ci": true,
"include-flake-template": false
}' && \
cd pnt-new && \
git init && \
git commit --allow-empty -m "initial commit (empty)" && \
git add . && \
nix run .#fix-template-names -- python-nix-template && \
nix run nixpkgs#uv -- lock && \
direnv allow ... After that completes then just run
now runs in the ci of python-nix-template. If you test it locally, note it should automatically recover from this error
error:
… while calling the 'derivationStrict' builtin
at <nix/derivation-internal.nix>:34:12:
33|
34| strict = derivationStrict drvAttrs;
| ^
35|
… while evaluating derivation 'pnt-new-3.12'
whose name attribute is located at /nix/store/vdyq3cl640n39sclkc8br4xgd09fdj2w-source/pkgs/stdenv/generic/make-derivation.nix:375:7
… while evaluating attribute 'UV_PYTHON' of derivation 'pnt-new-3.12'
(stack trace truncated; use '--show-trace' to show the full, detailed trace)
error: internal error: accessed dependencies from pyproject.nix project, not uv.lock |
No, what I meant was to create a minimal template that is still functional, as in I recommend taking a look at both The Haskell program is just hello world, ![]() Yet, it specifies some external library dependencies to demonstrate that they are readibly available in ![]() And the ![]() Does it make sense? Can you think of what the most minimal yet functional Python template would look like? |
This is what I was responding to and did indeed provide in what you get with the default parameters I've embedded in a one-liner in #424 (comment).
I hadn't thought about a separate justfile for the template, but can certainly figure out a way of removing the recipes that are only relevant to developing the template itself. There actually aren't nearly as many of those as you might think because I'm providing support for the two most widely used package management ecosystems in python via both uv and pixi. Of course some of the secrets management recipes could go.
yes this would mean providing another justfile that removes all the recipes that only pertain to developing the template repository itself will mark this as draft again until I have a chance to think about how to work toward what you're interested in. There might be a bit of a gap in time here as the current version https://github.com/sciexp/python-nix-template/tree/v0.1.0 works fairly well for our purposes aside from the workarounds for the omnix issues I have posted. Do let me know if any of those turn out to be the result of user error. Many thanks for your review as it will help to ensure we eventually arrive at a clean base template to include in the registry! |
let me know: If those tasks from the revised OP were completed as described, do you think we'd be significantly closer to synchronizing our understanding of what would be most useful? |
The CI https://github.com/juspay/omnix/actions/runs/13745303957/job/38449517038?pr=424 error
---- command::init::om_init stdout ----
🕍 Testing template: /nix/store/zdwrzskc47c7ic36jix50b3vpcb51k00-source
🕍 Testing template: /nix/store/vzvw6cy4gjcj1dp28z6pkf0qqycnp5h6-source
🧪 [/nix/store/vzvw6cy4gjcj1dp28z6pkf0qqycnp5h6-source#python-nix-template] Running test: default
PathAssert nix/modules/template.nix; exist? (false) in "/tmp/nix-shell.T43KED/.tmp4FhMxs/output"
PathAssert .vscode; exist? (true) in "/tmp/nix-shell.T43KED/.tmp4FhMxs/output"
PathAssert pyproject.toml; exist? (true) in "/tmp/nix-shell.T43KED/.tmp4FhMxs/output"
PathAssert flake.nix; exist? (true) in "/tmp/nix-shell.T43KED/.tmp4FhMxs/output"
PathAssert .github/workflows/ci.yaml; exist? (true) in "/tmp/nix-shell.T43KED/.tmp4FhMxs/output"
Error: Command error: Process exited unsuccessfully. exit_code=Some(1) stderr=warning: ignoring untrusted flake configuration setting 'extra-substituters'.
Pass '--accept-flake-config' to trust it
warning: ignoring untrusted flake configuration setting 'extra-trusted-public-keys'.
Pass '--accept-flake-config' to trust it
error (ignored): error: end of string reached
error:
… while calling the 'derivationStrict' builtin
at <nix/derivation-internal.nix>:34:12:
33|
34| strict = derivationStrict drvAttrs;
| ^
35|
… while evaluating derivation 'python-nix-template-3.12'
whose name attribute is located at /nix/store/vdyq3cl640n39sclkc8br4xgd09fdj2w-source/pkgs/stdenv/generic/make-derivation.nix:375:7
… while evaluating attribute 'NIX_PYPROJECT_DEPS' of derivation 'python-nix-template-3.12'
(stack trace truncated; use '--show-trace' to show the full, detailed trace)
error: path '/nix/store/i6dpfqq0lwivlbmw4047yv675ja50ss5-source/packages/pnt-functional' does not exist results from the package root
|
Yes, those tasks look good to me. I just initialized your template from that linked comment, by running:
This created the following layout: ![]()
Hopefully that gives you some concrete idea. I'd also suggest that you start from scratch, from a new empty repo and add files only as necessary, instead of trimming the existing one. |
For better or worse there are two package managers in the python ecosystem with widespread use: PyPI and Conda. All widely used tools release to both. uv, which builds PyPI packages, has functioning support for workspaces as mirrored in uv2nix and so it gets one lockfile in the repo root that applies to each of the packages in the Although pixi, which builds Conda packages, is working on mirroring uv's workspace support, it's not ironed out yet so each of the subdirectories of Within the conda ecosystem, there are 3 distinct and widely utilized clis. Specifically focusing on the question about two files here vs why they exist at all above
The two are because this is how uv workspaces are specified and instantiated https://docs.astral.sh/uv/concepts/projects/workspaces/ see uv2nix's test fixtures https://github.com/pyproject-nix/uv2nix/blob/61f6313f438656b57e653b96301fce14a7eed82e/lib/fixtures/workspace-flat/packages/pkg-a/pyproject.toml
The "release" key in the package.json's are one of the supported ways of providing a configuration file that contains what is required to automate release of each of the relevant versions of each of the monorepo packages using semantic-release in the python-nix-template github actions workflows to be included in the template: https://github.com/sciexp/python-nix-template/blob/7e21527def1cfd83cec1c6b7dc6acb3fc4ff084b/.github/workflows/package-release.yaml#L153-L170
Not sure what the questions is here. I suspect you think maybe the folder names are a bizarre duplication, but they are not. This is essentially mandated by the structure of a flat uv workspace as exemplified, for example, in this uv2nix test fixture: https://github.com/pyproject-nix/uv2nix/blob/61f6313f438656b57e653b96301fce14a7eed82e/lib/fixtures/workspace-flat/packages/pkg-a/src/pkg_a/__init__.py What you have copied above is the exact same form as
from uv2nix but applying to one of the two packages currently included in python-nix-template. If you are asking what an
We could, but my criterion for a successful template instantiation was being able to successfully execute
yes this would mean providing another justfile that removes the recipes that only pertain to developing the template repository itself; but more are required than you might imagine to clearly support both uv and pixi, which is considered a requirement for python projects that intend to make themselves available for wide adoption. it could probably easily be reduced to 35. 1 or 2 wouldn't make much sense, and you'll forgive me for getting the sense you're heading in the direction of hyperbole here.
Well, once
are resolved, one or two of them that serve as workaround shell applications could certainly go. We could discuss this further if some of the more pertinent issues above are resolved. The only particularly "non-standard" nix file aside from the two shell applications, |
I think a good fraction of your surprise at the apparent complexity of python-nix-template factors into two distinct categories:
If you don't do much to handle 1 for prospective users, then there's no point in yet another python template. As such, while we can certainly put in effort to reduce the complexity of issues arising from factor 2 (~30%), we won't be able to do much about factor 1 that doesn't significantly reduce the value of the prospective service python-nix-template intends to provide. On the other hand, we definitely would love to figure out as much of a middle-ground as possible to allow relatively less-seasoned users to slowly scale up the usage of the set of features python-nix-template offers, and, from this perspective, we hope to be able to find a reasonable balance here working with that 30% added complexity that we might have the latitude to better optimize without creating a template that is ONLY useful for those who want to try out "hello world" and leaves the people dealing with the complicated reality of the underlying ecosystem hanging out to dry. |
It's quite unlikely we'd be able to consider doing this. If you feel strongly about this, we should probably consider closing the PR for now and revisiting at some point in the future when that could be tenable. I'd give you some opportunity to think a bit more about what I've said above before responding further. It may be that a good approach to
could help achieve both goals so that the FIRST layout tree one receives in the simplest default template instantiation is MUCH simpler than what we currently have and the complexity could be toggled on feature-by-feature as needed. That's probably the ideal outcome in my view. |
I'll take a look at this later, but as quick response: omnix registry is rather nascent and is still evolving (#282, #336, #314). I'd like to support 3rd party registries1, so Footnotes
|
Yes, see the footnote of parent comment. The 3 templates are defined here, including tests for them: https://github.com/juspay/nixos-unified-template/blob/main/modules/flake/template.nix |
Yes I totally understand. Many thanks again for all of your work in the nix ecosystem and omnix itself. |
Oh yes. Of course, I have used this in my personal nix-config, but this was initially adapted around the time we were discussing juspay/nixos-unified-template#7 and before you had I certainly understand where you're coming from about having the default template for any language/framework in omnix-registry be quite minimal. I just want to be able to include the minimal template instance in a repo where people can quickly toggle on the additional features as desired because I know the vast majority of users will EVENTUALLY realize they need them or could benefit from them and nobody should actually be wasting their time sorting any of these things out multiple times in slightly different ways if they can at least take a quick look at one working approach via a feature toggle they can adapt as they need for their own purposes. |
Signed-off-by: Cameron Smith <[email protected]>
Signed-off-by: Cameron Smith <[email protected]>
15acc19
to
70402a6
Compare
om init
does not replace strings under matching directory name #425 inbased on #424 (comment)