Manifest generation seems stuck possibly because current generation code breaks

[ardumont]

The README mentions "The nix-community Buildkite CI generates it each day by picking a
Nixpkgs commit built by Hydra". But the last commit referenced from the main page [1]
dates back from october 11th 2021 [2].

Trying to trigger locally the generation ends up with failures. Be it testing with the
"unstable" release [3] or the current latest release "22.05" [4].

I was hoping to have a look and understand how to fix either #1 or #4 but i'm stuck now.

Any idea how to fix this, please?

Cheers,

[1] https://nix-community.github.io/nixpkgs-swh/

[2] https://github.com/NixOS/nixpkgs/tree/e4ef597edfd8a0ba5f12362932fc9b1dd01a0aef

[3]

$ git clone https://github.com/nix-community/nixpkgs-swh && cd nixpkgs-swh
$ ./scripts/generate.sh build/ unstable
...
substitution of path '/nix/store/r2jd6ygnmirm2g803mksqqjm4y39yi6i-git-2.33.1' succeeded
** Generate sources.json and README for release unstable
*** Query Hydra to get latest build commit
*** Generate sources-unstable.json for commit b7a6fde153d9470afdb6aa1da51c4117f03b84ed ...
warning: unknown setting 'experimental-features'
unpacking 'https://github.com/NixOS/nixpkgs/archive/b7a6fde153d9470afdb6aa1da51c4117f03b84ed.tar.gz'...
trace: lib.crossLists is deprecated, use lib.cartesianProductOfSets instead
trace: warning: literalExample is deprecated, use literalExpression instead, or use literalDocBook for a non-Nix description.
trace: warning: lib.mkFixStrictness has no effect and will be removed. It returns its argument unmodified, so you can just remove any calls.
trace: `lib.nixpkgsVersion` is deprecated, use `lib.version` instead!
trace: warning: lib.readPathsFromFile is deprecated, use a list instead
trace: Warning: `showVal` is deprecated and will be removed in the next release, please use `traceSeqN`
trace: lib.zip is deprecated, use lib.zipAttrsWith instead
error: undefined variable 'ocamlNativeNuildInputs' at /nix/store/1plbhwlwyr223l7cc0l08bzp064ar70l-b7a6fde153d9470afdb6aa1da51c4117f03b84ed.tar.gz/pkgs/applications/science/logic/coq/default.nix:86:15
(use '--show-trace' to show detailed location information)

real    2m48.312s
user    2m54.012s
sys     0m9.243s

[4]

$ git clone https://github.com/nix-community/nixpkgs-swh && cd nixpkgs-swh
$ ./scripts/generate.sh build/ 22.05
** Generate sources.json and README for release 22.05
*** Query Hydra to get latest build commit
*** Generate sources-22.05.json for commit fe76645aaf2fac3baaa2813fd0089930689c53b5 ...
warning: unknown setting 'experimental-features'
unpacking 'https://github.com/NixOS/nixpkgs/archive/fe76645aaf2fac3baaa2813fd0089930689c53b5.tar.gz'...
trace: lib.crossLists is deprecated, use lib.cartesianProductOfSets instead
trace: warning: literalExample is deprecated, use literalExpression instead, or use literalDocBook for a non-Nix description.
trace: warning: lib.mkFixStrictness has no effect and will be removed. It returns its argument unmodified, so you can just remove any calls.
trace: `lib.nixpkgsVersion` is deprecated, use `lib.version` instead!
trace: warning: lib.readPathsFromFile is deprecated, use a list instead
trace: Warning: `showVal` is deprecated and will be removed in the next release, please use `traceSeqN`
trace: lib.zip is deprecated, use lib.zipAttrsWith instead
trace: warning: `stdenv.glibc` is deprecated and will be removed in release 22.11. Please use `pkgs.glibc` instead.
trace: warning: The isXen attribute is deprecated. All Nixpkgs kernels that support it now have Xen enabled.
trace: warning: override is stubbed for NixOS kernel tests, not applying changes these arguments: features kernelPatches randstructSeed

error: attribute 'ELK-7' missing, at /nix/store/jyra8v8bpnddgqkksjma20pl41v8rvmc-fe76645aaf2fac3baaa2813fd0089930689c53b5.tar.gz/pkgs/tools/misc/logstash/7.x.nix:77:32
(use '--show-trace' to show detailed location information)

real    3m5.046s
user    3m25.098s
sys     0m9.861s

[ardumont]

Reading this again, I see something that i missed last time.

First, the /nix/sources.json is a bit old. But that's not really a problem. I mentioned
it since I got side-tracked by this (it's only for the current tooling and not regarding
the nixpkgs builds).

The actual problems are that current upstream builds are failing (unstable [1], 22.05
[2], 21.11 [3] ...). The routine generate.sh checks the latest hydra build and
retrieves nixpkgs commits which hold error of one sort [1] or another [2] [3].

It'd be nice if the hydra query [4] could actually filter on the evaluation build status
to only retain the last successful build (because i assumed that build would be
currently in failure? [5]).

Any thoughts on that ^?

[1] On unstable, commit 2001e2b31c565bcdf7bc13062b8d7cfccaca05b8, there is a typoed
variable: ocamlNativeNuildInputs instead of ocamlNativeBuildInputs in the coq
manifest...

error: undefined variable 'ocamlNativeNuildInputs' at /nix/store/gi7b5i0ldwcmylv7dffhmvyx6bgnlqkv-2001e2b31c565bcdf7bc13062b8d7cfccaca05b8.tar.gz/pkgs/applications/science/logic/coq/default.nix:89:15

[2] For the release-22.05, commit c132d0837dfb9035701dcd8fc91786c605c855c3, it's yet
another error about a missing ELK-7 attribute missing (or somesuch in earlier log
extract here).

error: attribute 'ELK-7' missing, at /nix/store/fa4v6l20yyikakkwkn7rj6kyhricj68h-c132d0837dfb9035701dcd8fc91786c605c855c3.tar.gz/pkgs/tools/misc/logstash/7.x.nix:77:32

[3] Same goes for release-21.11, commit eabc38219184cc3e04a974fe31857d8e0eac098d

error: attribute 'ELK-7' missing, at /nix/store/fmlbjz4zm5brb0n21mksd20mg68l16jq-eabc38219184cc3e04a974fe31857d8e0eac098d.tar.gz/pkgs/tools/misc/logstash/7.x.nix:74:32
(use '--show-trace' to show detailed location information)

[4] https://github.com/nix-community/nixpkgs-swh/blob/master/scripts/generate.sh#L19-L22

[5] It seems to be in failure in all versions i've checked afaiui...

https://hydra.nixos.org/jobset/nixos/release-21.11/evals?page=1
https://hydra.nixos.org/jobset/nixos/release-22.05/evals?page=1
https://hydra.nixos.org/jobset/nixos/trunk-combined/evals?page=1

Cheers,

[Artturin]

fixed those 2 eval errors, lmk if there's more

[LeSuisse]

I started to open a few PRs in order to fix the eval issues:

• freshrss: fix passthru.tests NixOS/nixpkgs#214963
• coq: remove undefined attribute ocamlPropagatedNativeBuildInputs in passthru NixOS/nixpkgs#214965
• metricbeat7: fix passthru.tests NixOS/nixpkgs#214969

Also noted the following issues:

• nixopsUnstable.passthru.tests https://github.com/NixOS/nixpkgs/blob/50981b1486c6fea1892f4e1f4e92e9c85f9ede93/pkgs/applications/networking/cluster/nixops/default.nix#L89
• linux.tests https://github.com/NixOS/nixpkgs/blob/5e32ae261893eab62d7ddc064cccf37e8eb1fd9a/pkgs/os-specific/linux/kernel/generic.nix#L227
• dotnet-aspnetcore.passthru.packages https://github.com/NixOS/nixpkgs/blob/50981b1486c6fea1892f4e1f4e92e9c85f9ede93/pkgs/development/compilers/dotnet/build-dotnet.nix#L131

[nlewo]

@ardumont Sorry, I postponed to answer to your issue... 😕

@LeSuisse @Artturin Thanks a lot for your PRs. I'm starting to review them and will have more time on next week (holidays!).