pkgconf test pipeline failures

[dustinkirkland]

We have a handful (~135) packages in wolfi that should be able to successfully use our pkgconf pipeline, but for various reason are not passing those tests. See the attached branch, which adds pkgconf tests to packages which ship a pkgconf/*.pc file: https://github.com/dustinkirkland/wolfi-os/tree/pkgconf.failure

Most of these appear to be "real" test failures, in that various package dependencies are missing. There's perhaps some missing or faulty logic in our SCA?

[xnox]

unexpected items in failure basket. will check; in theory given the rolling openssf rebuilds these all should have fallen into place, will dig into what's wrong.

[dustinkirkland]

Here's a log of "make package/%" for each of the failing tests: https://gist.github.com/dustinkirkland/6f15a129fd8c2b75216dda6f487907b7

[murraybd]

@dustinkirkland This should fix it for some packages.

diff --git a/pipelines/test/pkgconf.yaml b/pipelines/test/pkgconf.yaml
index d36b0393c..6a1258265 100644
--- a/pipelines/test/pkgconf.yaml
+++ b/pipelines/test/pkgconf.yaml
@@ -16,7 +16,7 @@ pipeline:
         if grep -q "^Name:" $pc_file; then
           lib_name=$(basename "$pc_file" ".pc")
           # Ensure .pc file is installed in usr/lib/pkgconfig/*.pc
-          echo "$pc_file" | grep -q "^usr/lib/pkgconfig/${lib_name}.pc$"
+          echo "$pc_file" | grep -q "^usr/lib\|share/pkgconfig/${lib_name}.pc$"
           # Try to get the library name from the pc filename
           # Run some package config tests, which will likely be used by
           # other packages that have build dependencies on this dev package

[murraybd]

PR here #32701

[dustinkirkland]

Ooh....let me test that....

[dustinkirkland]

Okay, with that patch, these package tests now succeed:

• Catch2
• bash-completion
• cli11
• eigen
• eudev
• fish
• gi-docgen
• gtk-doc
• hicolor-icon-theme
• hwdata
• iso-codes
• kmod
• libbpf
• mimalloc
• mimalloc2
• nfs-ganesha
• nlohmann-json
• numactl
• samba
• scdoc
• shared-mime-info
• spirv-headers
• systemd
• util-macros
• wayland-protocols
• xbitmaps
• xcb-proto
• xkeyboard-config
• xorgproto
• xtrans
• yajl

[murraybd]

Some debugging of one of the failures:

/home/build # pkgconf --print-errors --exists glew; echo $?
Package osmesa was not found in the pkg-config search path.
Perhaps you should add the directory containing `osmesa.pc'
to the PKG_CONFIG_PATH environment variable
Package 'osmesa', required by 'glu', not found
1

An apk add of mesa-dev sorted that package at least.

[murraybd]

This PR adds some more output (--print-errors from above) to the pipeline so should be helpful.

#32715

[murraybd]

Here are some updated test results using the 2 changes to the pkgconf pipeline:

https://gist.github.com/murraybd/6608af12ad1d0010b446a52eb36a385e

[murraybd]

The following are the failing packages from the above gist:

• apr-util
• at-spi2-core
• audit - audit will fail the pkgconf test pipeline #37188
• brew - brew will fail pkgconf test pipeline #34006
• checkov - checkov will fail pkgconf test pipeline #34007
• flac
• freerdp
• gcc-6 - gcc-6 will fail pkgconf test pipeline #34009
• gi-docgen - gi-docgen will fail pkgconf test pipeline #34010
• glew
• glu
• graphene
• gtk-2.0
• highway
• kubeflow-katib - kubeflow-katib will fail pkgconf test pipeline #34320
• libassuan
• libatasmart
• libbpf - libbpf will fail pkgconf test pipeline #34322
• libeconf - libeconf will fail the pkgconf test pipeline #37189
• libepoxy
• libestr
• libfastjson
• libgdiplus
• libgeotiff
• libical
• libice
• libinput
• libksba
• liblangtag
• libmemcached - libmemcached will fail pkgconf test pipeline #34332
• librdkafka
• librelp
• libslirp
• libsm
• libsndfile
• libsrt
• libtracefs
• libva
• libvterm - libvterm will fail the pkgconf test pipeline #37190
• libxaw
• libxcomposite
• libxcursor
• libxdmcp
• libxext
• libxfixes
• libxft
• libxi
• libxinerama
• libxkb
• libxmu
• libxrandr
• libxrender
• libxscrnsaver
• libxss
• libxt
• libxtst
• metric-collector-for-apache-cassandra-4.1- metric-collector-for-apache-cassandra-4.1 will fail pkgconf test pipeline #34356
• mimalloc2
• mlflow - mlflow will fail pkgconf test pipeline #34358
• multipath-tools
• neon - neon will fail pkgconf test pipeline #34360
• nfs-ganesha - nfs-ganesha will fail pkgconf test pipeline #34361
• ninja-build - ninja-build will fail pkgconf test pipeline #34362
• openexr
• openipmi
• portaudio
• procps - procps will fail pkgconf test pipeline #34364
• proj
• protobuf - protobuf will fail pkgconf test pipeline #34366
• pulseaudio
• py3-cairo
• py3-sentencepiece - py3-sentencepiece will fail pkgconf test pipeline #34369
• py3.10-tensorflow-core - py3.10-tensorflow-core will fail pkgconf test pipeline #34367
• qpdf - qpdf will fail pkgconf test pipeline #34370
• re2 - re2 will fail pkgconf test pipeline #34371
• reflex - reflex will fail pkgconf test pipeline #34372
• rpm
• rtmpdump - rtmpdump will fail the pkgconf test pipeline #37192
• subversion
• tcl
• tesseract
• tevent
• tk
• woff2
• wv
• xmlsec
• yara
• zfs

[kaniini]

Would it be useful if pkgconf had a machine readable version of —print-errors?

[dustinkirkland]

Machine readable is fine....but I'd find more useful, a handy, definitive error message that explains what exactly is wrong and how to fix it, @kaniini

[kaniini]

I can look into improving the error messages. Originally, we used the same error messages as the original freedesktop.org pkg-config for compatibility, but have deviated in the name of usability in other error reporting areas already.

[sanjayk0508]

The following are the failing packages from the above gist:

• apr-util
• at-spi2-core
• audit - installs .pc files in /usr/local/lib
• brew
• checkov
• flac
• freerdp
• gcc-6
• gi-docgen - installs .pc files in /usr/local/share
• glew
• glu
• graphene
• gtk-2.0
• highway
• kubeflow-katib
• MORE

Hi @murraybd, I’m planning to work on fixing a few failing packages from the recent issues. I noticed that several packages are failing due to missing dependencies or .pc files for required libraries like libcrypto, libxml-2.0, and libgsf-1. Could you provide some guidance on how to address these missing dependencies in the test environment? Specifically, are we supposed to update the apk dependencies, adjust the PKG_CONFIG_PATH, or modify the pipeline to handle missing .pc files more gracefully? Any examples or steps to properly configure the environment and resolve these issues would be really helpful for getting started.

[murraybd]

Hi @murraybd, I’m planning to work on fixing a few failing packages from the recent issues. I noticed that several packages are failing due to missing dependencies or .pc files for required libraries like libcrypto, libxml-2.0, and libgsf-1. Could you provide some guidance on how to address these missing dependencies in the test environment? Specifically, are we supposed to update the apk dependencies, adjust the PKG_CONFIG_PATH, or modify the pipeline to handle missing .pc files more gracefully? Any examples or steps to properly configure the environment and resolve these issues would be really helpful for getting started.

The missing dependencies should be tracked down and added to the apk dependencies, so for example I looked at apr-util and added libapr-dev to the runtime dependencies of apr-util-dev and rebuilt the package ran the test and then pkgconf was happy.

#34655

However, I expect there to be some dependency chains here e.g. libxrandr-dev fails due to xrender being missing and then make test/xrender with the pkgconf pipeline added fails.

[stevebeattie]

• libbpf - libbpf will fail pkgconf test pipeline #34322

For libbpf, this is because the libraries and .pc file end up packaged in /usr/lib64, not /usr/lib/; not sure if there are any others for which this is the case.

[dustinkirkland]

Almost certainly there are others... I can run a query for you and confirm.

…

On Thu, Jan 2, 2025, 16:11 Steve Beattie ***@***.***> wrote: - libbpf - libbpf will fail pkgconf test pipeline #34322 <#34322> For libbpf, this is because the libraries and .pc file end up packaged in /usr/lib64, not /usr/lib/; not sure if there are any others for which this is the case. — Reply to this email directly, view it on GitHub <#32554 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGMZBQTRA5KXPJUJKJY4Y32IW2STAVCNFSM6AAAAABQ6BSJB6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRYGQ2DINRXHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[murraybd]

• libbpf - libbpf will fail pkgconf test pipeline #34322

For libbpf, this is because the libraries and .pc file end up packaged in /usr/lib64, not /usr/lib/; not sure if there are any others for which this is the case.

All the issues I opened should contain log excerpts of the failures and searching github issues might be faster than searching the whole archive.