forcing autoinstall to rebuild all modules

[anbe42]

Writing with my Debian hat on ...

Whenever I upgrade a linux-headers-* package (and it does not change its name), I'd like to rebuild all dkms modules against these headers. Assuming that the ABI is stable if the ABI-version part in the package name (and kernel version) did not change is not a safe bet and the cases where the it actually causes problems are the nasty ones. (Also for kernels in experimental that always use an ABI version of 0, this assumption is outright wrong and modules must be rebuilt on upgrade.)

But dkms currently finds an old build of matching module/modversion/kvers/arch and keeps that. How can I best get around that? The relevant trigger would be that one activated from /etc/kernel/header_postinst.d/dkms.

Debian kernel maintainers are working on improving the situation, e.g. bumping the version in the package name much more often s.t. such required re-build events should be much less frequent (as there would be much more new-build events).

[evelikov]

Last time I went dumpster diving ... err through git log ... I wasn't able to find concrete details behind the current original_module handling (and it's respective collisions).

I haven't gone through a great deal of exploring all the possible corner-cases, although my initial inclination is that those can go, especially since it seems to be disabled on Debian/Ubuntu.

At a glance, the original_modules idea falls short:

• does not consider header/kernel package version updates, like the ones mentioned
• explicit end-user config changes - eg. openzfs runs configure doing it's PRE_BUILD hook that reads end-user settings in /etc...
• implicit end-user environment changes - eg. installed another gcc/ld/etc

Or in general: non-unique/reproducible environment and module binary. Doesn't seem like we capture enough information about the environment used, hence one cannot create a reproducible binary. Conversely we assume that a given module name (or path really) is unique where in practise the binary can vary greatly.

IMHO removing said code will be a great first step in the right direction.

The second step is to unwrap the current "force" handling. Namely: we have the --force CLI only option, the undocumented --force-version-override CLI only option (commit 9bbef17) and completely undocumented /usr/share/dkms/modules_to_force_install location (commit a929d05). The exact how we address this depends on distributions:

• what level of granularity are you interested in - force individual module(s), only certain version(s) of the module or global for all module/versions
• how is the force config handled - via the dkms, module or a third-party package

@anbe42 any preferences on the above?

@xnox considering modules_to_force_install and force-version-override were added by colleagues, do you have any pointers how Canonical is using them? Similarly, would removing the original_module handling affect your work? Seemingly it's not applicable on Debian/Ubuntu platforms, although I could be wrong ;-)

@scaronni do you recall any of the historical reasons behind original_module? Would removing it affect any of the packages/UX that you maintain?

[scaronni]

I think 99% of the people either use dkms in a very basic way (add, build, install and remove) of non-existing modules, or they go on the convoluted way of embedding scripts in the configuration files (for example https://github.com/openzfs/zfs/blob/master/scripts/dkms.mkconf and https://github.com/umlaeute/v4l2loopback/blob/main/dkms.conf).

I fall in the first category :)

Regarding the original_module part, that should be needed in case you need to replace running modules with newer versions from DKMS packages. The AMD provided drivers do that: https://www.amd.com/en/support/linux-drivers

[anbe42]

After a sucessful

dkms autoinstall -k 6.7.8.9

I'd like to undo the effect of that autoinstall, e.g. (pseudocode)

dkms unbuild -k 6.7.8.9 --all-autoinstall-modules

(i.e. preserve any manually built/installed AUTOBUILD="" modules) such that a subsequent

dkms autoinstall -k 6.7.8.9

would restore the state after the first autoinstall, but with freshly built modules (since we e.g. changed CONFIG_FOO="bar" to CONFIG_FOO="baz" inbetween).

This is not really related to the handling of "original modules".

[anbe42]

I'm going to try this in kernel_postinst.d_dkms.in, right before calling dkms_autoinstaller:

case $0 in *header_postinst.d*)
    # unbuild all autoinstalled modules for this kernel to ensure they get
    # rebuilt against the updated headers by the next autoinstall below
    for mod_ver in $(dkms status -k "$inst_kern" 2>/dev/null | grep ': installed' | cut -d, -f1 | sort -u)
    do  
        dkms_conf="/var/lib/dkms/$mod_ver/source/dkms.conf"
        AUTOINSTALL=
        autoinstall=$(. "$dkms_conf" >/dev/null 2>&1; echo $AUTOINSTALL)
        test -n "$autoinstall" || continue
        dkms unbuild -k "$inst_kern" "$mod_ver"
    done
    ;;
esac

and then on Debian a rebuild of all modules for a single kernel can be triggered with dpkg-reconfigure linux-headers-6.5.0-5-amd64

[evelikov]

Handling this in the distro/packaging hooks makes sense. On thing to perhaps consider: how is this going to work if both kernel and module get updated in the same update cycle. Especially if the AUTOINSTALL value varies across the dkms module update.

Fwiw the Arch hook removes all old ones and rebuilds/installs them regardless of the AUTOINSTALL flag.

[anbe42]

dkms autoinstall on linux-headers-$KVER/linux-image-$KVER upgrade is usually a noop since the modules are already installed. This would only change if foo-dkms was upgraded at the same time.
So if both packages get upgraded in the same run (the expectedly common case), dkms autoinstall is run twice anyway and it does not really matter if we inject the unbuild-all -k $KVERS before the first or second. We do not do it in both cases.

In the sane world $KVER is bumped every time and there are no upgrades but new installations (and some time later removals) that have to build (delete) all modules anyway.

I'm not exactly sure what is the expected behavior of dkms w.r.t. the AUTOINSTALL="" (i.e. the default) setting in dkms.conf. (In parentheses I have the current (3.0.12-2 in experimental/NEW) behavior in Debian. In Debian, there can be only one version of a certain dkms package be installed at the same time.)

• on installation of foo-dkms (do not build/install)
• on upgrade of foo-dkms (remove from all kernels, do not build/install)
• on removal of foo-dkms (remove from all kernels)
• on installation of linux-$KVERS (do not build/install)
• on upgrade of linux-$KVERS (do not touch)
• on removal of linux-$KVERS (remove)

I.e. we never build/install it and remove it only if it is unavoidable.