You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
deploy/pvm: fix PVM kernel build on modern distros & make pvm_setup.sh re-runnable / single-sided (#112)
* deploy: fix "lz4c: command not found" when building PVM kernels
The pvm_host config enables CONFIG_KERNEL_LZ4=y, and both pvm_host and
pvm_guest enable CONFIG_RD_LZ4=y / CONFIG_HAVE_KERNEL_LZ4=y, which makes
the kernel build invoke `lz4c` at link/packaging time. Modern
distributions (recent Debian/Ubuntu, TencentOS/OpenCloudOS, etc.) ship
only the `lz4` binary in their `lz4` package and no longer provide the
legacy `lz4c` alias, so `build-pvm-host-kernel-pkg.sh` and
`build-pvm-guest-vmlinux.sh` fail with:
/bin/sh: lz4c: command not found
Teach deploy/pvm/common.sh to guarantee `lz4c` availability:
- Add the `lz4` package to install_deps_rpm / install_deps_deb and to
the ensure_build_tools bootstrap lists (dnf/yum, apt-get, zypper),
so normal runs pick it up automatically.
- Introduce ensure_lz4c(): if `lz4c` is still missing after the
package install (or when SKIP_DEPS=1), fall back to installing the
`lz4` package directly and, as a last resort, drop a tiny
/usr/local/bin/lz4c shim that execs the modern `lz4` binary. The
shim keeps the kernel Makefiles working without patching the
upstream source tree.
- Call ensure_lz4c() at the end of ensure_build_tools() so both
entry-point scripts (host package build and guest vmlinux build)
pick up the fix with no changes of their own.
Assisted-by: Anthropic:claude-opus-4-7
Signed-off-by: Like Xu <likexu@tencent.com>
* deploy/pvm: install python3 so kernel libbpf builds don't fail with Error 127
Fresh RPM/DEB minimal environments (TencentOS Server 3.x, Debian 12
cloud, etc.) do not ship a python3 interpreter by default. When
pvm_setup.sh drives the host/guest kernel builds on such a machine, the
kernel's tools/bpf build aborts very early with a misleading error:
fi; install -m 644 libbpf_legacy.h '...'/include/bpf'
make[8]: *** [Makefile:160: .../libbpf/bpf_helper_defs.h] Error 127
make[7]: *** [Makefile:63: .../libbpf/libbpf.a] Error 2
make[6]: *** [Makefile:77: bpf/resolve_btfids] Error 2
make[5]: *** [Makefile:1376: tools/bpf/resolve_btfids] Error 2
make[3]: *** [debian/rules:25: build-arch] Error 2
The failing rule is tools/lib/bpf/Makefile:160, which generates
bpf_helper_defs.h by invoking scripts/bpf_doc.py (shebang:
"#!/usr/bin/env python3"). Error 127 means "command not found" and
refers to the python3 interpreter, not to the `install` line that
make happens to print right before it. Missing python3 then cascades
into libbpf.a -> resolve_btfids -> vmlinux -> bindeb-pkg failures.
Add python3 to every path that provisions build dependencies so the
issue is fixed automatically on a fresh run, and also recovered from
when users bypass the top-level installer with SKIP_DEPS=1:
- common.sh: add python3 to install_deps_rpm / install_deps_deb.
- common.sh: ensure_build_tools() now detects a missing python3
(with a comment explaining the misleading Error 127) and adds it
to the dnf/yum, apt-get and zypper package lists used to bootstrap
the toolchain.
- pvm_setup.sh: add python3 to install_common_build_deps() for both
the RPM and DEB branches, so the one-shot setup pre-installs it
before the host and guest builds fan out in parallel.
No other behaviour changes; existing callers, env overrides and the
lz4/lz4c shim handling are untouched.
Assisted-by: Anthropic:claude-opus-4-7
Signed-off-by: Like Xu <likexu@tencent.com>
* deploy/pvm: make re-runs robust when BRANCH is a tag or a commit SHA
clone_source() in deploy/pvm/common.sh silently assumes BRANCH is a
branch name: after fetching, it dereferences origin/<BRANCH>. Since
the kernel baseline was pinned to a release tag, a shallow tag fetch
only advances FETCH_HEAD and creates no origin/<tag> ref, so the
second (and every subsequent) run of pvm_setup.sh aborts inside the
reset step, leaving the source tree in an indeterminate state for
the rest of the build. The first run usually succeeds, so the
failure only shows up on re-runs and looks like flaky behaviour.
Make the update path treat BRANCH as an opaque git ref -- branch,
tag, or bare commit SHA -- so re-runs always converge to the same
commit regardless of what kind of ref the caller configured, and
fail loudly with an actionable message when the ref cannot be
resolved at all.
Known limitations:
- The fresh-clone path still goes through `git clone --branch`,
which does not accept a bare commit SHA. BRANCH must therefore
resolve to a named ref (branch or tag) on the first run; bare
SHAs are only supported for subsequent updates of an existing
tree.
- The update path re-shallows on top of the existing clone.
History older than the pinned commit is not reachable locally,
which is intentional for build speed but means `git log` /
`git bisect` inside ${SRC_DIR} are of limited use.
- Switching REPO_URL between runs rewrites `origin` in place; any
local changes or untracked files under ${SRC_DIR} are discarded
by design to guarantee the tree matches the requested ref.
Assisted-by: Anthropic:claude-opus-4-7
Signed-off-by: Like Xu <likexu@tencent.com>
* deploy/pvm: let pvm_setup.sh build only the guest or only the host
pvm_setup.sh has been strictly all-or-nothing: it always rebuilds
both the pvm-host package and the pvm-guest vmlinux, then installs
the host package, touches GRUB, wires up kvm_pvm auto-load and
nudges the user to reboot. That shape is correct for a first-time
bring-up but wrong for two common day-2 scenarios:
* Refresh only the guest vmlinux on a node that is already
running the desired pvm-host kernel. Rebuilding and reinstalling
the host kernel in that case is at best noise and at worst
dangerous, since the trailing REBOOT banner can trick an
operator into rebooting a production node unnecessarily.
* Rebuild only the pvm-host package on a node whose guest vmlinux
is pinned and must not be overwritten, e.g. when validating a
host-side kernel fix against a fixed guest image.
Add first-class support for both modes through symmetric env vars
and CLI flags, keeping the default (build both sides in parallel,
install, place, prompt reboot) bit-for-bit unchanged. The end-of-run
summary also adapts to the guest-only case so the reboot / GRUB
cmdline reminder is suppressed when the running kernel was not
touched. Per-side build logs are appended rather than overwritten,
and single-sided runs additionally tee pvm_setup.sh's own output
into the matching log so the file is a faithful transcript of the
run; a per-run banner makes successive runs easy to tell apart.
Known limitations:
- The two skip knobs are mutually exclusive in practice: asking
to skip both sides is accepted but degenerates into a no-op
warning, since there is nothing left to do.
- Guest-only runs deliberately do not touch GRUB_DEFAULT or
/etc/modules-load.d/kvm_pvm.conf. A node that has never had the
pvm-host package installed cannot be brought up with
--guest-only alone; a prior full or --host-only run is required.
- Host-only runs skip the placement step entirely, so the in-repo
assets/ and cubetoolbox runtime vmlinux are left at whatever
version a previous run (or manual copy) put there. Callers that
need a matching guest image must run the guest side explicitly.
- Log files are append-only from this script; there is no built-in
rotation. Long-lived development hosts should rotate or truncate
pvm-setup-{host,guest}-build.log out of band.
- The redirect that captures pvm_setup.sh's own output into the
per-side log is only installed for single-sided runs. A default
(parallel) run still prints to the terminal only; its per-side
build logs continue to live in the two build-dir log files as
before.
Assisted-by: Anthropic:claude-opus-4-7
Signed-off-by: Like Xu <likexu@tencent.com>
* deploy/pvm: pin OpenCloudOS kernel builds to tag 6.6.69-1.cubesandbox
The host/guest kernel build scripts currently clone the OpenCloudOS
kernel tree at the moving branch linux-6.6/cube/pvm. Because that branch
advances over time, consecutive runs of build-pvm-host-kernel-pkg.sh or
build-pvm-guest-vmlinux.sh can pick up different source trees and
produce non-reproducible artifacts; it also makes it harder to reason
about which exact kernel a given CubeSandbox release was built against,
and to match the in-tree pvm_host / pvm_guest .config files (which were
derived from a specific snapshot) to the sources they were generated
from.
Pin both scripts, and the configs/README.md pointer that documents
their provenance, to the released tag 6.6.69-1.cubesandbox instead of
the branch. The default REPO_URL is unchanged, and BRANCH is still
overridable via the environment for developers who need to track the
tip of linux-6.6/cube/pvm.
- build-pvm-host-kernel-pkg.sh: default BRANCH -> 6.6.69-1.cubesandbox
- build-pvm-guest-vmlinux.sh: default BRANCH -> 6.6.69-1.cubesandbox
- configs/README.md: document the tag as the config provenance
Signed-off-by: Like Xu <likexu@tencent.com>
---------
Signed-off-by: Like Xu <likexu@tencent.com>
Co-authored-by: Like Xu <likexu@tencent.com>
0 commit comments