partial Merge bitcoin#28083: ci: Use DOCKER_BUILDKIT for lint image

fa2f18a ci: Use DOCKER_BUILDKIT for lint image (MarcoFalke)

Pull request description:

  Currently the lint docker/podman image has many issues:

  * It relies on an EOL debian version.
  * It relies on a debian version different from the one used in the CI lint task.
  * It relies on the legacy docker build command, which requires the user to make `cd ./ci/lint/` before the build step.
  * It doesn't use the `.python-version` file, but a hardcoded version.

  Fix all issues by using the recommended `DOCKER_BUILDKIT=1` to generate the image.

  Also:
  * Rename `/tmp/python` to `/python_build`.
  * Compress all `pip install` commands into one.
  * Bump `.python-version`.

ACKs for top commit:
  jamesob:
    ACK bitcoin@fa2f18a

Tree-SHA512: 804b384904ad753845667998841cc7825f4229933ca2c42af021384713486ec3cca80ba58612d37557fba7ee1921439dacca5e1236aac0557dd75bd9a2f1875d

Dash adaptations:
- ci/lint/04_install.sh: adopted upstream's single multi-line `pip3 install` invocation but kept Dash's pinned versions (codespell 2.2.1, flake8 5.0.4, mypy 0.981, pyzmq 24.0.1) and Dash's `# NOTE: BUMP ALSO contrib/containers/ci/ci-slim.Dockerfile` comment; upstream's version bumps come from a separate commit Dash has not backported and are out of scope here.
- .python-version: left at Dash's 3.10.19 rather than taking upstream's 3.8.16->3.8.17 bump; Dash is already ahead of upstream on this file.
- ci/lint/Dockerfile: Dash's copy differed from upstream (FROM python:3.10.19-bookworm instead of python:3.8-buster), but upstream deletes the file wholesale in favour of ci/lint_imagefile, so the Dash variant was removed too.
- ci/lint_imagefile: taken verbatim from upstream, including `WORKDIR /bitcoin`; Dash's existing lint tooling (docker-entrypoint.sh safe.directory, test/lint/README.md mount path and `bitcoin-linter` tag) already uses /bitcoin, and upstream's commit does not rename it, so no rebranding was applied.

Omitted (partial backport):
- .cirrus.yml: Dash deleted .cirrus.yml from its tree (it uses GitHub Actions / GitLab CI, not Cirrus). The cherry-pick re-introduced upstream's whole file as a delete/modify conflict; resolved by keeping it deleted, so the python_cache folder change /tmp/python -> /python_build has no home in Dash.

Author: DCG-Claude

ci/lint/04_install.sh

@@ -13,33 +13,32 @@ ${CI_RETRY_EXE} apt-get update
 # - gpg (used by verify-commits)
 ${CI_RETRY_EXE} apt-get install -y curl xz-utils git gpg
 
-if [ -z "${SKIP_PYTHON_INSTALL}" ]; then
-    PYTHON_PATH=/tmp/python
-    if [ ! -d "${PYTHON_PATH}/bin" ]; then
-      (
-        git clone https://github.com/pyenv/pyenv.git
-        cd pyenv/plugins/python-build || exit 1
-        ./install.sh
-      )
-      # For dependencies see https://github.com/pyenv/pyenv/wiki#suggested-build-environment
-      ${CI_RETRY_EXE} apt-get install -y build-essential libssl-dev zlib1g-dev \
-        libbz2-dev libreadline-dev libsqlite3-dev curl llvm \
-        libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
-        clang
-      env CC=clang python-build "$(cat "${BASE_ROOT_DIR}/.python-version")" "${PYTHON_PATH}"
-    fi
-    export PATH="${PYTHON_PATH}/bin:${PATH}"
-    command -v python3
-    python3 --version
+PYTHON_PATH="/python_build"
+if [ ! -d "${PYTHON_PATH}/bin" ]; then
+  (
+    git clone https://github.com/pyenv/pyenv.git
+    cd pyenv/plugins/python-build || exit 1
+    ./install.sh
+  )
+  # For dependencies see https://github.com/pyenv/pyenv/wiki#suggested-build-environment
+  ${CI_RETRY_EXE} apt-get install -y build-essential libssl-dev zlib1g-dev \
+    libbz2-dev libreadline-dev libsqlite3-dev curl llvm \
+    libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
+    clang
+  env CC=clang python-build "$(cat "./.python-version")" "${PYTHON_PATH}"
 fi
+export PATH="${PYTHON_PATH}/bin:${PATH}"
+command -v python3
+python3 --version
 
 # NOTE: BUMP ALSO contrib/containers/ci/ci-slim.Dockerfile
-${CI_RETRY_EXE} pip3 install codespell==2.2.1
-${CI_RETRY_EXE} pip3 install flake8==5.0.4
-${CI_RETRY_EXE} pip3 install lief==0.13.2
-${CI_RETRY_EXE} pip3 install mypy==0.981
-${CI_RETRY_EXE} pip3 install pyzmq==24.0.1
-${CI_RETRY_EXE} pip3 install vulture==2.6
+${CI_RETRY_EXE} pip3 install \
+  codespell==2.2.1 \
+  flake8==5.0.4 \
+  lief==0.13.2 \
+  mypy==0.981 \
+  pyzmq==24.0.1 \
+  vulture==2.6
 
 SHELLCHECK_VERSION=v0.8.0
 curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | \

ci/lint/Dockerfile

@@ -5,6 +5,8 @@ export LC_ALL=C
 # of the mounted bitcoin src dir.
 git config --global --add safe.directory /bitcoin
 
+export PATH="/python_build/bin:${PATH}"
+
 if [ -z "$1" ]; then
   LOCAL_BRANCH=1 bash -ic "./ci/lint/06_script.sh"
 else

ci/lint/docker-entrypoint.sh

@@ -0,0 +1,19 @@
+# See test/lint/README.md for usage.
+
+FROM debian:bookworm
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LC_ALL=C.UTF-8
+
+COPY ./.python-version /.python-version
+COPY ./ci/lint/docker-entrypoint.sh /entrypoint.sh
+COPY ./ci/lint/04_install.sh /install.sh
+
+RUN /install.sh && \
+  echo 'alias lint="./ci/lint/06_script.sh"' >> ~/.bashrc && \
+  chmod 755 /entrypoint.sh && \
+  rm -rf /var/lib/apt/lists/*
+
+
+WORKDIR /bitcoin
+ENTRYPOINT ["/entrypoint.sh"]

ci/lint_imagefile

@@ -7,10 +7,8 @@ To run linters locally with the same versions as the CI environment, use the inc
 Dockerfile:
 
 ```sh
-cd ./ci/lint
-docker build -t bitcoin-linter .
+DOCKER_BUILDKIT=1 docker build -t bitcoin-linter --file "./ci/lint_imagefile" ./
 
-cd /root/of/bitcoin/repo
 docker run --rm -v $(pwd):/bitcoin -it bitcoin-linter
 ```