Added a local test matrix for Ubuntu 22, 24, and 26, and Windows 11

test/compat/Dockerfile.ubuntu

@@ -0,0 +1,30 @@
+# Parametrized image for the Ubuntu LTS legs of the compatibility matrix.
+# Build with: docker build --build-arg BASE=ubuntu:24.04 -f Dockerfile.ubuntu .
+ARG BASE=ubuntu:22.04
+FROM ${BASE}
+
+ENV DEBIAN_FRONTEND=noninteractive
+# Python >= 3.12 (Ubuntu 24.04+) removed distutils from stdlib; setuptools
+# provides the shim, but only in "local" mode (its vendored copy).
+ENV SETUPTOOLS_USE_DISTUTILS=local
+
+# Everything the app needs at runtime comes from the OS package manager
+# (numpy, Pillow, networkx, requests) per the "use what the OS provides" rule.
+# Only build tooling and the Python dev headers are added on top.
+RUN apt-get update && apt-get install -y --no-install-recommends \
+        python3 \
+        python3-dev \
+        python3-setuptools \
+        python3-numpy \
+        python3-pil \
+        python3-networkx \
+        python3-requests \
+        build-essential \
+        rsync \
+        ca-certificates \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY entrypoint.sh /entrypoint.sh
+RUN sed -i 's/\r$//' /entrypoint.sh && chmod +x /entrypoint.sh
+
+ENTRYPOINT ["/entrypoint.sh"]

test/compat/README.md

@@ -0,0 +1,181 @@
+---
+name: compat-matrix
+description: >-
+  Cross-OS / cross-Minecraft-version build-and-render compatibility harness for
+  The-Minecraft-Overviewer. Builds the c_overviewer C extension against each
+  environment's numpy and Pillow, then renders a set of test worlds against
+  their matching-version Minecraft client jars and reports a pass/fail matrix.
+when_to_use: >-
+  Use before merging changes that touch the C extension, setup.py, dependency
+  pins, or version-specific texture/world handling. Run it to confirm Overviewer
+  still builds and renders on Ubuntu 22.04 / 24.04 / 26.04 (numpy/Pillow from
+  apt) and Windows 11 with Python >= 3.12 (numpy/Pillow from pip), across the
+  supported Minecraft world formats.
+entrypoints:
+  - run_matrix.py   # host orchestrator (Windows): drives all OS legs
+  - render_test.py  # in-environment worker: build + render + result.json
+inputs:
+  worlds:   "Derek-Single (MC 1.21.11), Tester (MC 26.1), GiveEr26_2 (MC 26.2)"
+  textures: "matching-version Minecraft client jars from .minecraft/versions"
+outputs:
+  - "per-leg renders + result.json under the --work dir"
+  - "a printed PASS/FAIL matrix; process exit code 0 iff every leg passed"
+requires:
+  - docker            # for the Ubuntu legs
+  - "a C toolchain"   # gcc in the containers; MSVC for the Windows leg
+  - "installed MC client jars for 1.21.11, 26.1, 26.2"
+  - "the three test worlds present in the saves dir"
+network: "yes — fetches matching Pillow sdist C headers from PyPI at build time"
+safety: "read-only on real saves; worlds are copied to a scratch dir before rendering"
+---
+
+# Overviewer compatibility matrix
+
+This folder verifies that Overviewer **builds and renders** across the operating
+systems and Minecraft versions we support, using each platform's *own* numpy and
+Pillow (apt-provided on Ubuntu, pip on Windows). It exists because several
+incompatibilities only appear on specific combinations — e.g. numpy 2.x build
+flags on gcc, the Pillow 12 C-API change, and Minecraft 26.2 texture relocations.
+
+## High-level quickstart
+
+Run the whole matrix from the **repo root on the Windows host** (needs Docker
+Desktop running and the Minecraft client jars installed):
+
+```bash
+python test/compat/run_matrix.py
+```
+
+That stages the test worlds to a scratch dir, then runs four legs — Windows
+native plus Ubuntu 22.04 / 24.04 / 26.04 containers — building the C extension
+and rendering all three worlds in each. It prints a matrix like:
+
+```
+scenario           python   numpy      result   per-world
+Windows-native     3.14.5   2.5.0      PASS     Derek-Single=ok Tester=ok GiveEr26_2=ok
+Ubuntu 22.04       3.10.12  1.21.5     PASS     Derek-Single=ok Tester=ok GiveEr26_2=ok
+Ubuntu 24.04       3.12.3   1.26.4     PASS     Derek-Single=ok Tester=ok GiveEr26_2=ok
+Ubuntu 26.04       3.14.4   2.3.5      PASS     Derek-Single=ok Tester=ok GiveEr26_2=ok
+```
+
+Exit code is `0` only if every leg passed. Common subsets:
+
+```bash
+# Just the Ubuntu legs (skip the Windows native build):
+python test/compat/run_matrix.py --skip-windows
+
+# A single Ubuntu version:
+python test/compat/run_matrix.py --skip-windows --ubuntu 26.04
+
+# Point at non-default saves / versions / scratch locations:
+python test/compat/run_matrix.py \
+    --saves    "D:/mc/saves" \
+    --versions "D:/mc/versions" \
+    --work     "D:/tmp/ovmatrix"
+```
+
+Renders and a machine-readable `result.json` for each leg land under
+`--work/out/<leg>/` (default `--work` is `…/Temp/ov_compat_matrix`). Open any
+`…/out/<leg>/<world>/index.html` to eyeball a render.
+
+## Prerequisites
+
+- **Docker** running (Ubuntu legs). Base images `ubuntu:22.04/24.04/26.04` are pulled on demand.
+- **Minecraft client jars** for each world's version under the versions dir, i.e.
+  `versions/1.21.11/1.21.11.jar`, `versions/26.1/26.1.jar`, `versions/26.2/26.2.jar`.
+  These supply version-accurate textures (server jars contain no textures).
+- **The three test worlds** in the saves dir: `Derek-Single`, `Tester`, `GiveEr26_2`.
+- **A C compiler** — gcc is installed inside the containers; the Windows leg needs the
+  MSVC build tools already present on the host.
+- **Network access** — each leg downloads the matching Pillow source distribution to get
+  `Imaging.h` (neither pip wheels nor apt ship Pillow's C headers).
+
+---
+
+## Deeper dive
+
+### Files
+
+| File | Role |
+|------|------|
+| `run_matrix.py` | Host orchestrator. Stages worlds, runs the Windows-native leg in a fresh venv and each Ubuntu leg in a container, collects `result.json`, prints the matrix. |
+| `render_test.py` | The per-environment worker. Runs *inside* a leg: prints env + numpy/Pillow provenance, ensures Pillow headers, builds `c_overviewer`, renders each world, writes `result.json`. OS-agnostic (stdlib + project deps only). |
+| `Dockerfile.ubuntu` | Parametrized image (`--build-arg BASE=ubuntu:<ver>`). Installs OS-provided deps via apt (`python3-numpy python3-pil python3-networkx python3-requests`, build tooling). |
+| `entrypoint.sh` | Container entry: copies the read-only repo to a writable `/work`, then invokes `render_test.py` against the mounted worlds/versions/output. |
+
+### What each leg does
+
+1. **Report the environment** — OS, Python, and numpy/Pillow version **and file path**, so
+   you can confirm *which* numpy/Pillow was used (apt vs pip, 1.x vs 2.x).
+2. **Ensure Pillow C headers** — read `PIL.__version__`, download that exact Pillow sdist,
+   extract `src/libImaging/*.h`, and expose them via `PIL_INCLUDE_DIR`.
+3. **Build** `c_overviewer` with `setup.py build_ext --inplace` against the leg's numpy.
+4. **Render** each world to its own output dir with the matching client jar as `texturepath`.
+5. **Judge** — a world passes if Overviewer exits 0 and every requested dimension produced
+   `> 0` PNG tiles. The leg passes if all worlds pass.
+
+### World / version / texture map (defined in `render_test.py`)
+
+| World | MC version | Dimensions rendered | Layout exercised |
+|-------|-----------|---------------------|------------------|
+| `Derek-Single` | 1.21.11 | overworld | legacy (`region/`, `DIM*`) |
+| `Tester` | 26.1 | overworld, nether, end | new `dimensions/minecraft/*` |
+| `GiveEr26_2` | 26.2 | overworld, nether, end | new layout + 26.2 texture relocations |
+
+To add a world, extend the `WORLDS` dict in `render_test.py` (and have a matching
+`versions/<ver>/<ver>.jar` and a copy of the world available).
+
+### Why containers, and the dev-box fallback trap
+
+The Ubuntu legs run in **isolated containers that mount only the matching client jar**.
+This is deliberate: on a developer machine with many Minecraft versions installed,
+Overviewer silently **falls back to older jars** for any texture missing from the target
+version's jar. That masks version-specific texture breakage (e.g. the Minecraft 26.2
+bed/sign/pillar relocations) — a render can "pass" on the dev box yet fail for a real user
+who only has 26.2. The container is the honest test; always trust it over a local run.
+
+### Dependency policy (per OS)
+
+- **Ubuntu**: use what the OS provides — numpy, Pillow, networkx, requests all come from
+  `apt`. Nothing runtime is `pip`-installed. (`SETUPTOOLS_USE_DISTUTILS=local` is set so
+  `setup.py` still imports `distutils` on Python ≥ 3.12, which removed it from stdlib.)
+- **Windows**: everything from `pip` in a throwaway venv created by the orchestrator
+  (relaxed `numpy>=1.21,<3`, `pillow>=10,<13`, plus networkx/requests — not the
+  Windows-packaging extras).
+
+### Interpreting results
+
+- `result.json` (per leg) contains the full env block and per-world tile counts — diff it
+  across runs to see what changed.
+- A leg with `result == null` in the matrix means the container build/run itself failed
+  (e.g. a compile error) — read that leg's stdout, not its (absent) `result.json`.
+- A world that builds but renders **0 tiles** is almost always a missing/renamed texture
+  for that Minecraft version — run that one world with `--verbose` against the matching
+  jar to see the exact `Could not find the textures …` path.
+
+### Running a single leg by hand (debugging)
+
+In-place worker (uses the current interpreter's numpy/Pillow):
+
+```bash
+python test/compat/render_test.py \
+    --worlds  "<staged worlds dir>" \
+    --versions "<versions dir>" \
+    --output  "<output dir>"
+```
+
+One Ubuntu image + container directly:
+
+```bash
+docker build --build-arg BASE=ubuntu:26.04 -t ov-compat-2604 -f test/compat/Dockerfile.ubuntu test/compat
+docker run --rm \
+    -v "<repo>:/repo:ro" \
+    -v "<staged worlds>:/worlds:ro" \
+    -v "<versions>:/mc-versions:ro" \
+    -v "<output>:/output" \
+    ov-compat-2604
+```
+
+> Note: this harness is a developer/CI utility, kept separate from the unit test suite
+> (`test/test_*.py`). It shells out to Docker and a platform compiler and is not collected
+> by `pytest`.

test/compat/render_test.py

@@ -0,0 +1,186 @@
+#!/usr/bin/env python3
+"""In-environment compatibility worker for the numpy / MC-version render matrix.
+
+Runs *inside* one environment (Windows native, or an Ubuntu container) and:
+
+  1. prints the OS, Python, and numpy/Pillow version + provenance (so we can
+     prove which numpy was used -- apt vs pip, 1.x vs 2.x);
+  2. ensures Pillow's C headers are available (neither pip nor apt ship
+     ``Imaging.h``) by fetching the matching Pillow sdist headers if needed and
+     exposing them via ``PIL_INCLUDE_DIR``;
+  3. builds the ``c_overviewer`` C extension against this environment's numpy;
+  4. renders each test world against its *matching-version* Minecraft client
+     jar and checks the render produced tiles;
+  5. writes a JSON result summary.
+
+It is intentionally dependency-light (stdlib + the project's own deps) so the
+same file works unmodified on every target OS.
+"""
+
+import argparse
+import glob
+import io
+import json
+import os
+import platform
+import subprocess
+import sys
+import tarfile
+import urllib.request
+
+# world name -> (version dir / jar basename, [dimensions])
+# Matching-version client jars supply version-accurate textures.
+WORLDS = {
+    "Derek-Single": ("1.21.11", ["overworld"]),                  # legacy layout
+    "Tester":       ("26.1",    ["overworld", "nether", "end"]),  # 26.1 new layout
+    "GiveEr26_2":   ("26.2",    ["overworld", "nether", "end"]),  # 26.2 new layout
+}
+
+
+def log(msg):
+    print(msg, flush=True)
+
+
+def numpy_provenance():
+    import numpy
+    return {"version": numpy.__version__, "path": os.path.dirname(numpy.__file__)}
+
+
+def pillow_provenance():
+    import PIL
+    return {"version": PIL.__version__, "path": os.path.dirname(PIL.__file__)}
+
+
+def find_imaging_header():
+    """Return a directory containing Imaging.h if one is already discoverable."""
+    candidates = []
+    env = os.environ.get("PIL_INCLUDE_DIR")
+    if env:
+        candidates.extend(env.split(os.pathsep))
+    import PIL
+    pil_dir = os.path.dirname(PIL.__file__)
+    candidates.append(pil_dir)
+    candidates.append(os.path.join(pil_dir, "include"))
+    candidates.append(os.path.join(sys.prefix, "include"))
+    for c in candidates:
+        if c and os.path.exists(os.path.join(c, "Imaging.h")):
+            return c
+    return None
+
+
+def ensure_pillow_headers(workdir):
+    """Make sure Imaging.h is reachable; fetch the matching sdist if not.
+
+    Modern Pillow wheels (and Debian/Ubuntu's python3-pil) do not ship the C
+    headers that Overviewer's extension #includes, so download the source
+    distribution that matches the *installed* Pillow version and expose its
+    libImaging headers via PIL_INCLUDE_DIR.
+    """
+    found = find_imaging_header()
+    if found:
+        os.environ["PIL_INCLUDE_DIR"] = found
+        log("Pillow headers already present at: %s" % found)
+        return found
+
+    import PIL
+    version = PIL.__version__
+    log("Pillow headers missing; fetching matching sdist for Pillow %s ..." % version)
+    meta_url = "https://pypi.org/pypi/pillow/%s/json" % version
+    with urllib.request.urlopen(meta_url) as r:
+        meta = json.load(r)
+    sdist = next(u for u in meta["urls"] if u["packagetype"] == "sdist")
+    dest = os.path.join(workdir, "pil-headers")
+    os.makedirs(dest, exist_ok=True)
+    with urllib.request.urlopen(sdist["url"]) as r:
+        data = r.read()
+    with tarfile.open(fileobj=io.BytesIO(data), mode="r:gz") as tf:
+        for m in tf.getmembers():
+            base = os.path.basename(m.name)
+            if base.endswith(".h") and "/libImaging/" in m.name.replace("\\", "/"):
+                m.name = base
+                tf.extract(m, dest)
+    if not os.path.exists(os.path.join(dest, "Imaging.h")):
+        raise RuntimeError("could not obtain Imaging.h from Pillow %s sdist" % version)
+    os.environ["PIL_INCLUDE_DIR"] = dest
+    log("Pillow headers extracted to: %s" % dest)
+    return dest
+
+
+def build_extension(repo):
+    log(">>> Building c_overviewer against numpy %s ..." % numpy_provenance()["version"])
+    subprocess.run([sys.executable, "setup.py", "build_ext", "--inplace"],
+                   cwd=repo, check=True)
+
+
+def render_world(repo, name, versions_dir, worlds_dir, out_root):
+    version, dims = WORLDS[name]
+    jar = os.path.join(versions_dir, version, "%s.jar" % version)
+    world_path = os.path.join(worlds_dir, name)
+    out_dir = os.path.join(out_root, name)
+    conf = os.path.join(out_root, "%s.conf" % name)
+    os.makedirs(out_dir, exist_ok=True)
+
+    lines = [
+        "worlds[%r] = r%r" % (name, world_path),
+        "outputdir = r%r" % out_dir,
+        "texturepath = r%r" % jar,
+        'rendermode = "normal"',
+    ]
+    for d in dims:
+        lines.append("renders[%r] = {'world': %r, 'title': %r, 'dimension': %r}"
+                     % (d, name, d, d))
+    with open(conf, "w") as f:
+        f.write("\n".join(lines) + "\n")
+
+    log("\n#### Rendering %s (mc %s, dims: %s)" % (name, version, ", ".join(dims)))
+    proc = subprocess.run([sys.executable, "overviewer.py", "--config=%s" % conf],
+                          cwd=repo)
+    tiles = {d: len(glob.glob(os.path.join(out_dir, d, "**", "*.png"), recursive=True))
+             for d in dims}
+    ok = proc.returncode == 0 and all(t > 0 for t in tiles.values())
+    log("   exit=%d tiles=%s -> %s" % (proc.returncode, tiles, "PASS" if ok else "FAIL"))
+    return {"world": name, "mc_version": version, "exit_code": proc.returncode,
+            "tiles": tiles, "passed": ok}
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--repo", default=os.getcwd())
+    ap.add_argument("--worlds", required=True, help="dir containing the test worlds")
+    ap.add_argument("--versions", required=True, help="dir containing <ver>/<ver>.jar")
+    ap.add_argument("--output", required=True, help="dir to write renders into")
+    ap.add_argument("--result-json", default=None, help="optional path for JSON result")
+    args = ap.parse_args()
+
+    os.makedirs(args.output, exist_ok=True)
+    env = {
+        "os": platform.platform(),
+        "python": sys.version.split()[0],
+        "numpy": numpy_provenance(),
+        "pillow": pillow_provenance(),
+    }
+    log("=" * 64)
+    log("OS      : %s" % env["os"])
+    log("Python  : %s (%s)" % (env["python"], sys.executable))
+    log("numpy   : %(version)s  [%(path)s]" % env["numpy"])
+    log("Pillow  : %(version)s  [%(path)s]" % env["pillow"])
+    log("=" * 64)
+
+    ensure_pillow_headers(args.output)
+    build_extension(args.repo)
+
+    results = [render_world(args.repo, name, args.versions, args.worlds, args.output)
+               for name in WORLDS]
+    summary = {"env": env, "results": results,
+               "passed": all(r["passed"] for r in results)}
+
+    out_json = args.result_json or os.path.join(args.output, "result.json")
+    with open(out_json, "w") as f:
+        json.dump(summary, f, indent=2)
+    log("\n==== %s ====" % ("ALL WORLDS PASSED" if summary["passed"] else "FAILURES PRESENT"))
+    log("result json: %s" % out_json)
+    return 0 if summary["passed"] else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())