Ninja build (#231)

* Ninja build for C/ASM sources

* ninja on linux

* Remove old build flags

* Update CI and diff settings

* Change links to dropbox

* Add link rules for ninja

* Add rules for packing and verifying, update README and CI

* allow chain

* gcc depfile include dirs

* nostdinc

* Different tools installation

Co-authored-by: Theodoros Tyrovouzis <[email protected]>

.devcontainer/Dockerfile

@@ -23,11 +23,18 @@ RUN dpkg --add-architecture i386 \
 # RUN su vscode -c "source /usr/local/share/nvm/nvm.sh && npm install -g <your-package-here>" 2>&1
 
 WORKDIR /blobs
-ARG TOOLS_URL="https://cdn.discordapp.com/attachments/767477506861170710/870608469558980649/tools.zip"
+ARG TOOLS_URL="https://www.dropbox.com/s/4b0hwciadfk8yuo/tools.zip?dl=1"
 ADD ${TOOLS_URL} ./tools.zip
 RUN mkdir -p ./artifacts/orig/pal \
-    && echo "2D78EEB2B90A2E57DB4B5DC55F37468B115B344498B8875D2EC8ADDE5AB8B243 tools.zip" | sha256sum --check --status \
+    && echo "40cbee5ce207d8ca71b6611f8f7560e6f135562c25e8fa303f916011e99517ca tools.zip" | sha256sum --check --status \
     && unzip -o tools.zip \
     && rm tools.zip \
     && chmod +x ./tools/devkitppc/bin/powerpc-eabi-as \
-    && mv ./tools/blobs/* ./artifacts/orig/pal
+    && chmod +x ./tools/devkitppc/bin/powerpc-eabi-gcc \
+    && mv ./tools/blobs/* ./artifacts/orig/pal \
+
+ARG NINJA_URL="https://github.com/ninja-build/ninja/releases/download/v1.11.1/ninja-win.zip"
+ADD ${NINJA_URL} ./ninja.zip
+RUN unzip -o ninja.zip \
+    && rm ninja.zip
+    && chmod +x ./ninja.exe

.github/workflows/build.yml

@@ -33,12 +33,12 @@ jobs:
 
       - name: Extract tooling
         env:
-          TOOLS_URL: 'https://cdn.discordapp.com/attachments/767477506861170710/870608469558980649/tools.zip'
+          TOOLS_URL: 'https://www.dropbox.com/s/4b0hwciadfk8yuo/tools.zip?dl=1'
         run: |
           # Download tooling
           Invoke-WebRequest "$Env:TOOLS_URL" -OutFile .\tools.zip
           $hash = (Get-FileHash '.\tools.zip').Hash
-          if ($hash -ne '2D78EEB2B90A2E57DB4B5DC55F37468B115B344498B8875D2EC8ADDE5AB8B243')
+          if ($hash -ne '40cbee5ce207d8ca71b6611f8f7560e6f135562c25e8fa303f916011e99517ca')
           {
             echo "Invalid hash: $hash"
             exit 1
@@ -55,9 +55,13 @@ jobs:
           Remove-Item '.\tools.zip'
           Remove-Item -LiteralPath 'tools2' -Force -Recurse
 
+      - name: Configure mkw
+        shell: bash
+        run: python configure.py --regen_asm
+
       - name: Build mkw
         shell: bash
-        run: python build.py --regen_asm
+        run: ninja
 
       - name: Get DOL progress
         run: python -m mkwutil.progress.percent_decompiled --short --part DOL > badge_dol_progress.txt

.gitignore

@@ -75,3 +75,8 @@ doxygen
 
 # diff.py expected
 expected
+
+# ninja
+build.ninja
+.ninja_deps
+.ninja_log

HACK.md

@@ -12,7 +12,7 @@ Contains the reference artifacts from the original game, and the generated final
 - `./artifacts/target/pal/main.dol`: Ouptut main executable.
 - `./artifacts/target/pal/StaticR.rel`: Output library.
 
-### `./build.py`: Main build script
+### `./configure.py`: Ninja build script generator
 
 Uses `mkwutil` to rebuild MKW from decompiled sources and verifies the result.
 

README.md

@@ -22,15 +22,16 @@ While the original game was written and compiled as C++03, several modern C++ fe
 Every fully understood piece of reverse engineered data has been documented in a consistent doxygen style.
 
 ## Dependencies
-- DevKitPro (for the ppc-eabi assembler)
+- DevKitPro (for the ppc-eabi assembler, and gcc dependency files)
 - CodeWarrior compilers (in `tools`)
+- Ninja
 - Python 3
 - Place a copy of Mario Kart Wii's PAL binaries:
   - `artifacts/orig/pal/main.dol`
   - `artifacts/orig/pal/StaticR.rel`
 
 ## Python Workspace
-
+This build system assumes that a `python` command is available that points to a compatibly python 3 interpreter
 ### venv
 
 It is recommended to setup a Python virtual environment to simplify workspace setup.
@@ -63,7 +64,7 @@ Run `pip install -e .` to do that.
 
 ## Building
 
-Run `python3 ./build.py` to build the game and verify build authenticity. Final results:
+Run `python ./configure.py` to create a build script and `ninja` to build the project. You may need to rerun `configure` after editing the slices. Final results:
   - `artifacts/target/pal/main.dol`
   - `artifacts/target/pal/StaticR.rel`
 
@@ -91,7 +92,7 @@ The dead-stripping feature can be re-enabled by:
   - [pack/rel_slices.csv](./pack/rel_slices.csv)
 - Entries must be sorted in the spreadsheet (current limitation).
 - Add your new build target to `mkwutil/sources.py`.
-- Run `build.py`.
+- Run `configure.py` and `ninja`
 
 ### pre-commit