# AI Runner CI/CD Pipeline

This document outlines the Continuous Integration and Continuous Deployment pipeline used by the AI Runner project.

## Overview

The AI Runner project uses GitHub Actions for CI/CD automation, Docker for build environment consistency, and itch.io for distributing the built application. The pipeline includes:

1. Running tests to ensure code quality
2. Building a Docker image containing the build environment for Linux
3. Using the Docker image to build AI Runner for Linux
4. Building AI Runner for Windows directly on GitHub Actions
5. Deploying the built applications to itch.io
6. Publishing the Python package to PyPI

## Components

### Docker Build Environment

AI Runner uses a Docker container to ensure a consistent build environment for Linux builds. The Docker image is defined in `/package/Dockerfile` and includes:

- Ubuntu 22.04 as the base image
- Python 3.10 and required dependencies
- Qt 6.7.0 built from source
- CUDA and other ML/AI dependencies

This Docker image is built and published to GitHub Container Registry (ghcr.io) for use in subsequent build processes.

### GitHub Actions Workflows

#### Testing Workflow

The `tests.yml` workflow handles code quality checks and testing. This workflow:

1. Sets up a Python environment
2. Installs test dependencies (pytest, pytest-cov, flake8)
3. Runs linting with flake8 to check for syntax errors and code style
4. Runs pytest with coverage reporting
5. Uploads test results as artifacts

This workflow runs on every push to the develop and master branches, on pull requests, and can be triggered manually.

#### Master Release Pipeline

The `release-pipeline.yml` workflow serves as the coordinator for all platform builds. When a new release is published or the workflow is manually triggered, it:

1. Runs the testing workflow to ensure code quality
2. Extracts the version number from `setup.py`
3. Triggers all platform-specific build workflows (Linux, Windows, PyPI)
4. Creates a release summary report
5. Uploads the summary as an artifact

This master workflow simplifies the release process and ensures all platforms are built with the same version.

#### Linux Build Workflow

The `linux-dispatch.yml` workflow handles building the Linux version of AI Runner and deploying it to itch.io. This workflow:

1. Builds the Docker image using the Dockerfile and Docker Buildx
2. Pushes the Docker image to GitHub Container Registry with caching enabled
3. Pulls the Docker image and runs it to build AI Runner
4. Uses the Butler CLI to deploy the built application to itch.io
5. Uploads build artifacts for easy access

This workflow is triggered either manually via workflow_dispatch, when a new release is published, or by the master release pipeline.

#### Windows Build Workflow

The `windows-dispatch.yml` workflow handles building the Windows version of AI Runner and deploying it to itch.io. This workflow:

1. Sets up a Windows environment with Python 3.10
2. Installs required dependencies, including PyInstaller and PyTorch with CUDA support
3. Builds AI Runner using PyInstaller
4. Creates the appropriate directory structure for itch.io deployment
5. Downloads and sets up Butler CLI for Windows
6. Deploys the built application to itch.io under the Windows channel
7. Uploads build artifacts for easy access

This workflow is also triggered either manually, when a new release is published, or by the master release pipeline.

#### PyPI Workflow

The `pypi-dispatch.yml` workflow handles publishing the Python package to PyPI. This workflow:

1. Sets up a Python environment
2. Builds the Python package
3. Publishes the package to PyPI using GitHub's PyPI publish action
4. Uploads package artifacts for archiving

This workflow is also triggered either manually, when a new release is published, or by the master release pipeline.

#### Dependency Review Workflow

The `dependency-review.yml` workflow performs security review of dependencies when pull requests are created. It:

1. Scans the dependency manifest files
2. Identifies known vulnerabilities in dependencies
3. Comments a summary in the PR
4. Optionally blocks merging if critical vulnerabilities are found

### Build and Deployment Scripts

Several scripts in the `package` directory manage the build and deployment process:

- `build.sh`: The main build script run inside the Docker container for Linux builds. It installs dependencies, builds AI Runner using PyInstaller, and deploys to itch.io using Butler.
- `dobuild.py`: A Python script that pulls the latest code, builds the package, and installs it in the Docker container.
- `linux.itch.toml`: Configuration file for the Linux itch.io deployment.
- `windows.itch.toml`: Configuration file for the Windows itch.io deployment.

## Workflow

1. When a new release is created or a manual build is triggered:
   - The master release pipeline is triggered
   - Tests are run to ensure code quality
   - The pipeline coordinates builds for all platforms:
     - For Linux builds:
       - The Docker image is built or pulled from GitHub Container Registry
       - The Docker container builds AI Runner using PyInstaller
       - The built application is deployed to itch.io via Butler
     - For Windows builds:
       - The Windows runner environment is set up with dependencies
       - AI Runner is built using PyInstaller
       - The built application is deployed to itch.io via Butler
     - The Python package is published to PyPI
   - A release summary is generated and uploaded as an artifact

2. For pull requests to the master branch:
   - Tests are run to check for regressions
   - Dependency vulnerabilities are checked
   - Results are commented in the PR

## Version Management

Version numbers are extracted from the `setup.py` file and used for tagging releases on itch.io. The current version pattern follows semantic versioning (MAJOR.MINOR.PATCH).

## Deployment Targets

- **itch.io**: The built applications are deployed to itch.io under the project "capsizegames/ai-runner" with separate channels for each platform:
  - Ubuntu Linux: `ubuntu` channel
  - Windows: `windows` channel
- **PyPI**: The Python package is published to PyPI with the name "airunner"

## GitHub Container Registry

AI Runner uses GitHub Container Registry (ghcr.io) to store and distribute the Docker image used for building the Linux application. This requires:

- GitHub Personal Access Token (PAT) with appropriate permissions
- Repository secrets for authentication

## Environment Variables and Secrets

The CI/CD pipeline requires several environment variables and secrets:

- `BUTLER_API_KEY`: API key for itch.io deployment
- `CR_PAT`: GitHub Container Registry Personal Access Token
- `CR_REPO`: GitHub Container Registry repository
- `CR_USERNAME`: GitHub Container Registry username
- `ITCHIO_USERNAME`: itch.io username
- `PYPI_API_TOKEN`: API token for PyPI publishing
- `GITHUB_TOKEN`: GitHub token for workflow dispatch (automatically provided by GitHub)

## Troubleshooting

If you encounter issues with the CI/CD pipeline, check the following:

1. **Test failures**: Check the GitHub Actions logs for test failures and fix any failing tests.
2. **Docker build failures**: Ensure the Dockerfile is up-to-date and all dependencies are correctly specified.
3. **GitHub Actions failures**: Check the GitHub Actions logs for specific error messages.
4. **itch.io deployment issues**: Verify that the Butler API key is correct and that the itch.io project exists.
5. **PyPI publishing problems**: Ensure the PyPI API token is valid and has the correct permissions.
6. **Workflow coordination issues**: Make sure the master release workflow has the necessary permissions to trigger other workflows.

## Future Improvements

Planned improvements for the CI/CD pipeline:

1. Add more comprehensive automated testing 
2. Add macOS builds for broader platform support 
3. Implement automated semantic versioning
4. Add status notifications for build progress (email, Slack, or Discord integration)
5. Add performance regression testing