Remove dbt-bigquery cache warming (#42)

* Initial plan

* Remove dbt-bigquery cache warming from Docker build

Co-authored-by: brabster <[email protected]>

* Update documentation to reflect dbt-bigquery removal

Co-authored-by: brabster <[email protected]>

* Update feature document to remove outdated cache warming reference

Co-authored-by: brabster <[email protected]>

* Add security posture analysis to changelog and update copilot instructions

Co-authored-by: brabster <[email protected]>

* Update changelog to reference PR #42

Co-authored-by: brabster <[email protected]>

---------

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: brabster <[email protected]>

Author: Copilot

.github/copilot-instructions.md

@@ -1,6 +1,6 @@
 ## Project overview
 
-This repository produces container images for data-centric development on Google Cloud Platform. The images include Terraform, Google Cloud SDK, Python, and dbt-bigquery. They are designed for use in GitHub Codespaces and GitHub Actions.
+This repository produces container images for data-centric development on Google Cloud Platform. The images include Terraform, Google Cloud SDK, and Python. They are designed for use in GitHub Codespaces and GitHub Actions.
 
 **Always review `GEMINI.md` for complete project information, goals, and principles.**
 
@@ -81,6 +81,17 @@ Code reviews must include:
 
 The reviewer must strive to identify impactful changes, rather than cosmetic or stylistic changes.
 
+## Changelog requirements
+
+All changelog entries must include:
+
+- A **Security** section that analyzes the security posture impact of the change
+- The security analysis should include:
+  - Direct security implications of the change (e.g., reduced attack surface, improved authentication)
+  - **Supply Chain Posture Impact** or **Threat Model Impact** subsection explaining how the change affects security
+  - **Security Posture Impact** conclusion (Positive, Neutral, or Negative)
+- See existing entries in `CHANGELOG.md` for examples of the required format and level of detail
+
 ## Responding to review feedback
 
 When addressing code review comments:

CHANGELOG.md

@@ -4,6 +4,30 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## [[#41](https://github.com/brabster/terraform-bootstrap-gcp/pull/42)] - Remove dbt-bigquery cache warming
+
+### Removed
+
+- Removed dbt-bigquery cache warming from the Docker build process.
+- Deleted requirements.txt file used for pre-warming the pip cache.
+- Removed dbt-bigquery from the dependencies table in README as it is no longer pre-installed.
+
+### Changed
+
+- Updated README to remove documentation about pre-warmed pip cache for dbt-bigquery.
+
+### Rationale
+
+The cache warming provided minimal benefit while adding complexity to the build process and image size. Users can install dbt-bigquery and other Python packages as needed for their specific use case.
+
+### Security
+
+- Removing pre-installed Python packages reduces the attack surface by eliminating dependencies that may not be needed by all users.
+- Simplifies the supply chain by removing dbt-bigquery and its transitive dependencies from the image.
+
+  - **Supply Chain Posture Impact:** This change improves the project's supply chain security posture by removing unnecessary dependencies from the base image. Users now explicitly install only the Python packages they need, reducing the number of packages that must be monitored for vulnerabilities. This aligns with the principle of minimal dependencies and reduces the image's attack surface.
+  - **Security Posture Impact:** Positive
+
 ## [[#35](https://github.com/brabster/terraform-bootstrap-gcp/pull/36)] - Add git CLI completion support
 
 ### Added

Dockerfile

@@ -58,13 +58,3 @@ RUN --mount=type=secret,id=proxy_cert,required=false \
 USER ubuntu
 
 WORKDIR /home/ubuntu
-
-# Pre-warm pip cache
-COPY requirements.txt .
-RUN VENV_PATH=$(mktemp -d) \
-    && python3 -m venv "$VENV_PATH" \
-    && . "$VENV_PATH"/bin/activate \
-    && pip install -r requirements.txt \
-    && pip freeze > .preinstalled_requirements.txt \
-    && rm -rf "$VENV_PATH" \
-    && rm requirements.txt

README.md

@@ -5,7 +5,7 @@
 
 **THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND under the [MIT License](LICENCE).** Commercial use is permitted, but the author recommends copying or forking to avoid supply chain risks. Clarity and minimialism are goals to support realistic consumer auditing.
 
-This repository provides a Docker container image for cloud and data engineering. It includes essential tools for working with Google Cloud Platform, Terraform, and dbt.
+This repository provides a Docker container image for cloud and data engineering. It includes essential tools for working with Google Cloud Platform, Terraform, and Python.
 
 The image is built and published to the GitHub Container Registry every day. This process ensures it has the latest software versions and security updates.
 
@@ -27,12 +27,9 @@ This image relies on the following direct dependencies. Maintainers of these dep
 | Infrastructure as Code | `terraform`        | HashiCorp                  |
 | Cloud SDK              | Google Cloud SDK   | Google                     |
 | Language               | `python`           | Python Software Foundation |
-| Data transformation    | `dbt-bigquery`     | dbt Labs                   |
 | Version control        | `git`              | Canonical                  |
 | Shell completion       | `bash-completion`  | Canonical                  |
 
-The `dbt-bigquery` Python package and its dependencies are pre-loaded into the `pip` cache to reduce network requests to PyPI.
-
 ## Image tagging strategy
 
 The image has two types of tags:

prompts/features/02-default-user-ubuntu.md

@@ -30,7 +30,7 @@ refinement: |
   - The plan depends on the user/group configuration of the `docker.io/ubuntu:rolling` base image.
 
 - **Potential Issues & Mitigations:**
-  - **File Permissions:** Processes inside the container will run as the `ubuntu` user. The `Dockerfile` creates a pre-warmed pip cache, which will now be owned by `ubuntu`. This is the desired behavior and is not expected to cause issues.
+  - **File Permissions:** Processes inside the container will run as the `ubuntu` user. This is the desired behavior and is not expected to cause issues.
   - **Dev Container Context:** There is no direct impact on the dev container or Codespaces, as they use `/.devcontainer/Dockerfile`, which is not being modified. This isolates the change to the production image.
   - **GitHub Actions Context:** The `docker-publish.yml` workflow is not affected by the user change *within* the image it builds. The new verification step will be added to the existing `image_test` job to ensure correctness.