Add GCE Image support#20
Open
furkansahin wants to merge 9 commits intomainfrom
Open
Conversation
furkansahin
force-pushed
the
make-gce-images-public
branch
from
48c1931 to
cafb5d0
Compare
Add build-gce.sh that wraps the standard build with GCE-specific post-processing: GRUB reinstall for BIOS/UEFI boot, Google guest agent installation, and tar.gz packaging for gcloud image import. Extract post-processing into gce-postprocess.sh so it can be reused by both the standalone script and the CI workflow. Supports x86_64 (BIOS + VIRTIO_SCSI_MULTIQUEUE) and ARM64 (UEFI + GVNIC). Add GCE cleanup to cleanup-images.yml alongside MinIO, R2, and AWS: deletes GCE images and GCS tarballs by version suffix. Also fix chroot builds: set HOME/GOPATH in setup_packages.sh and use linux-modules-extra for gVNIC driver support on GCP. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add upload_gce toggle alongside MinIO, R2, and AWS AMI. GCE steps post-process a copy of the raw image (preserving the original for other targets), upload the tar.gz to GCS, create a GCE image, and grant public access. Both x64 and arm64 jobs get GCE steps with arch-specific guest OS features. The create-ubicloud-pr job generates a pg_gce_image migration alongside the existing pg_aws_ami migration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
furkansahin
force-pushed
the
make-gce-images-public
branch
from
abe3dc7 to
4813132
Compare
Drop the gcp_project workflow_dispatch input and read the project from the SA key JSON via google-github-actions/auth@v2's project_id output. The auth action already parses the credentials_json and exposes its project_id; threading the same value back through a separate input invites mismatch (e.g., key for ubicloud-images but input pointing at pelagic-logic-394811). Add id: gcp_auth to both x64 and arm64 auth steps and replace six inputs.gcp_project references with steps.gcp_auth.outputs.project_id.
Drop the gcs_bucket workflow_dispatch input and read the bucket from secrets.GCS_BUCKET. This matches the AWS_S3_BUCKET and R2_BUCKET pattern already used elsewhere in the workflow, leaving only GCE-image-relevant toggles in workflow_dispatch.
GCE image names must match [a-z](?:[-a-z0-9]{0,61}[a-z0-9])? — no
dots allowed. The shared image_suffix input uses "YYYYMMDD.X.Y"
format (fine for AMIs) which makes GCE reject the create with:
Invalid value for field 'resource.name': 'postgres-ubuntu-2204-x64-20260428.1.0'
Replace dots with hyphens in the GCE-specific Set GCE image name
step (both x64 and arm64), so suffix=20260428.1.0 yields
postgres-ubuntu-2204-x64-20260428-1-0. AMI/MinIO/R2/GCS object
names keep the dotted suffix verbatim.
The arm64 build job conditionally ran on (build_arm64 || upload_aws_ami), so the job exists when only build_arm64 is selected. But the AWS-specific steps inside the job were gated only on !build_only, so they fired whenever arm64 ran - forcing every arm64 build to upload to S3, import a snapshot, register an AMI, copy across regions, etc., even with the AWS checkbox off. Add inputs.upload_aws_ami to the 12 AWS-only steps in arm64, matching the gating already used in the x64 job. Build_arm64 without upload_aws_ami now produces only the local raw image (plus MinIO/R2/GCE if those toggles are on).
Three fixes to keep cleanup-images.yml symmetric with the build:
1. Drop gcp_project input; resolve project from the SA key via
google-github-actions/auth@v2's project_id output (id: gcp_auth).
2. Drop gcs_bucket input; read bucket name from secrets.GCS_BUCKET,
matching AWS_S3_BUCKET / R2_BUCKET pattern.
3. Compute a separate {x64,arm64}_gce_image_name with dots replaced
by hyphens, mirroring the build workflow's sanitization. Without
this the cleanup looked for postgres-ubuntu-2204-x64-20260428.1.0
which never exists in GCE - the actually-created image is named
postgres-ubuntu-2204-x64-20260428-1-0. Tarball path uses the same
sanitized name so the GCS object lookup matches the upload.
MinIO/R2/AWS cleanup paths keep the dotted name as before.
iamKunalGupta noted on PR #20 that the workflows authenticate to GCP with a long-lived service account JSON key (secrets.GCP_SA_KEY), and recommended Workload Identity Federation per https://docs.cloud.google.com/iam/docs/workload-identity-federation. Replace the credentials_json input on all three google-github-actions/auth@v2 sites (postgres-vm-image.yml lines 538 and 1029, cleanup-images.yml line 312) with workload_identity_provider + service_account, both read from non-secret repo variables (vars.WIF_PROVIDER, vars.WIF_SERVICE_ACCOUNT). The provider path embeds the project number, pool ID, and provider ID; the service_account remains postgres-vm-images@ubicloud-images.iam.gserviceaccount.com so existing IAM bindings on the SA continue to apply unchanged. Add permissions: id-token: write at the workflow level on cleanup-images.yml (postgres-vm-image.yml already had it). This lets GitHub mint the OIDC token that the auth action exchanges for short-lived GCP credentials via the WIF pool. GCP-side setup that must exist before this commit lands: - Workload Identity Pool github-pool in ubicloud-images. - OIDC provider github-provider on that pool, issuer https://token.actions.githubusercontent.com, attribute condition assertion.repository_owner == 'ubicloud' so only ubicloud-org repos can mint tokens. - iam.workloadIdentityUser binding on postgres-vm-images@ubicloud-images.iam.gserviceaccount.com for principalSet ...attribute.repository/ubicloud/postgres-vm-images so impersonation is scoped to this single repo. After this lands and a dispatch run confirms the new auth path, the GCP_SA_KEY secret should be removed and the SA's underlying JSON key revoked in GCP IAM. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.