-
Notifications
You must be signed in to change notification settings - Fork 8
Add AWS deployment assets #230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
166500a
Add AWS single-node deployment assets
rgarcia 19e9ee9
Publish CloudFormation quickstart from CI
rgarcia ecf94c2
Use CloudFormation launch badge in docs
rgarcia 97ef22e
Flatten AWS deploy assets
rgarcia ad70bef
Remove redundant header clone
rgarcia c904c76
Use configured API port in validation
rgarcia 7f7501b
Consolidate AWS CloudFormation deployment
rgarcia 0ec80d5
Document nested virtualization launch helper
rgarcia d10fabb
Remove duplicate AWS validation script
rgarcia File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| name: Deploy Assets | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| pull_request: | ||
| paths: | ||
| - ".github/workflows/deploy.yml" | ||
| - ".gitignore" | ||
| - "deploy/**" | ||
| push: | ||
| paths: | ||
| - ".github/workflows/deploy.yml" | ||
| - ".gitignore" | ||
| - "deploy/**" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| validate: | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| PACKER_VERSION: "1.15.3" | ||
| TERRAFORM_VERSION: "1.15.3" | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: actions/setup-go@v6 | ||
| with: | ||
| go-version: "1.25.4" | ||
| cache: false | ||
|
|
||
| - name: Install validation tools | ||
| run: | | ||
| set -euo pipefail | ||
| python3 -m pip install --user cfn-lint | ||
|
|
||
| arch="$(uname -m)" | ||
| case "$arch" in | ||
| x86_64|amd64) hc_arch=amd64 ;; | ||
| aarch64|arm64) hc_arch=arm64 ;; | ||
| *) echo "unsupported arch: $arch" >&2; exit 1 ;; | ||
| esac | ||
|
|
||
| tmp_dir="$(mktemp -d)" | ||
| trap 'rm -rf "$tmp_dir"' EXIT | ||
| curl -fsSL -o "$tmp_dir/terraform.zip" "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_${hc_arch}.zip" | ||
| curl -fsSL -o "$tmp_dir/packer.zip" "https://releases.hashicorp.com/packer/${PACKER_VERSION}/packer_${PACKER_VERSION}_linux_${hc_arch}.zip" | ||
| unzip -q "$tmp_dir/terraform.zip" -d "$tmp_dir/terraform" | ||
| unzip -q "$tmp_dir/packer.zip" -d "$tmp_dir/packer" | ||
| sudo install -m 0755 "$tmp_dir/terraform/terraform" /usr/local/bin/terraform | ||
| sudo install -m 0755 "$tmp_dir/packer/packer" /usr/local/bin/packer | ||
|
|
||
| - name: Validate CloudFormation | ||
| run: | | ||
| cfn-lint deploy/aws/cloudformation/template.yaml | ||
| go test ./deploy/aws/cloudformation | ||
|
|
||
| - name: Validate scripts | ||
| run: bash -n deploy/aws/scripts/validate.sh | ||
|
|
||
| - name: Validate Terraform | ||
| run: | | ||
| terraform -chdir=deploy/aws/terraform init -backend=false | ||
| terraform -chdir=deploy/aws/terraform fmt -check | ||
| terraform -chdir=deploy/aws/terraform validate | ||
|
|
||
| - name: Validate Packer | ||
| run: | | ||
| packer init deploy/aws/ami/packer/hypeman.pkr.hcl | ||
| packer fmt -check deploy/aws/ami/packer/hypeman.pkr.hcl | ||
| packer validate deploy/aws/ami/packer/hypeman.pkr.hcl | ||
|
|
||
| publish-cloudformation: | ||
| runs-on: ubuntu-latest | ||
| needs: validate | ||
| if: (github.event_name == 'push' || github.event_name == 'workflow_dispatch') && github.ref == 'refs/heads/main' | ||
| environment: aws-publish | ||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| env: | ||
| AWS_REGION: us-east-1 | ||
| CLOUDFORMATION_TEMPLATE_BUCKET: kernel-hypeman-cloudformation-prod | ||
| CLOUDFORMATION_TEMPLATE_KEY: v1/hypeman/template.yaml | ||
| CLOUDFORMATION_TEMPLATE_SOURCE: deploy/aws/cloudformation/template.yaml | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: aws-actions/configure-aws-credentials@v4 | ||
| with: | ||
| role-to-assume: arn:aws:iam::613957054632:role/github-actions-hypeman-cloudformation-publisher | ||
| aws-region: ${{ env.AWS_REGION }} | ||
|
|
||
| - name: Publish CloudFormation template | ||
| run: | | ||
| set -euo pipefail | ||
|
|
||
| aws s3 cp \ | ||
| "$CLOUDFORMATION_TEMPLATE_SOURCE" \ | ||
| "s3://$CLOUDFORMATION_TEMPLATE_BUCKET/$CLOUDFORMATION_TEMPLATE_KEY" \ | ||
| --content-type application/x-yaml \ | ||
| --cache-control no-cache | ||
|
|
||
| aws s3api head-object \ | ||
| --bucket "$CLOUDFORMATION_TEMPLATE_BUCKET" \ | ||
| --key "$CLOUDFORMATION_TEMPLATE_KEY" \ | ||
| >/dev/null | ||
|
|
||
| curl -fsSL \ | ||
| "https://$CLOUDFORMATION_TEMPLATE_BUCKET.s3.$AWS_REGION.amazonaws.com/$CLOUDFORMATION_TEMPLATE_KEY" \ | ||
| >/dev/null |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| # Deploy Hypeman | ||
|
|
||
| This directory contains maintained deployment assets for running Hypeman outside local development. | ||
|
|
||
| ## AWS Quickstart | ||
|
|
||
| The fastest path is the hosted CloudFormation template. It creates one EC2 host with nested virtualization enabled, installs Hypeman, exposes the Hypeman API only to the CIDR you choose, and returns the commands needed to connect through AWS Systems Manager. | ||
|
|
||
| [](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https%3A%2F%2Fkernel-hypeman-cloudformation-prod.s3.us-east-1.amazonaws.com%2Fv1%2Fhypeman%2Ftemplate.yaml&stackName=hypeman) | ||
|
|
||
| Use `us-east-1` for the published template. Choose a VPC and subnet, set `AllowedApiCidr` to your current IP range or trusted VPN CIDR, keep SSH disabled unless you need it, then create the stack. | ||
|
|
||
| Useful stack outputs: | ||
|
|
||
| | Output | Purpose | | ||
| | --- | --- | | ||
| | `HypemanEndpoint` | Base URL for remote Hypeman API access | | ||
| | `SsmSessionCommand` | Session Manager command for host access | | ||
| | `CreateTokenCommand` | Command that generates a JWT on the host | | ||
| | `InstanceId` | EC2 instance running Hypeman | | ||
|
|
||
| ## Use Hypeman | ||
|
|
||
| After the stack reaches `CREATE_COMPLETE`, run the `SsmSessionCommand` output and generate a token: | ||
|
|
||
| ```sh | ||
| sudo hypeman-create-token remote-user 8760h | ||
| ``` | ||
|
|
||
| On your local machine, install the CLI and point it at the `HypemanEndpoint` output: | ||
|
|
||
| ```sh | ||
| curl -fsSL https://get.hypeman.sh/cli | bash | ||
|
|
||
| mkdir -p ~/.config/hypeman | ||
| cat > ~/.config/hypeman/cli.yaml <<EOF | ||
| base_url: http://<public-ip>:8080 | ||
| api_key: "<jwt-from-hypeman-create-token>" | ||
| EOF | ||
|
|
||
| hypeman ps | ||
| ``` | ||
|
|
||
| Build, push, and run a sandbox image: | ||
|
|
||
| ```sh | ||
| mkdir -p /tmp/hypeman-claude-code | ||
| cat > /tmp/hypeman-claude-code/Dockerfile <<'EOF' | ||
| FROM node:22-bookworm-slim | ||
| RUN npm install -g @anthropic-ai/claude-code | ||
| WORKDIR /workspace | ||
| CMD ["sleep", "infinity"] | ||
| EOF | ||
|
|
||
| docker build -t local/claude-code-sandbox:latest /tmp/hypeman-claude-code | ||
| hypeman push local/claude-code-sandbox:latest sandbox/claude-code:latest | ||
|
|
||
| until hypeman image get sandbox/claude-code:latest | grep -qi ready; do | ||
| sleep 2 | ||
| done | ||
|
|
||
| hypeman run --name claude-code-sandbox sandbox/claude-code:latest | ||
| hypeman exec claude-code-sandbox -- claude --version | ||
| ``` | ||
|
|
||
| Clean up the sandbox when you are done: | ||
|
|
||
| ```sh | ||
| hypeman stop claude-code-sandbox | ||
| hypeman rm claude-code-sandbox | ||
| ``` | ||
|
|
||
| ## Terraform | ||
|
|
||
| Use Terraform if you want the same CloudFormation template managed from your existing infrastructure workflow: | ||
|
|
||
| ```sh | ||
| cd deploy/aws/terraform | ||
| terraform init | ||
| terraform apply \ | ||
| -var="region=us-east-1" \ | ||
| -var="vpc_id=vpc-..." \ | ||
| -var="subnet_id=subnet-..." \ | ||
| -var="allowed_api_cidr=$(curl -fsSL https://checkip.amazonaws.com)/32" \ | ||
| -var="instance_type=c8i.2xlarge" | ||
| ``` | ||
|
|
||
| Inspect outputs with: | ||
|
|
||
| ```sh | ||
| terraform output | ||
| ``` | ||
|
|
||
| Delete the deployment with: | ||
|
|
||
| ```sh | ||
| terraform destroy | ||
| ``` | ||
|
|
||
| ## CloudFormation Source | ||
|
|
||
| The source template lives at `deploy/aws/cloudformation/template.yaml`. The `Deploy Assets` GitHub workflow validates it on pull requests and publishes it from `main` to: | ||
|
|
||
| ```text | ||
| https://kernel-hypeman-cloudformation-prod.s3.us-east-1.amazonaws.com/v1/hypeman/template.yaml | ||
| ``` | ||
|
|
||
| To delete a console-launched stack: | ||
|
|
||
| ```sh | ||
| aws cloudformation delete-stack \ | ||
| --region us-east-1 \ | ||
| --stack-name hypeman | ||
| ``` | ||
|
|
||
| ## Packer | ||
|
|
||
| The Packer starter at `deploy/aws/ami/packer/hypeman.pkr.hcl` builds a Hypeman AMI if you want to pre-bake the host setup: | ||
|
|
||
| ```sh | ||
| packer init deploy/aws/ami/packer/hypeman.pkr.hcl | ||
| packer build \ | ||
| -var="region=us-east-1" \ | ||
| -var="source_ami=ami-..." \ | ||
| deploy/aws/ami/packer/hypeman.pkr.hcl | ||
| ``` | ||
|
|
||
| ## Defaults | ||
|
|
||
| | Setting | Default | | ||
| | --- | --- | | ||
| | Region | `us-east-1` | | ||
| | Instance type | `c8i.2xlarge` | | ||
| | Hypeman API port | `8080` | | ||
| | Admin access | AWS Systems Manager Session Manager | | ||
| | SSH | Disabled unless explicitly enabled | | ||
| | Root volume | 100 GiB encrypted EBS | | ||
| | Hypeman version | Latest release with a matching artifact | | ||
|
|
||
| The deployment expects an Intel C8i, M8i, or R8i instance type with EC2 nested virtualization support. Stop or delete the stack when you are done testing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| packer { | ||
| required_plugins { | ||
| amazon = { | ||
| version = ">= 1.3.0" | ||
| source = "github.com/hashicorp/amazon" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| variable "region" { | ||
| type = string | ||
| default = "us-east-1" | ||
| } | ||
|
|
||
| variable "instance_type" { | ||
| type = string | ||
| default = "c8i.large" | ||
| } | ||
|
|
||
| variable "hypeman_version" { | ||
| type = string | ||
| default = "latest" | ||
| } | ||
|
|
||
| source "amazon-ebs" "hypeman" { | ||
| region = var.region | ||
| instance_type = var.instance_type | ||
| ssh_username = "ubuntu" | ||
| ami_name = "hypeman-{{timestamp}}" | ||
| ami_description = "Hypeman API host image" | ||
|
|
||
| source_ami_filter { | ||
| filters = { | ||
| name = "ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*" | ||
| root-device-type = "ebs" | ||
| virtualization-type = "hvm" | ||
| } | ||
| owners = ["099720109477"] | ||
| most_recent = true | ||
| } | ||
|
|
||
| launch_block_device_mappings { | ||
| device_name = "/dev/sda1" | ||
| volume_size = 100 | ||
| volume_type = "gp3" | ||
| encrypted = true | ||
| delete_on_termination = true | ||
| } | ||
| } | ||
|
|
||
| build { | ||
| sources = ["source.amazon-ebs.hypeman"] | ||
|
|
||
| provisioner "shell" { | ||
| inline = [ | ||
| "set -euxo pipefail", | ||
| "sudo apt-get update", | ||
| "sudo DEBIAN_FRONTEND=noninteractive apt-get install -y ca-certificates curl docker.io e2fsprogs erofs-utils iproute2 iptables jq openssl qemu-system-x86 qemu-utils tar", | ||
| "sudo systemctl enable docker", | ||
| "if [ '${var.hypeman_version}' = 'latest' ]; then curl -fsSL https://raw.githubusercontent.com/kernel/hypeman/main/scripts/install.sh | sudo bash; else curl -fsSL https://raw.githubusercontent.com/kernel/hypeman/main/scripts/install.sh | sudo VERSION='${var.hypeman_version}' bash; fi", | ||
| "sudo systemctl stop hypeman", | ||
| "sudo rm -f /root/.config/hypeman/cli.yaml", | ||
| "sudo cloud-init clean --logs", | ||
| ] | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| [ | ||
| { | ||
| "ParameterKey": "VpcId", | ||
| "ParameterValue": "vpc-xxxxxxxx" | ||
| }, | ||
| { | ||
| "ParameterKey": "SubnetId", | ||
| "ParameterValue": "subnet-xxxxxxxx" | ||
| }, | ||
| { | ||
| "ParameterKey": "AllowedApiCidr", | ||
| "ParameterValue": "203.0.113.10/32" | ||
| }, | ||
| { | ||
| "ParameterKey": "InstanceType", | ||
| "ParameterValue": "c8i.2xlarge" | ||
| } | ||
| ] |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
install the native compression libraries used by snapshot compression