-
Notifications
You must be signed in to change notification settings - Fork 1
Add compose commands #51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
sjmiller609
wants to merge
6
commits into
main
Choose a base branch
from
hypeship/add-compose-command
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
99d7c20
Add compose commands
sjmiller609 26e7b74
Document compose commands
sjmiller609 6924e2a
Move compose logic to lib package
sjmiller609 5b9353d
Split compose package by purpose
sjmiller609 7f3f4ed
Apply suggestion from @sjmiller609
sjmiller609 652f99f
Apply suggestion from @sjmiller609
sjmiller609 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,104 @@ | ||
| # Command Features | ||
|
|
||
| ## Compose | ||
|
|
||
| `hypeman compose` is a lightweight way to declare a small workload and apply it through the existing Hypeman API. It is not a replacement for every Docker Compose feature; it focuses on the pieces Hypeman already manages well: images, instances, restart policy, health checks, and ingresses. | ||
|
|
||
| Compose files default to `hypeman.compose.yaml`: | ||
|
sjmiller609 marked this conversation as resolved.
Outdated
|
||
|
|
||
| ```yaml | ||
| version: 1 | ||
| name: hypeship-otel | ||
|
|
||
| services: | ||
| otelcol: | ||
| image: otel/opentelemetry-collector-contrib:0.108.0 | ||
| cmd: ["--config=env:OTELCOL_CONFIG"] | ||
| env: | ||
| OTELCOL_CONFIG: ${file:otelcol.yaml} | ||
| SIGNOZ_ACCESS_TOKEN: ${env:SIGNOZ_ACCESS_TOKEN} | ||
| resources: | ||
| vcpus: 8 | ||
| memory: 4GB | ||
| restart: | ||
| policy: on_failure | ||
| backoff: 5s | ||
| max_attempts: 10 | ||
| healthcheck: | ||
| http: | ||
| port: 13133 | ||
| path: / | ||
| interval: 10s | ||
| timeout: 2s | ||
| failure_threshold: 3 | ||
| ingress: | ||
| - hostname: otel.example.com | ||
| host_port: 443 | ||
| target_port: 4318 | ||
| tls: true | ||
| ``` | ||
|
|
||
| ### Commands | ||
|
|
||
| Preview the changes: | ||
|
|
||
| ```sh | ||
| hypeman compose plan -f hypeman.compose.yaml | ||
| ``` | ||
|
|
||
| Apply the file: | ||
|
|
||
| ```sh | ||
| hypeman compose up -f hypeman.compose.yaml | ||
| ``` | ||
|
|
||
| Delete resources owned by the file: | ||
|
|
||
| ```sh | ||
| hypeman compose down -f hypeman.compose.yaml | ||
| ``` | ||
|
|
||
| `up` waits for newly created instances to reach `Running` by default. Use `--wait=false` to skip that wait, or `--wait-timeout 30s` to change the per-instance timeout. | ||
|
|
||
| If a managed instance or ingress exists but the rendered spec changed, `up` reports that replacement is required and exits without changing resources. Re-run with `--replace` to recreate changed resources. | ||
|
|
||
| All compose commands honor global output flags such as `--format json`, `--format yaml`, and `--transform`. | ||
|
|
||
| ### How It Works | ||
|
|
||
| `plan` renders the desired resources from the compose file, checks whether referenced images exist, then compares the desired instances and ingresses against existing resources. | ||
|
|
||
| `up` applies the plan in order: | ||
|
|
||
| 1. ensure referenced images exist and are ready | ||
| 2. create or replace instances | ||
| 3. create or replace ingresses | ||
|
|
||
| `down` deletes only instances and ingresses tagged as owned by the compose file. Images are left in place because they can be shared by normal `hypeman run` usage or other compose files. | ||
|
|
||
| Instances and ingresses get compose ownership tags: | ||
|
|
||
| ```text | ||
| hypeman.compose.name | ||
| hypeman.compose.service | ||
| hypeman.compose.resource | ||
| hypeman.compose.hash | ||
| ``` | ||
|
|
||
| The hash is computed from the rendered resource spec before ownership tags are added. Re-running the same file is idempotent: matching resources are reported as unchanged, changed managed resources require `--replace`, and unmanaged resources with the same name are reported as conflicts. | ||
|
|
||
| ### Environment Values | ||
|
|
||
| Environment values can embed local files or environment variables: | ||
|
|
||
| ```yaml | ||
| env: | ||
| OTELCOL_CONFIG: ${file:otelcol.yaml} | ||
| SIGNOZ_ACCESS_TOKEN: ${env:SIGNOZ_ACCESS_TOKEN} | ||
| ``` | ||
|
|
||
| File paths are resolved relative to the compose file. Missing files or environment variables fail before any resources are applied. | ||
|
|
||
| ### OTel Collector Example | ||
|
|
||
| The OTel collector can run from the upstream collector image without rebuilding it. Put the collector config in `otelcol.yaml`, reference it with `${file:otelcol.yaml}`, and pass `--config=env:OTELCOL_CONFIG` as the service command. Restart policy and healthcheck settings are applied to the instance create request, while ingress exposes only the collector port you choose. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| package compose | ||
|
|
||
| import ( | ||
| "github.com/kernel/hypeman-go" | ||
| "github.com/kernel/hypeman-go/option" | ||
| ) | ||
|
|
||
| const ( | ||
| composeTagName = "hypeman.compose.name" | ||
| composeTagService = "hypeman.compose.service" | ||
| composeTagResource = "hypeman.compose.resource" | ||
| composeTagHash = "hypeman.compose.hash" | ||
|
|
||
| composeResourceInstance = "instance" | ||
| composeResourceIngress = "ingress" | ||
| ) | ||
|
|
||
| type Runner struct { | ||
| file string | ||
| spec composeSpec | ||
| client hypeman.Client | ||
| opts []option.RequestOption | ||
| } | ||
|
|
||
| type UpOptions struct { | ||
| Replace bool | ||
| Wait bool | ||
| WaitTimeout string | ||
| Verbose bool | ||
| } | ||
|
|
||
| type Plan struct { | ||
| Name string `json:"name"` | ||
| File string `json:"file"` | ||
| Actions []Action `json:"actions"` | ||
| Summary Summary `json:"summary"` | ||
| } | ||
|
|
||
| type Summary struct { | ||
| Create int `json:"create"` | ||
| Replace int `json:"replace"` | ||
| Delete int `json:"delete"` | ||
| Unchanged int `json:"unchanged"` | ||
| Skip int `json:"skip"` | ||
| Conflict int `json:"conflict"` | ||
| } | ||
|
|
||
| type Action struct { | ||
| Action string `json:"action"` | ||
| Type string `json:"type"` | ||
| Name string `json:"name"` | ||
| Service string `json:"service,omitempty"` | ||
| Reason string `json:"reason"` | ||
|
|
||
| instanceID string | ||
| ingressID string | ||
| instanceInput map[string]any | ||
| ingressInput hypeman.IngressNewParams | ||
| } | ||
|
|
||
| func NewRunner(file string, client hypeman.Client, opts ...option.RequestOption) (*Runner, error) { | ||
| spec, err := loadComposeSpec(file) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &Runner{ | ||
| file: file, | ||
| spec: spec, | ||
| client: client, | ||
| opts: opts, | ||
| }, nil | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,136 @@ | ||
| package compose | ||
|
|
||
| import ( | ||
| "os" | ||
| "path/filepath" | ||
| "testing" | ||
|
|
||
| "github.com/stretchr/testify/assert" | ||
| "github.com/stretchr/testify/require" | ||
| ) | ||
|
|
||
| func TestLoadComposeSpecInterpolatesFilesAndEnv(t *testing.T) { | ||
| dir := t.TempDir() | ||
| require.NoError(t, os.WriteFile(filepath.Join(dir, "otelcol.yaml"), []byte("receivers: {}\n"), 0644)) | ||
| t.Setenv("SIGNOZ_ACCESS_TOKEN", "secret-token") | ||
|
|
||
| composePath := filepath.Join(dir, "hypeman.compose.yaml") | ||
| require.NoError(t, os.WriteFile(composePath, []byte(` | ||
| version: 1 | ||
| name: hypeship-otel | ||
| services: | ||
| otelcol: | ||
| image: otel/opentelemetry-collector-contrib:0.108.0 | ||
| cmd: ["--config=env:OTELCOL_CONFIG"] | ||
| env: | ||
| OTELCOL_CONFIG: ${file:otelcol.yaml} | ||
| SIGNOZ_ACCESS_TOKEN: ${env:SIGNOZ_ACCESS_TOKEN} | ||
| `), 0644)) | ||
|
|
||
| spec, err := loadComposeSpec(composePath) | ||
| require.NoError(t, err) | ||
|
|
||
| service := spec.Services["otelcol"] | ||
| assert.Equal(t, "receivers: {}\n", service.Env["OTELCOL_CONFIG"]) | ||
| assert.Equal(t, "secret-token", service.Env["SIGNOZ_ACCESS_TOKEN"]) | ||
| } | ||
|
|
||
| func TestBuildComposeInstanceInputIncludesFuturePolicyFields(t *testing.T) { | ||
| service := composeServiceSpec{ | ||
| Image: "otel/opentelemetry-collector-contrib:0.108.0", | ||
| Cmd: []string{"--config=env:OTELCOL_CONFIG"}, | ||
| Env: map[string]string{ | ||
| "OTELCOL_CONFIG": "receivers: {}\n", | ||
| }, | ||
| Resources: composeResourcesSpec{ | ||
| Vcpus: 8, | ||
| Memory: "4GB", | ||
| BandwidthUpload: "300Mbps", | ||
| BandwidthDownload: "300Mbps", | ||
| }, | ||
| Restart: &composeRestartSpec{ | ||
| Policy: "on-failure", | ||
| Backoff: "5s", | ||
| MaxAttempts: 10, | ||
| StableAfter: "10m", | ||
| }, | ||
| Health: &composeCheckSpec{ | ||
| HTTP: &composeHTTPCheckSpec{Port: 13133, Path: "/", ExpectedStatus: 200}, | ||
| Interval: "10s", | ||
| Timeout: "2s", | ||
| FailureThreshold: 3, | ||
| }, | ||
| } | ||
|
|
||
| input := buildComposeInstanceInput("hypeship-otel-otelcol", service) | ||
|
|
||
| assert.Equal(t, "hypeship-otel-otelcol", input["name"]) | ||
| assert.Equal(t, service.Image, input["image"]) | ||
| assert.Equal(t, []string{"--config=env:OTELCOL_CONFIG"}, input["cmd"]) | ||
| assert.Equal(t, "4GB", input["size"]) | ||
| assert.Equal(t, 8, input["vcpus"]) | ||
| assert.Equal(t, map[string]any{ | ||
| "backoff": "5s", | ||
| "max_attempts": 10, | ||
| "policy": "on_failure", | ||
| "stable_after": "10m", | ||
| }, input["restart_policy"]) | ||
| assert.Equal(t, service.Health, input["health_check"]) | ||
| assert.Equal(t, map[string]any{ | ||
| "bandwidth_download": "300Mbps", | ||
| "bandwidth_upload": "300Mbps", | ||
| }, input["network"]) | ||
| } | ||
|
|
||
| func TestDesiredResourcesUseDeterministicNamesAndTags(t *testing.T) { | ||
| runner := Runner{ | ||
| spec: composeSpec{ | ||
| Version: 1, | ||
| Name: "hypeship-otel", | ||
| Services: map[string]composeServiceSpec{ | ||
| "otelcol": { | ||
| Image: "otel/opentelemetry-collector-contrib:0.108.0", | ||
| Ingress: []composeIngressRuleSpec{ | ||
| {Hostname: "otel.example.com", HostPort: 443, TargetPort: 4318, TLS: true}, | ||
| }, | ||
| }, | ||
| }, | ||
| }, | ||
| } | ||
|
|
||
| instances, ingresses, images, err := runner.desiredResources() | ||
| require.NoError(t, err) | ||
|
|
||
| require.Equal(t, []string{"otel/opentelemetry-collector-contrib:0.108.0"}, images) | ||
| require.Len(t, instances, 1) | ||
| assert.Equal(t, "hypeship-otel-otelcol", instances[0].Name) | ||
| assert.Equal(t, composeResourceInstance, instances[0].Input["tags"].(map[string]string)[composeTagResource]) | ||
| assert.NotEmpty(t, instances[0].Input["tags"].(map[string]string)[composeTagHash]) | ||
|
|
||
| require.Len(t, ingresses, 1) | ||
| assert.Equal(t, "hypeship-otel-otelcol-0", ingresses[0].Name) | ||
| assert.Equal(t, composeResourceIngress, ingresses[0].Input.Tags[composeTagResource]) | ||
| assert.Equal(t, "hypeship-otel-otelcol", ingresses[0].Input.Rules[0].Target.Instance) | ||
| assert.Equal(t, int64(4318), ingresses[0].Input.Rules[0].Target.Port) | ||
| } | ||
|
|
||
| func TestValidateComposeSpecRejectsInvalidNames(t *testing.T) { | ||
| err := validateComposeSpec(&composeSpec{ | ||
| Version: 1, | ||
| Name: "BadName", | ||
| Services: map[string]composeServiceSpec{ | ||
| "api": {Image: "alpine:latest"}, | ||
| }, | ||
| }) | ||
|
|
||
| require.EqualError(t, err, "compose name must contain only lowercase letters, digits, and dashes") | ||
| } | ||
|
|
||
| func TestConflictBlockers(t *testing.T) { | ||
| blockers := conflictBlockers([]Action{ | ||
| {Action: "create", Type: "image", Name: "alpine:latest"}, | ||
| {Action: "conflict", Type: "instance", Name: "app-api", Reason: "name exists without compose ownership"}, | ||
| }) | ||
|
|
||
| require.Equal(t, []string{" instance app-api: name exists without compose ownership"}, blockers) | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.