|
| 1 | +# NVIDIA Brev Provider |
| 2 | + |
| 3 | +Read when: |
| 4 | + |
| 5 | +- choosing `provider: nvidia-brev`; |
| 6 | +- using Crabbox on NVIDIA Brev GPU workspaces; |
| 7 | +- changing `internal/providers/nvidiabrev`. |
| 8 | + |
| 9 | +NVIDIA Brev is a **direct-only Linux SSH-lease** provider. Crabbox shells out |
| 10 | +to the local `brev` CLI to create, list, refresh, stop, and delete Brev |
| 11 | +workspaces. After Brev writes its SSH config, Crabbox uses the normal SSH |
| 12 | +transport for sync, `run`, `ssh`, `status`, `list`, and `stop`. |
| 13 | + |
| 14 | +Crabbox does not store or accept Brev secrets. Authentication stays in the Brev |
| 15 | +CLI's own credential store. |
| 16 | + |
| 17 | +## Prerequisites |
| 18 | + |
| 19 | +- Install the Brev CLI and authenticate it: |
| 20 | + |
| 21 | + ```sh |
| 22 | + brev login |
| 23 | + ``` |
| 24 | + |
| 25 | +- Confirm Brev can list workspaces: |
| 26 | + |
| 27 | + ```sh |
| 28 | + brev ls --json |
| 29 | + ``` |
| 30 | + |
| 31 | +- For OAuth login, select the organization used for lifecycle operations: |
| 32 | + |
| 33 | + ```sh |
| 34 | + brev set |
| 35 | + ``` |
| 36 | + |
| 37 | + API-key and workspace contexts may provide an explicit organization scope |
| 38 | + without an active OAuth organization. |
| 39 | + |
| 40 | +- Keep OpenSSH and `rsync` available locally for Crabbox's SSH workflow. |
| 41 | +- Use an account with enough Brev quota and a GPU type that is available in the |
| 42 | + selected Brev cloud. |
| 43 | + |
| 44 | +## Capabilities |
| 45 | + |
| 46 | +| Capability | Supported | |
| 47 | +| --- | --- | |
| 48 | +| OS targets | Linux only | |
| 49 | +| SSH | Yes, from Brev's generated SSH config | |
| 50 | +| Crabbox sync (rsync over SSH) | Yes | |
| 51 | +| Provider-managed sync | No | |
| 52 | +| Desktop / browser / code | No | |
| 53 | +| Actions hydration | Yes, as a normal Linux SSH lease | |
| 54 | +| Coordinator (broker) | No - direct only | |
| 55 | +| Tailscale | No, Brev CLI-managed SSH access is used | |
| 56 | +| Cleanup | Yes, for Crabbox-owned workspaces with local claims | |
| 57 | + |
| 58 | +Aliases: `brev`, `nvidia`. |
| 59 | + |
| 60 | +## Configuration |
| 61 | + |
| 62 | +```yaml |
| 63 | +provider: nvidia-brev |
| 64 | +target: linux |
| 65 | +nvidiaBrev: |
| 66 | + cli: brev |
| 67 | + type: "" |
| 68 | + gpuName: A100 |
| 69 | + provider: "" |
| 70 | + mode: vm |
| 71 | + launchable: "" |
| 72 | + startupScript: "" |
| 73 | + releaseAction: delete |
| 74 | + target: container |
| 75 | + user: "" |
| 76 | + workRoot: /tmp/crabbox |
| 77 | +``` |
| 78 | +
|
| 79 | +Defaults: |
| 80 | +
|
| 81 | +- `cli`: `brev` |
| 82 | +- `gpuName`: `A100` |
| 83 | +- `mode`: `vm` |
| 84 | +- `releaseAction`: `delete` |
| 85 | +- `target`: `container` |
| 86 | +- `workRoot`: `/tmp/crabbox` |
| 87 | + |
| 88 | +`startupScript` follows Brev's native syntax: use an inline command such as |
| 89 | +`pip install torch`, or prefix a local file path with `@`, for example |
| 90 | +`@setup.sh` or `@/opt/setup.sh`. Local `@file` startup scripts are accepted only |
| 91 | +from trusted user config, environment overrides, or command-line flags; project |
| 92 | +config cannot select local files. |
| 93 | + |
| 94 | +Provider flags: |
| 95 | + |
| 96 | +```text |
| 97 | +--nvidia-brev-cli |
| 98 | +--nvidia-brev-org |
| 99 | +--nvidia-brev-type |
| 100 | +--nvidia-brev-gpu-name |
| 101 | +--nvidia-brev-provider |
| 102 | +--nvidia-brev-mode |
| 103 | +--nvidia-brev-launchable |
| 104 | +--nvidia-brev-startup-script |
| 105 | +--nvidia-brev-release-action |
| 106 | +--nvidia-brev-target |
| 107 | +--nvidia-brev-user |
| 108 | +--nvidia-brev-work-root |
| 109 | +``` |
| 110 | + |
| 111 | +Environment overrides: |
| 112 | + |
| 113 | +```text |
| 114 | +CRABBOX_NVIDIA_BREV_CLI |
| 115 | +CRABBOX_NVIDIA_BREV_ORG |
| 116 | +CRABBOX_NVIDIA_BREV_TYPE |
| 117 | +CRABBOX_NVIDIA_BREV_GPU_NAME |
| 118 | +CRABBOX_NVIDIA_BREV_PROVIDER |
| 119 | +CRABBOX_NVIDIA_BREV_MODE |
| 120 | +CRABBOX_NVIDIA_BREV_LAUNCHABLE |
| 121 | +CRABBOX_NVIDIA_BREV_STARTUP_SCRIPT |
| 122 | +CRABBOX_NVIDIA_BREV_RELEASE_ACTION |
| 123 | +CRABBOX_NVIDIA_BREV_TARGET |
| 124 | +CRABBOX_NVIDIA_BREV_USER |
| 125 | +CRABBOX_NVIDIA_BREV_WORK_ROOT |
| 126 | +``` |
| 127 | + |
| 128 | +`nvidiaBrev.org` scopes read-only inventory through `brev ls --org`. Brev's |
| 129 | +mutating commands and `brev refresh` do not accept that selector, so Crabbox |
| 130 | +rejects lifecycle and SSH resolution when `org` is configured. Use `brev set` |
| 131 | +to select the active organization before running mutating Crabbox commands. |
| 132 | + |
| 133 | +## Lifecycle |
| 134 | + |
| 135 | +### Doctor |
| 136 | + |
| 137 | +`doctor` is read-only. It checks the Brev CLI version and lists the current Brev |
| 138 | +inventory: |
| 139 | + |
| 140 | +```sh |
| 141 | +crabbox doctor --provider nvidia-brev |
| 142 | +crabbox doctor --provider nvidia-brev --json |
| 143 | +``` |
| 144 | + |
| 145 | +It does not create, stop, or delete a workspace. |
| 146 | + |
| 147 | +### Acquire |
| 148 | + |
| 149 | +`warmup` and `run` create cost-bearing Brev workspaces: |
| 150 | + |
| 151 | +```sh |
| 152 | +crabbox warmup --provider nvidia-brev --slug gpu-smoke --keep |
| 153 | +crabbox run --provider nvidia-brev -- nvidia-smi |
| 154 | +``` |
| 155 | + |
| 156 | +Acquire flow: |
| 157 | + |
| 158 | +1. list existing workspaces to allocate a Crabbox slug; |
| 159 | +2. run `brev create <name> --detached` with the configured Brev selectors; |
| 160 | +3. wait for the workspace to report ready; |
| 161 | +4. run `brev refresh`; |
| 162 | +5. read the Brev SSH config from `~/.brev/ssh_config`; |
| 163 | +6. resolve the configured target alias; |
| 164 | +7. wait for SSH readiness; |
| 165 | +8. write a local Crabbox lease claim and run the normal SSH workflow. |
| 166 | + |
| 167 | +Crabbox names workspaces as `crabbox-<slug>-<lease-suffix>` so `list` and |
| 168 | +`cleanup` can distinguish Crabbox-owned Brev workspaces from manual ones. |
| 169 | + |
| 170 | +If `brev create` returns an ambiguous transport error, Crabbox checks inventory |
| 171 | +for the deterministic workspace name and stores a recovery claim. Non-kept |
| 172 | +workspaces are deleted when ownership is confirmed; kept workspaces remain |
| 173 | +manageable by lease ID or slug. Name-only recovery claims report `failed` until |
| 174 | +the workspace appears and can be cleared by explicit release after the recovery |
| 175 | +grace period. |
| 176 | + |
| 177 | +### SSH target selection |
| 178 | + |
| 179 | +`nvidiaBrev.target` controls which Brev SSH config host Crabbox selects: |
| 180 | + |
| 181 | +- `container` (default) selects the workspace host alias. |
| 182 | +- `host` selects the `<workspace-name>-host` alias. |
| 183 | + |
| 184 | +Brev may emit either a direct `HostName`/`Port` target or a `ProxyCommand`. |
| 185 | +Crabbox supports both forms as long as the SSH config entry includes a user and |
| 186 | +identity file. |
| 187 | + |
| 188 | +### Release and cleanup |
| 189 | + |
| 190 | +The default release action is `delete`: |
| 191 | + |
| 192 | +```sh |
| 193 | +crabbox stop --provider nvidia-brev gpu-smoke |
| 194 | +``` |
| 195 | + |
| 196 | +`delete` records the claim as deleting, clears its SSH endpoint, runs |
| 197 | +`brev delete`, and keeps polling until Brev inventory confirms that the |
| 198 | +workspace is absent. The local claim remains available for a later `stop` or |
| 199 | +`cleanup` retry if polling is interrupted. |
| 200 | + |
| 201 | +Crabbox stores the active Brev organization ID in each claim. Because Brev |
| 202 | +mutations do not accept an organization selector, lifecycle commands reject an |
| 203 | +active-org mismatch and retain the claim. Run `brev set` for the lease's |
| 204 | +original organization, then retry the command. |
| 205 | + |
| 206 | +If the active organization changes while a workspace is being created, Crabbox |
| 207 | +does not delete from either organization automatically. It retains a recovery |
| 208 | +claim with both observed organization IDs for manual reconciliation, avoiding |
| 209 | +deletion of an unrelated same-named workspace. |
| 210 | + |
| 211 | +If `nvidiaBrev.releaseAction` is `stop`, Crabbox runs `brev stop`, keeps the |
| 212 | +local claim, and records the lease as stopped for later reuse or cleanup. |
| 213 | + |
| 214 | +`cleanup` only mutates Brev workspaces that have matching local Crabbox claims: |
| 215 | + |
| 216 | +```sh |
| 217 | +crabbox cleanup --provider nvidia-brev --dry-run |
| 218 | +crabbox cleanup --provider nvidia-brev |
| 219 | +``` |
| 220 | + |
| 221 | +Manual Brev workspaces and unclaimed Crabbox-looking workspaces are skipped |
| 222 | +rather than deleted blindly. |
| 223 | + |
| 224 | +## Examples |
| 225 | + |
| 226 | +Run a GPU smoke and delete the workspace after the command: |
| 227 | + |
| 228 | +```sh |
| 229 | +crabbox run --provider nvidia-brev -- nvidia-smi |
| 230 | +``` |
| 231 | + |
| 232 | +Warm a reusable GPU workspace, run a CUDA check, then release it: |
| 233 | + |
| 234 | +```sh |
| 235 | +crabbox warmup --provider nvidia-brev --slug cuda-box --keep |
| 236 | +crabbox run --provider nvidia-brev --id cuda-box --no-sync -- python3 - <<'PY' |
| 237 | +print("cuda-ready") |
| 238 | +PY |
| 239 | +crabbox ssh --provider nvidia-brev --id cuda-box |
| 240 | +crabbox status --provider nvidia-brev --id cuda-box --wait |
| 241 | +crabbox stop --provider nvidia-brev cuda-box |
| 242 | +``` |
| 243 | + |
| 244 | +Choose a different Brev GPU selector: |
| 245 | + |
| 246 | +```sh |
| 247 | +crabbox run \ |
| 248 | + --provider nvidia-brev \ |
| 249 | + --nvidia-brev-gpu-name L40S \ |
| 250 | + -- nvidia-smi |
| 251 | +``` |
| 252 | + |
| 253 | +## Live smoke |
| 254 | + |
| 255 | +The repository live smoke is intentionally separate from default CI because it |
| 256 | +can create a billable GPU workspace: |
| 257 | + |
| 258 | +```sh |
| 259 | +CRABBOX_NVIDIA_BREV_LIVE=1 scripts/live-nvidia-brev-smoke.sh |
| 260 | +``` |
| 261 | + |
| 262 | +The script builds or reuses `bin/crabbox`, runs `doctor`, creates one workspace |
| 263 | +with `--keep=false` so partial acquisition failures roll back, proves |
| 264 | +`nvidia-smi` through `crabbox run`, lists the lease, and then deletes the lease |
| 265 | +with `crabbox stop`. It prints a stable classification: |
| 266 | + |
| 267 | +- `live_nvidia_brev_smoke_passed` when the full GPU path passes; |
| 268 | +- `environment_blocked` for missing CLI/auth/configuration; |
| 269 | +- `provider_quota_blocked` for quota or rate-limit failures; |
| 270 | +- `capacity_blocked` when Brev cannot allocate the requested GPU; |
| 271 | +- `validation_failed` when Crabbox output is malformed or missing the smoke |
| 272 | + lease. |
| 273 | + |
| 274 | +Without `CRABBOX_NVIDIA_BREV_LIVE=1`, the script exits successfully with |
| 275 | +`classification=environment_blocked reason=CRABBOX_NVIDIA_BREV_LIVE_not_enabled` |
| 276 | +and creates no workspace. |
| 277 | + |
| 278 | +## Cost and privacy discipline |
| 279 | + |
| 280 | +- Treat every `warmup` or `run` without an existing `--id` as cost-bearing. |
| 281 | +- Prefer `releaseAction: delete` for disposable test runs. |
| 282 | +- Run `crabbox list --provider nvidia-brev --all` before and after live tests |
| 283 | + when auditing inventory. |
| 284 | +- Do not put Brev secrets, organization identifiers, private SSH material, or |
| 285 | + local credential files in repository config, docs, scripts, logs, or command |
| 286 | + arguments. |
| 287 | +- Use neutral slugs and examples in shared repos. |
| 288 | + |
| 289 | +Related docs: |
| 290 | + |
| 291 | +- [Provider reference](README.md) |
| 292 | +- [Provider backends](../provider-backends.md) |
| 293 | +- [Provider feature overview](../features/providers.md) |
0 commit comments