Skip to content

fix: skip local llama server in cloud mode#1363

Open
gabsprogrammer wants to merge 3 commits into
Light-Heart-Labs:mainfrom
gabsprogrammer:gabs/fix-cloud-compose-no-local-llama
Open

fix: skip local llama server in cloud mode#1363
gabsprogrammer wants to merge 3 commits into
Light-Heart-Labs:mainfrom
gabsprogrammer:gabs/fix-cloud-compose-no-local-llama

Conversation

@gabsprogrammer

@gabsprogrammer gabsprogrammer commented May 24, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes cloud/private-cloud installs so Dream Server does not start or depend on the local llama-server stack when DREAM_MODE=cloud, while keeping normal hosted cloud installs on the hosted provider route.

This branch was rebased on current main after #1364 and preserves the upstream cloud-mode contract/test changes.

Changes include:

  • keep the cloud compose stack on the base + cloud overlay, with local llama-server and local dependency overlays excluded
  • pass DREAM_MODE through compose resolution paths so restart/update/preflight use the same stack as install
  • preserve port overrides during Linux/macOS/Windows install flows, including WEBUI_PORT for port 3000 conflicts and Hermes proxy ports
  • keep Phase 12 cloud-aware so it checks/prewarms litellm instead of the local llama service
  • keep Hermes config and data/persona/SOUL.md generation outside the local-model-only flow, so cloud installs still render the bind-mounted file
  • configure Hermes in cloud mode with LiteLLM route names (default for hosted cloud, local-lan for private-cloud/LM Studio) instead of sending the raw model id to LiteLLM
  • keep the hosted LiteLLM default/cloud routes on hosted providers
  • add an explicit local-lan route backed by CLOUD_LLM_BASE_URL, CLOUD_LLM_MODEL, and CLOUD_LLM_API_KEY for private-cloud OpenAI-compatible upstreams like LM Studio (http://<host-lan-ip>:1234/v1)
  • make dashboard-api cloud-aware so readiness/model/status paths use litellm rather than reporting llama-server as missing in cloud mode
  • make scripts/dream-preflight.sh cloud-aware so it checks litellm, logs litellm, and skips local GPU probing instead of failing on an intentionally absent llama-server
  • prompt for private-cloud API keys silently and avoid echoing existing/default secrets into terminal scrollback
  • keep Linux, macOS, and Windows installers aligned for the same cloud/private-cloud path

Review follow-up

Addresses the requested changes from review/re-review:

  • Phase 12 no longer waits on local llama-server in cloud mode.
  • Hermes SOUL.md/config rendering is not trapped inside the local-model-only block.
  • Hosted cloud default is restored; LM Studio/private LAN is explicit via local-lan.
  • Hermes now routes private-cloud installs through the local-lan LiteLLM alias when CLOUD_LLM_BASE_URL is present.
  • Interactive private-cloud API key entry is silent and the default secret is not printed.
  • The dashboard readiness path no longer blocks cloud installs on an absent local llama-server.
  • The companion scripts/dream-preflight.sh no longer reports cloud installs as missing llama-server.
  • The branch is rebased on current main and keeps dream-server/tests/test-linux-cloud-mode.sh.

Validation

  • bash dream-server/tests/test-linux-cloud-mode.sh
  • bash -n dream-server/scripts/dream-preflight.sh dream-server/tests/test-linux-cloud-mode.sh
  • bash -n dream-server/installers/phases/11-services.sh dream-server/installers/macos/install-macos.sh dream-server/tests/test-linux-cloud-mode.sh
  • python -m py_compile dream-server/extensions/services/dashboard-api/config.py dream-server/extensions/services/dashboard-api/helpers.py dream-server/extensions/services/dashboard-api/main.py dream-server/extensions/services/dashboard-api/settings.py
  • PYTHONPATH=. python -m pytest tests/test_main.py tests/test_helpers.py tests/test_settings_env.py -q in extensions/services/dashboard-api
  • PYTHONPATH=. python -m pytest tests/test_features.py tests/test_config.py tests/test_extensions.py tests/test_templates.py -q in extensions/services/dashboard-api
  • bash dream-server/tests/test-network-timeouts.sh
  • PowerShell parser checks for changed Windows installer files
  • python dream-server/tests/test-render-runtime-configs.py
  • bash dream-server/tests/test-service-registry.sh
  • python -m py_compile dream-server/bin/dream-host-agent.py
  • git diff --check

Note: bash dream-server/tests/contracts/test-installer-contracts.sh could not run in this local Windows shell because jq is not installed here; it exits immediately with [FAIL] jq is required before exercising the PR changes.

@gabsprogrammer

Copy link
Copy Markdown
Contributor Author

Summary

This fixes cloud/API mode installs where DreamServer is configured to use an external OpenAI-compatible endpoint, such as LM Studio hosted on the LAN.

In cloud mode, the installer skips local GGUF model setup, but compose resolution could still include the local llama-server stack and local-mode dependency overlays. That caused dream-llama-server to start, fail because the local GGUF model was missing, and then block the rest of the stack as unhealthy.

This PR:

  • adds a docker-compose.cloud.yml overlay that keeps llama-server out of the default cloud stack
  • makes compose resolution pass DREAM_MODE explicitly instead of relying on ambient env behavior
  • ensures cloud mode uses docker-compose.base.yml + docker-compose.cloud.yml
  • prevents cloud mode from including compose.local.yaml overlays that depend on local inference

Validation

  • bash -n on changed shell scripts
  • python -m py_compile dream-server/bin/dream-host-agent.py
  • verified cloud resolver includes docker-compose.cloud.yml and excludes compose.local.yaml
  • verified docker compose config --services excludes llama-server in cloud mode
  • verified local CPU mode still includes llama-server

@gabsprogrammer

Copy link
Copy Markdown
Contributor Author

What I've added now:

WEBUI_PORT and other ports now respect override on Linux/macOS/Windows.

Linux/Windows now check more real ports before composing, including Hermes, LiteLLM, Dashboard, etc.

In cloud mode, Hermes no longer points to llama-server; it points to LiteLLM/cloud.

Open WebUI/LiteLLM/Hermes no longer declare llama-server as a fixed dependency in the metadata, because in cloud mode this is false.

@gabsprogrammer

gabsprogrammer commented May 24, 2026

Copy link
Copy Markdown
Contributor Author

Achei mais um problema real: cloud mode não estava configurando direito o caso “private cloud” dele, tipo DreamServer na VM/WSL e LM Studio no host/LAN. Agora a PR adiciona CLOUD_LLM_BASE_URL, CLOUD_LLM_MODEL, CLOUD_LLM_API_KEY, faz LiteLLM usar isso, mantém Hermes/Open WebUI apontando para LiteLLM internamente, e preflight/validate param de exigir llama-server em cloud. Também validei porta custom tipo WEBUI_PORT=3007.

@Lightheartdevs Lightheartdevs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: request changes

I do not think this PR is safe to merge as-is. It overlaps heavily with #1364, is currently CONFLICTING against main, and the fleet cloud-mode contract fails on the PR head.

Findings

  1. [P1] Cloud installs still wait on local llama-server in Phase 12

    dream-server/installers/phases/12-health.sh:108-146

    This file still unconditionally runs the local llama-server health check and pre-warm path. In DREAM_MODE=cloud, llama-server is intentionally profiled out, so Phase 12 will wait/prewarm a service that should not exist. That preserves the original failure class this PR is meant to fix.

    Repro from fleet contract on tower2:
    /tmp/dream-cloud-mode-run-1363-audit

    The phase12-cloud-health-target step failed with:
    phase 12 still appears to wait on llama-server unconditionally

  2. [P1] Hermes SOUL/config generation is still inside the local-model-only block

    dream-server/installers/phases/11-services.sh:455-620

    The Hermes config and data/persona/SOUL.md render still sit under if [[ "${DREAM_MODE:-local}" != "cloud" ]]; then. That means cloud installs skip the generated SOUL.md file entirely. This can recreate the file-vs-directory bind-mount failure users saw, and the cloud-specific Hermes routing code inside that block is unreachable in cloud mode.

    Same fleet run failed hermes-soul-renders-in-cloud with:
    SOUL.md render is still inside local-model-only flow

  3. [P2] The default LiteLLM cloud route now assumes a private OpenAI-compatible LAN endpoint

    dream-server/config/litellm/cloud.yaml:2-6

    This changes default from the hosted cloud route to openai/${CLOUD_LLM_MODEL} backed by CLOUD_LLM_BASE_URL, which defaults elsewhere to http://host.docker.internal:1234/v1. That is good as a private-cloud/LM Studio route, but it changes normal hosted --cloud behavior: a user with only ANTHROPIC_API_KEY/OPENAI_API_KEY can now land on a default route pointing at a local endpoint they never started. I would keep the private-cloud route as an explicit alias, or only make it default when the installer confirms a private-cloud endpoint.

  4. [P2] The interactive installer prints the default API key in the prompt

    dream-server/install-core.sh:266-275

    If CLOUD_LLM_API_KEY or OPENAI_API_KEY is set to a real value, the prompt renders it as [${CLOUD_LLM_API_KEY}] and reads the replacement visibly. That exposes secrets in terminal scrollback/recordings. Please mask the default and use a silent prompt for key entry.

Additional notes

  • The PR is currently not mergeable: GitHub reports mergeable=CONFLICTING, mergeStateStatus=DIRTY after #1364 merged.
  • Relative to current main, this branch would also drop dream-server/tests/test-linux-cloud-mode.sh; please rebase on main and preserve that contract test.
  • CI is green, but it is not catching these cloud-mode contract regressions yet. The fleet run is the stronger signal here.

Validation I ran

  • Checked GitHub PR state and current CI: all CI jobs are green, but branch is conflicting.
  • Ran the fleet cloud-mode contract on tower2 against PR head 526d6377f969154fe3f41d35f53f9c5386868992:
    • cloud-compose-services-no-llama: pass
    • dream-cli-cloud-cache-invalidation: pass
    • product-linux-cloud-mode-test: fail, missing test from current main
    • phase12-cloud-health-target: fail
    • hermes-soul-renders-in-cloud: fail

Recommended path: rebase on current main, keep #1364's cloud-mode contract/test changes, then add the private-cloud CLOUD_LLM_* work as a narrower follow-up with tests for both hosted-cloud and LM Studio/private-cloud defaults.

@gabsprogrammer gabsprogrammer force-pushed the gabs/fix-cloud-compose-no-local-llama branch from 526d637 to ed0c3c7 Compare May 24, 2026 01:28
@gabsprogrammer

Copy link
Copy Markdown
Contributor Author

I rebased the PR on current main and addressed the requested changes.

Cloud mode now preserves the #1364 contract: it does not wait on or prewarm local llama-server, and Hermes SOUL/config rendering is outside the local-model-only flow. I also restored the hosted-cloud default route and made LM Studio/private LAN explicit through the local-lan route using CLOUD_LLM_*.

The interactive private-cloud API key prompt is now silent and no longer prints existing/default secrets. I also added Linux cloud-mode contract checks for the hosted vs private-cloud route split and the silent key prompt.

Validation passed locally, including:

  • bash dream-server/tests/test-linux-cloud-mode.sh
  • git diff --check

@Lightheartdevs Lightheartdevs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after rebase

Thanks for rebasing this on current main. The original blockers from my prior review look fixed now:

  • GitHub reports the branch mergeable against main.
  • bash -n and git diff --check pass.
  • bash dream-server/tests/test-linux-cloud-mode.sh passes on tower2.
  • Fleet cloud-mode contract passes on tower2: /tmp/dream-cloud-mode-run-1363-rerun/REPORT.md with OVERALL: PASS.
  • I did not find private hostnames, usernames, LAN IPs, or concrete secrets in the added diff; the secret-bearing values I saw are env-var references or dummy local keys.

I still cannot approve this head yet because the new private-cloud path appears only partially wired.

Findings

  1. [P1] Private-cloud local-lan route is added, but Hermes is not routed to it

    dream-server/config/litellm/cloud.yaml:12-16 adds an explicit local-lan model route backed by CLOUD_LLM_*, which is the right direction for LM Studio/private LAN. But the install still keeps CLOUD_LLM_MODEL separate from LLM_MODEL (dream-server/installers/phases/06-directories.sh:471, dream-server/installers/phases/06-directories.sh:536, dream-server/installers/phases/06-directories.sh:544), and Phase 11 configures Hermes in cloud mode to request ${LLM_MODEL:-default} (dream-server/installers/phases/11-services.sh:542-543).

    That means a private-cloud install can successfully avoid local llama-server, but Hermes still asks LiteLLM for the cloud-tier LLM_MODEL or the user's upstream model id, not for the local-lan route alias. cloud.yaml has named routes for default, cloud, local-lan, gpt4o, fast, minimax, and minimax-fast; it does not define a wildcard route equivalent to local.yaml. So the LM Studio endpoint can be configured and still not be the model route Hermes actually calls.

    This is important because it is the next likely user-facing failure after the compose fix: the stack starts, but the private-cloud path still 404s or routes to hosted defaults instead of the LAN OpenAI-compatible server. Please either set cloud/private-cloud consumers to the local-lan route when CLOUD_LLM_BASE_URL is present, add a deliberate wildcard/pass-through route for cloud config, or patch Hermes/Open WebUI config so the route alias and upstream model id are distinct and both used in the right place.

  2. [P2] scripts/dream-preflight.sh still reports cloud installs as missing llama-server

    This PR updates scripts/dream-preflight.sh to pass --dream-mode, but the script still resolves LLM_PORT, LLM_HEALTH, and LLM_CONTAINER from llama-server (dream-server/scripts/dream-preflight.sh:41-44), requires that container in the core container check (dream-server/scripts/dream-preflight.sh:58-65), labels/checks the API as llama-server (dream-server/scripts/dream-preflight.sh:68-77), and runs docker exec against the llama container for GPU detection (dream-server/scripts/dream-preflight.sh:88-90).

    In DREAM_MODE=cloud, llama-server is intentionally absent. So a user running ./scripts/dream-preflight.sh after a cloud install gets a false failure even when the fixed compose stack is correct. The top-level dream-preflight.sh was made cloud-aware; this companion script should either mirror that behavior or avoid accepting/resolving cloud compose flags.

Validation I ran

  • bash -n on changed shell entrypoints and installer phases.
  • git diff --check origin/main..HEAD.
  • DREAM_PYTHON_CMD=python3 bash dream-server/tests/test-linux-cloud-mode.sh on tower2.
  • Fleet cloud-mode contract on tower2 against head ed0c3c73661216867a58c71edd50e2b4b53d2969: /tmp/dream-cloud-mode-run-1363-rerun, OVERALL: PASS.
  • Manual review of cloud.yaml, Phase 06 env generation, Phase 11 Hermes config rendering, and both preflight scripts.

The old cloud-compose/SOUL/Phase-12 problems are fixed. I would keep this PR open, tighten the two follow-through pieces above, and then it should be in much better shape to approve.

@gabsprogrammer gabsprogrammer force-pushed the gabs/fix-cloud-compose-no-local-llama branch 2 times, most recently from ca9af95 to dc5bec5 Compare May 24, 2026 01:57
@gabsprogrammer

Copy link
Copy Markdown
Contributor Author

Re-review after rebase

Thanks for rebasing this on current main. The original blockers from my prior review look fixed now:

  • GitHub reports the branch mergeable against main.
  • bash -n and git diff --check pass.
  • bash dream-server/tests/test-linux-cloud-mode.sh passes on tower2.
  • Fleet cloud-mode contract passes on tower2: /tmp/dream-cloud-mode-run-1363-rerun/REPORT.md with OVERALL: PASS.
  • I did not find private hostnames, usernames, LAN IPs, or concrete secrets in the added diff; the secret-bearing values I saw are env-var references or dummy local keys.

I still cannot approve this head yet because the new private-cloud path appears only partially wired.

Findings

  1. [P1] Private-cloud local-lan route is added, but Hermes is not routed to it
    dream-server/config/litellm/cloud.yaml:12-16 adds an explicit local-lan model route backed by CLOUD_LLM_*, which is the right direction for LM Studio/private LAN. But the install still keeps CLOUD_LLM_MODEL separate from LLM_MODEL (dream-server/installers/phases/06-directories.sh:471, dream-server/installers/phases/06-directories.sh:536, dream-server/installers/phases/06-directories.sh:544), and Phase 11 configures Hermes in cloud mode to request ${LLM_MODEL:-default} (dream-server/installers/phases/11-services.sh:542-543).
    That means a private-cloud install can successfully avoid local llama-server, but Hermes still asks LiteLLM for the cloud-tier LLM_MODEL or the user's upstream model id, not for the local-lan route alias. cloud.yaml has named routes for default, cloud, local-lan, gpt4o, fast, minimax, and minimax-fast; it does not define a wildcard route equivalent to local.yaml. So the LM Studio endpoint can be configured and still not be the model route Hermes actually calls.
    This is important because it is the next likely user-facing failure after the compose fix: the stack starts, but the private-cloud path still 404s or routes to hosted defaults instead of the LAN OpenAI-compatible server. Please either set cloud/private-cloud consumers to the local-lan route when CLOUD_LLM_BASE_URL is present, add a deliberate wildcard/pass-through route for cloud config, or patch Hermes/Open WebUI config so the route alias and upstream model id are distinct and both used in the right place.
  2. [P2] scripts/dream-preflight.sh still reports cloud installs as missing llama-server
    This PR updates scripts/dream-preflight.sh to pass --dream-mode, but the script still resolves LLM_PORT, LLM_HEALTH, and LLM_CONTAINER from llama-server (dream-server/scripts/dream-preflight.sh:41-44), requires that container in the core container check (dream-server/scripts/dream-preflight.sh:58-65), labels/checks the API as llama-server (dream-server/scripts/dream-preflight.sh:68-77), and runs docker exec against the llama container for GPU detection (dream-server/scripts/dream-preflight.sh:88-90).
    In DREAM_MODE=cloud, llama-server is intentionally absent. So a user running ./scripts/dream-preflight.sh after a cloud install gets a false failure even when the fixed compose stack is correct. The top-level dream-preflight.sh was made cloud-aware; this companion script should either mirror that behavior or avoid accepting/resolving cloud compose flags.

Validation I ran

  • bash -n on changed shell entrypoints and installer phases.
  • git diff --check origin/main..HEAD.
  • DREAM_PYTHON_CMD=python3 bash dream-server/tests/test-linux-cloud-mode.sh on tower2.
  • Fleet cloud-mode contract on tower2 against head ed0c3c73661216867a58c71edd50e2b4b53d2969: /tmp/dream-cloud-mode-run-1363-rerun, OVERALL: PASS.
  • Manual review of cloud.yaml, Phase 06 env generation, Phase 11 Hermes config rendering, and both preflight scripts.

The old cloud-compose/SOUL/Phase-12 problems are fixed. I would keep this PR open, tighten the two follow-through pieces above, and then it should be in much better shape to approve.

Thanks for the re-review. I pushed a new head that addresses both follow-through issues.

Changes:

  • Hermes now uses LiteLLM route names in cloud mode:
    • default for hosted cloud
    • local-lan when CLOUD_LLM_BASE_URL is present for LM Studio/private LAN
  • scripts/dream-preflight.sh is now cloud-aware:
    • checks litellm instead of llama-server
    • logs litellm
    • skips local GPU probing in cloud mode
    • no longer reports cloud installs as missing local llama-server
  • Added regression coverage in dream-server/tests/test-linux-cloud-mode.sh for both the Hermes route selection and the companion preflight behavior.

Validation passed:

  • bash dream-server/tests/test-linux-cloud-mode.sh
  • bash -n dream-server/scripts/dream-preflight.sh dream-server/tests/test-linux-cloud-mode.sh
  • bash dream-server/tests/test-network-timeouts.sh
  • git diff --check

I could not run test-installer-contracts.sh locally because this Windows shell does not have jq; it exits immediately with [FAIL] jq is required before exercising the PR changes.

@gabsprogrammer gabsprogrammer force-pushed the gabs/fix-cloud-compose-no-local-llama branch from dc5bec5 to 546301b Compare May 25, 2026 16:16
@gabsprogrammer

Copy link
Copy Markdown
Contributor Author

Rebased this on current main and tightened the cloud/private-cloud split after re-checking the installer/runtime flow.

What changed in this update:

  • Kept DREAM_MODE=cloud as the deployment mode, because the important runtime boundary is whether DreamServer owns local inference (llama-server) or routes to an external OpenAI-compatible endpoint through LiteLLM.
  • Added private-cloud as the primary LiteLLM route for custom OpenAI-compatible upstreams such as LM Studio on the host/LAN.
  • Kept local-lan as a backward-compatible alias for early testers instead of making it the main route name.
  • Updated Linux and macOS Hermes config so hosted cloud uses the default LiteLLM route, while custom/private endpoints use private-cloud unless HERMES_LLM_MODEL explicitly overrides it.
  • Preserved the newer LEMONADE_EXTERNAL flow from main during the rebase.

Local validation run:

  • bash -n dream-server/install-core.sh dream-server/installers/phases/11-services.sh dream-server/installers/macos/install-macos.sh dream-server/scripts/dream-preflight.sh dream-server/tests/test-linux-cloud-mode.sh
  • bash dream-server/tests/test-linux-cloud-mode.sh
  • python3 -m py_compile dream-server/extensions/services/dashboard-api/config.py dream-server/extensions/services/dashboard-api/helpers.py dream-server/extensions/services/dashboard-api/main.py dream-server/extensions/services/dashboard-api/settings.py dream-server/bin/dream-host-agent.py
  • bash dream-server/tests/test-network-timeouts.sh
  • python3 dream-server/tests/test-render-runtime-configs.py
  • bash dream-server/tests/test-service-registry.sh
  • git diff --check origin/main...HEAD

The dashboard-api pytest subset was not run locally because this machine's current Python environment does not have pytest installed; the pushed CI run should cover that path.

@Lightheartdevs

Copy link
Copy Markdown
Collaborator

Codex re-review notes

I re-reviewed the latest head after the rebase and checked it against current main (ed1c39d6, including the newer CLI cache and Hermes timeout fixes).

What I checked:

  • PR head merges cleanly with current main.
  • Remote CI is green on the PR head.
  • git diff --check origin/main...HEAD passes after the merge-context check.
  • bash -n passes for the touched Linux/macOS installer/preflight shell paths I checked.
  • DREAM_PYTHON_CMD=python bash dream-server/tests/test-linux-cloud-mode.sh passes locally.
  • bash dream-server/tests/test-network-timeouts.sh passes locally.
  • python dream-server/tests/test-render-runtime-configs.py passes locally.
  • python -m py_compile passes for the touched dashboard-api modules and dream-host-agent.py.
  • Manual cloud compose resolution/config check with required env placeholders confirms the cloud stack includes litellm and excludes llama-server.

Local environment caveats:

  • python -m pytest dream-server/extensions/services/dashboard-api/tests/test_helpers.py dream-server/extensions/services/dashboard-api/tests/test_main.py -q was 139 passed / 1 failed. The failure is the existing Windows symlink privilege issue (WinError 1314) in TestDirSizeGb.test_symlinks_are_skipped, not a cloud-mode failure.
  • test-service-registry.sh is not meaningful in this Git-Bash environment: the YAML parser path reports every manifest/compose file as invalid, including unchanged files. CI/Linux is the useful signal for that one.

Assessment of the earlier requested-change items:

  • The old Phase 12/local-llama cloud blocker appears addressed by current-main behavior and the cloud-mode contract test.
  • The private-cloud route follow-through now looks wired: config/litellm/cloud.yaml defines private-cloud, Phase 11 selects private-cloud for Hermes when CLOUD_LLM_BASE_URL is present, and the companion preflight script checks litellm instead of llama-server in cloud mode.
  • The branch is not obsolete. It overlaps with prior cloud fixes, but it still carries useful follow-through: explicit CLOUD_LLM_* support, hosted-vs-private route split, cloud-aware preflight, port override preservation, and manifest dependency cleanup.

Risk/readiness: no new blocker found, but this is materially higher blast radius than a docs/doctor PR. It touches Linux, macOS, Windows installer/env generation, LiteLLM startup config, service manifests, preflight, and dashboard-api helper behavior. I would not batch it with unrelated changes. Recommendation is to merge only after a fresh CI run on the rebased head or one targeted cloud/private-cloud validation pass; with that, I think the previous changes-requested concerns can be considered addressed.

@gabsprogrammer gabsprogrammer force-pushed the gabs/fix-cloud-compose-no-local-llama branch from 546301b to 8049c61 Compare May 25, 2026 20:39
@gabsprogrammer

Copy link
Copy Markdown
Contributor Author

Updated this PR after the latest review and current main changes.

What changed in this update:

  • Rebased the branch on current main.
  • Resolved the dream-server/scripts/dream-support-bundle.sh conflict. The resolution keeps the newer support-bundle behavior from main and preserves the cloud-mode requirement that Compose resolution uses the recorded DREAM_MODE.
  • Re-checked the earlier review items after the rebase:
    • cloud mode still resolves docker-compose.base.yml + docker-compose.cloud.yml and excludes local llama-server overlays;
    • hosted cloud and private-cloud routes remain separate in config/litellm/cloud.yaml;
    • Hermes selects the private-cloud LiteLLM route when CLOUD_LLM_BASE_URL is present;
    • the companion preflight script checks litellm instead of llama-server in cloud mode;
    • the interactive private-cloud API key prompt remains silent;
    • the external Lemonade path from current main is preserved.

Local validation run:

  • git diff --check origin/main...HEAD
  • bash -n dream-server/install-core.sh dream-server/installers/phases/11-services.sh dream-server/installers/macos/install-macos.sh dream-server/installers/macos/lib/env-generator.sh dream-server/scripts/dream-preflight.sh dream-server/dream-preflight.sh dream-server/scripts/validate.sh dream-server/scripts/dream-support-bundle.sh dream-server/tests/test-linux-cloud-mode.sh
  • DREAM_PYTHON_CMD=python3 bash dream-server/tests/test-linux-cloud-mode.sh
  • bash dream-server/tests/test-network-timeouts.sh
  • python3 dream-server/tests/test-render-runtime-configs.py
  • python3 -m py_compile dream-server/extensions/services/dashboard-api/config.py dream-server/extensions/services/dashboard-api/helpers.py dream-server/extensions/services/dashboard-api/main.py dream-server/extensions/services/dashboard-api/settings.py dream-server/bin/dream-host-agent.py
  • bash dream-server/tests/test-support-bundle.sh
  • bash dream-server/tests/contracts/test-external-lemonade-contracts.sh
  • Manual cloud Compose check with placeholder required env values: cloud stack includes litellm and does not include llama-server.

Local caveats:

  • python3 -m pytest dream-server/extensions/services/dashboard-api/tests/test_helpers.py dream-server/extensions/services/dashboard-api/tests/test_main.py -q could not run locally because this Python environment does not have pytest installed.
  • Local PowerShell parser checks could not run because pwsh is not installed on this machine.

Remote CI after this push is green, including shell lint, Python lint, mypy, Docker Compose validation, API/frontend, integration smoke, macOS smoke, Linux distro smoke, and PowerShell lint on both Ubuntu and Windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants