feat(chat): explain why a reply is stalled when the endpoint isn't ready#108
Open
michaelroy-amd wants to merge 2 commits into
Open
feat(chat): explain why a reply is stalled when the endpoint isn't ready#108michaelroy-amd wants to merge 2 commits into
michaelroy-amd wants to merge 2 commits into
Conversation
Sending a message to a local endpoint whose model was still coming up showed only a generic "waiting for the agent…" spinner, indistinguishable from a slow reply or a hang. The chat input never consulted the readiness of the instance it was pointed at. Match the chat endpoint to its daemon-surfaced instance by port and, when that instance is not answering, replace the generic spinner with a specific reason: the model is still starting up, the endpoint has stopped, or it reported an error. A live instance, a remote gateway (no local instance to inspect), or an unknown state falls back to the plain spinner unchanged. Parsing and the reason mapping are pure, unit-tested helpers (port_from_base_url, chat_backend_wait_reason). Stacked on #101 (EAI-7352). Relates to EAI-7348 Signed-off-by: Michael Roy <michael.roy@amd.com>
… alone chat_backend_wait_reason matched the chat endpoint to a daemon-surfaced instance by TCP port only. Daemon-tracked instances are always co-located (scraped over loopback), but a remote gateway URL that happens to share a port number with a local managed service -- e.g. both on 8000 -- would false-match that unrelated local instance and could wrongly show a "still starting up" reason for a perfectly healthy remote endpoint. Add host_from_base_url and only attempt the match when the endpoint's host is loopback (reusing llm::is_loopback_host, now pub(crate)); a non-loopback host always falls back to the generic spinner regardless of port. Added host_from_base_url unit tests and a backend_wait_reason_ignores_remote_endpoint_sharing_a_local_port_number test exercising the exact false-match scenario (remote host, same port as a Starting local instance) to prove the generic spinner is kept. Regenerated THIRD_PARTY_NOTICES.txt (pre-existing ordering drift, unrelated to this change). Relates to EAI-7348 Signed-off-by: Michael Roy <michael.roy@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sending a message to a local endpoint whose model was still coming up showed only a generic
⠿ waiting for the agent…spinner — indistinguishable from a slow reply or an outright hang. The chat input never consulted the readiness of the instance it was pointed at.Root Cause
crates/rocm-dash-tui/src/ui/tabs/chat.rs::draw_input()rendered a fixed "waiting for the agent…" line wheneverchat_sendingwas set, regardless of whether the backing endpoint was actually up. The dashboard already tracks each managed instance's status inAppState.instances, but chat made no use of it.Changes
draw_input()now matches the chat endpoint to its daemon-surfaced instance (by port parsed from the configuredbase_url) and, when that instance is not answering, replaces the generic spinner with a specific reason:Starting→ "the model is still starting up — hang tight"Stopped→ "the endpoint has stopped — restart the service to chat"Error→ "the endpoint reported an error — check the service logs"Running) instance, a remote gateway URL (no local instance to inspect), an unmatched port, or anUnknownstate all fall back to the existing plain spinner — unchanged behavior.port_from_base_url,chat_backend_wait_reason), so the render path stays thin.Scope / Notes
InstanceStatusenum. The phase-aware startup detail (DOWNLOADING/LOADING/WARMUP from EAI-7355 / feat(serve): surface a coarse startup phase while a service comes up #107) lives in the other stack; if both land, the reason text here naturally benefits from the richerStartingstate without further changes to this file.Test Plan
cargo build -p rocm-dash-tuicargo clippy -p rocm-dash-tui --all-targets --all-features -- -D warnings(clean)cargo test -p rocm-dash-tui -- --test-threads=1— 547 + 16 + 5 green, incl. new tests:port_from_base_url_parses_local_endpoints,backend_wait_reason_reflects_instance_status,sending_input_surfaces_startup_reason_not_generic_spinner,sending_input_falls_back_to_generic_spinner_for_remote_endpointRelates to EAI-7348