Skip to content

docs(agents): add observability & tracing integration guide#716

Open
joshuadarron wants to merge 5 commits intodevelopfrom
docs/observability-guide
Open

docs(agents): add observability & tracing integration guide#716
joshuadarron wants to merge 5 commits intodevelopfrom
docs/observability-guide

Conversation

@joshuadarron
Copy link
Copy Markdown
Collaborator

@joshuadarron joshuadarron commented Apr 28, 2026

Summary

  • Adds docs/agents/ROCKETRIDE_OBSERVABILITY.md — a single source of truth for how external services consume runtime logs, lifecycle events, and pipeline traces over the WebSocket DAP channel (no OpenTelemetry / Prometheus / audit-log DB exists; everything flows through /task/service).
  • Wires the new doc into DOC_FILES in apps/vscode/src/agents/agent-manager.ts so the VS Code extension syncs it into .rocketride/docs/.
  • Adds the new file to every agent stub (claude-code.md, CLAUDE.md, AGENTS.md, cursor.mdc, windsurf.md, copilot-instructions.md) so installed workspaces see it alongside the existing README/quickstart/API references.

Test plan

  • pnpm -C apps/vscode build succeeds with the updated DOC_FILES.
  • Run the extension's installAll / autoInstall against a fresh workspace; confirm .rocketride/docs/ROCKETRIDE_OBSERVABILITY.md is created.
  • Confirm each generated stub (.claude/rules/rocketride.md, .cursor/rules/rocketride.mdc, .windsurf/rules/rocketride.md, CLAUDE.md, AGENTS.md, .github/copilot-instructions.md) lists the new ROCKETRIDE_OBSERVABILITY.md row.
  • Spot-check the doc against the actual server: WebSocket path /task/service, auth first-message contract, rrext_monitor types/scopes, and pipelineTraceLevel gating of apaevt_flow.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Added a new observability guide describing how to consume RocketRide runtime data (live WebSocket stream, auth, subscription scopes, supported event types, replay behavior, and on-demand query endpoints)
    • Updated documentation indexes across tools to reference the new observability guide and ensure it remains available in published docs

Documents how external services consume runtime logs, lifecycle events,
and pipeline traces over the WebSocket DAP channel so integrators have a
single source of truth on what the server actually exposes (no OTel,
Prometheus, or audit-log DB).

Wires the new doc into the .rocketride/docs sync list and lists it in
every agent stub (Claude Code, CLAUDE.md, AGENTS.md, Cursor, Windsurf,
Copilot) so installed workspaces surface it alongside the existing
README/quickstart/API references.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added docs Documentation module:vscode VS Code extension labels Apr 28, 2026
@github-actions
Copy link
Copy Markdown

No description provided.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 28, 2026

📝 Walkthrough

Walkthrough

Adds a RocketRide observability guide and wires it into the docs installer and multiple documentation index stubs; the installer will now include ROCKETRIDE_OBSERVABILITY.md in managed docs and not treat it as obsolete.

Changes

Cohort / File(s) Summary
Agent manager update
apps/vscode/src/agents/agent-manager.ts
Added ROCKETRIDE_OBSERVABILITY.md to the DOC_FILES list so the doc installer includes it during install/cleanup.
Observability guide
docs/agents/ROCKETRIDE_OBSERVABILITY.md
New observability integration guide documenting WebSocket DAP auth/subscribe flow, rrext_monitor subscription protocol, event types/schemas (apaevt_task, apaevt_status_update, apaevt_flow, apaevt_sse, output, etc.), recommended subscriptions, replay behavior, and ingestion guidance.
Docs index stubs
docs/stubs/AGENTS.md, docs/stubs/CLAUDE.md, docs/stubs/claude-code.md, docs/stubs/copilot-instructions.md, docs/stubs/cursor.mdc, docs/stubs/windsurf.md
Inserted references to ROCKETRIDE_OBSERVABILITY.md in documentation tables to surface observability/runtime guidance across relevant docs.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

Suggested reviewers

  • jmaionchi
  • stepmikhaylov
  • Rod-Christensen

Poem

🐰 I hopped to write a doc so bright,

Streams and traces in clear sight.
Subscriptions hum and messages flow,
Logs and tasks in tidy row. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'docs(agents): add observability & tracing integration guide' directly and clearly summarizes the main change: adding a comprehensive observability and tracing integration documentation guide for the agents module.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/observability-guide

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 7/8 reviews remaining, refill in 7 minutes and 30 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/agents/ROCKETRIDE_OBSERVABILITY.md`:
- Around line 56-58: The fenced code block containing the single-line example
"ws://<host>:5565/task/service" is missing a language identifier which triggers
markdownlint MD040; update that fenced block by adding a language tag (for
example "text") directly after the opening triple backticks so it reads ```text
to silence the lint and preserve plain formatting.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 48153fba-c8ce-45f1-9f64-58abf38858e7

📥 Commits

Reviewing files that changed from the base of the PR and between 4846445 and 2d8ab22.

📒 Files selected for processing (8)
  • apps/vscode/src/agents/agent-manager.ts
  • docs/agents/ROCKETRIDE_OBSERVABILITY.md
  • docs/stubs/AGENTS.md
  • docs/stubs/CLAUDE.md
  • docs/stubs/claude-code.md
  • docs/stubs/copilot-instructions.md
  • docs/stubs/cursor.mdc
  • docs/stubs/windsurf.md

Comment thread docs/agents/ROCKETRIDE_OBSERVABILITY.md Outdated
Adds the `text` language identifier to the WebSocket URL fenced code
block so markdownlint MD040 stops flagging it, per CodeRabbit review on
PR #716.

Co-Authored-By: Claude <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/agents/ROCKETRIDE_OBSERVABILITY.md`:
- Around line 450-453: The sentence fragment "by `(project_id, source,
startTime)`" should be folded into the previous line so the run-key guidance
reads as one clear instruction: update the `apaevt_task` → `pipeline_runs` line
to state the insert/update/reconcile behavior and then append "by `(project_id,
source, startTime)`" (e.g., "`apaevt_task` → `pipeline_runs` (insert on `begin`,
update on `end`, reconcile from `running` snapshot at startup) by `(project_id,
source, startTime)`.") Leave the following `apaevt_flow` line unchanged.
- Line 401: The docs are inconsistent: the POST /task endpoint uses query param
"trace" while the execute operation uses "pipelineTraceLevel"; update the
documentation to use a single standardized name (pick either "trace" or
"pipelineTraceLevel") across both the POST /task and execute entries or
explicitly state that they are aliases (e.g., "trace (alias:
pipelineTraceLevel)"). Locate the entries referencing POST /task and the execute
operation and make the parameter name consistent and add a short note about the
alias if you choose to support both.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: be5a159d-778d-496d-b4be-5255b104d7d0

📥 Commits

Reviewing files that changed from the base of the PR and between 2d8ab22 and a32dd07.

📒 Files selected for processing (1)
  • docs/agents/ROCKETRIDE_OBSERVABILITY.md

Comment thread docs/agents/ROCKETRIDE_OBSERVABILITY.md Outdated
Comment thread docs/agents/ROCKETRIDE_OBSERVABILITY.md
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
docs/agents/ROCKETRIDE_OBSERVABILITY.md (1)

418-421: ⚠️ Potential issue | 🟡 Minor

Fix detached run-key fragment in ingestion mapping.

Line 420 is a sentence fragment and reads as detached guidance. Fold it into the preceding bullet so the run-key rule is unambiguous.

✏️ Proposed doc fix
-  - `apaevt_task` → `pipeline_runs` (insert on `begin`, update on `end`,
-    reconcile from `running` snapshot at startup).
-    by `(project_id, source, startTime)`.
+  - `apaevt_task` → `pipeline_runs` (insert on `begin`, update on `end`,
+    reconcile from `running` snapshot at startup) keyed by
+    `(project_id, source, startTime)`.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/agents/ROCKETRIDE_OBSERVABILITY.md` around lines 418 - 421, The second
bullet is a detached fragment; merge the run-key fragment into the preceding
bullet so the ingestion mapping reads as one sentence: change the two bullets
describing `apaevt_task` → `pipeline_runs` and its run-key rule into a single
bullet that clearly states insert on `begin`, update on `end`, reconcile from
`running` snapshot at startup, and that the run-key is by `(project_id, source,
startTime)`; keep the following bullet for `apaevt_flow` → `pipeline_run_traces`
intact.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@docs/agents/ROCKETRIDE_OBSERVABILITY.md`:
- Around line 418-421: The second bullet is a detached fragment; merge the
run-key fragment into the preceding bullet so the ingestion mapping reads as one
sentence: change the two bullets describing `apaevt_task` → `pipeline_runs` and
its run-key rule into a single bullet that clearly states insert on `begin`,
update on `end`, reconcile from `running` snapshot at startup, and that the
run-key is by `(project_id, source, startTime)`; keep the following bullet for
`apaevt_flow` → `pipeline_run_traces` intact.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: a505586d-a682-495d-bfa2-231d82b30450

📥 Commits

Reviewing files that changed from the base of the PR and between a32dd07 and 5719fba.

📒 Files selected for processing (1)
  • docs/agents/ROCKETRIDE_OBSERVABILITY.md

@kwit75
Copy link
Copy Markdown
Collaborator

kwit75 commented May 1, 2026

Heads up — your Lin/Win failures here look like the parallel-test flake on should handle large data workflow (packages/client-typescript/tests/RocketRideClient.test.ts:1559), not anything in your diff. Workaround landed in #734 (merged as c11163a, adds -s to the test step). Rebase onto develop should turn this green. Underlying contention tracked in #741.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation module:vscode VS Code extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants