docs(hermes): document RCA evaluation tracks

[cerencamkiran]

Summary

Adds documentation for the Provider & Transport Attribution track of the Hermes RCA suite.

The new documentation covers:

• the purpose of the provider and transport attribution track
• the investigation capabilities validated by the suite
• scenario coverage for scenarios 001–006
• provider-specific evidence sources used during investigations
• tooling introduced to support provider and transport RCA workflows

This update provides additional context around the highest-frequency Hermes adapter failure classes and documents how provider-facing incidents are evaluated within the Hermes RCA program.

Documentation Only

No functional changes.

No test logic, fixtures, benchmark behavior, or evaluation criteria were modified.

[github-actions]

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

[greptile-apps]

Greptile Summary

This PR adds the "Part 1/5: Provider & Transport Attribution" section to the Hermes RCA suite README, documenting the purpose, scenario coverage, evidence sources, and tooling for provider-facing incident investigations.

• Covers six scenarios (001–006) spanning empty responses, HTTP 400s, overload conditions, IMDS overrides, dropped headers, and SSE line overflow.
• Documents the four evidence sources (hermes_provider_traffic, hermes_config, hermes_runtime_state, hermes_session_log) and the two investigation tools introduced for this track.
• The README now jumps directly from Part 1/5 to Part 5/5 — Parts 2–4 are not yet present, which is visually discontinuous but appears intentional if those tracks are being added in follow-up PRs.

Confidence Score: 4/5

Documentation-only change with no functional code modified; safe to merge.

The added content is accurate and well-structured. The only concern is that the document now skips from Part 1/5 directly to Part 5/5 with no stub or forward-reference for Parts 2–4, which could confuse readers until those sections land.

tests/synthetic/hermes_rca/README.md — the gap between Part 1 and Part 5 is worth a second look to confirm whether stub sections are needed before merging.

Important Files Changed

Filename	Overview
tests/synthetic/hermes_rca/README.md	Adds 90-line "Part 1/5: Provider & Transport Attribution" section; the document now jumps from Part 1/5 directly to Part 5/5, leaving Parts 2–4 undocumented in the current state.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Hermes Incident] --> B{Attribution Layer}
    B --> C[Part 1/5: Provider & Transport]
    B --> D[Parts 2-4: Runtime / Orchestration / Memory / Control]
    B --> E[Part 5/5: Surface Attribution]

    C --> C1[Scenario 001: Codex Empty Response]
    C --> C2[Scenario 002: OpenRouter 400]
    C --> C3[Scenario 003: MiniMax 529 Overload]
    C --> C4[Scenario 004: Bedrock IMDS Override]
    C --> C5[Scenario 005: Codex Headers Dropped]
    C --> C6[Scenario 006: SSE Line Overflow]

    C1 & C2 & C3 & C4 & C5 & C6 --> EV[Evidence Sources]
    EV --> EV1[hermes_provider_traffic]
    EV --> EV2[hermes_config]
    EV --> EV3[hermes_runtime_state]
    EV --> EV4[hermes_session_log]

    EV1 & EV2 --> T[Investigation Tools]
    T --> T1[get_hermes_provider_traffic]
    T --> T2[get_hermes_config]

    E --> AR[analog_registry.py\nParts 1-4 mappings]

Loading

_{Reviews (1): Last reviewed commit: "docs(hermes): document RCA evaluation tr..." | Re-trigger Greptile}

[greptile-apps]

Missing Parts 2–4 leave the document structurally incomplete

The README now jumps directly from ## Part 1/5 to ## Part 5/5, skipping Parts 2, 3, and 4 entirely. The Part 5 section already references "analog mappings from Parts 1–4 of the Hermes RCA suite" (line 120), so a reader landing here after this PR is merged will find a gap with no placeholder or forward-reference explaining that those sections are forthcoming. If Parts 2–4 are intended for follow-up PRs, adding a brief stub or note (e.g. ## Part 2/5–4/5: (forthcoming)) would prevent confusion.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[cerencamkiran]

To be continued