[Feature]: Research and add OpenAI WebSocket support (Responses API + Realtime)

[mikehostetler]

Summary

OpenAI now documents WebSocket support for LLM workflows, including:

• Responses API WebSocket mode
• Realtime API over WebSocket

ReqLLM currently has strong support for non-streaming requests and SSE-based streaming, but does not yet expose a dedicated OpenAI WebSocket transport path.

Why this matters

WebSockets can reduce repeated request overhead and enable persistent, low-latency interactions. For some workloads (especially conversational/session-based flows), this can provide better UX and more efficient transport semantics than one-off HTTP requests.

Proposed work

Research and propose an implementation plan for adding OpenAI WebSocket support to ReqLLM, then implement incrementally.

Areas to evaluate:

• API surface design in ReqLLM (how WebSocket session semantics should map to current high-level APIs)
• Provider architecture impact (new callbacks vs extending existing streaming callbacks)
• Event decoding and response assembly parity with existing ReqLLM.Response / ReqLLM.StreamChunk
• Error handling, reconnect behavior, and timeout/session lifecycle
• Usage reporting and compatibility with existing provider defaults
• Fixture/testing strategy for deterministic coverage

Suggested rollout

1. Research/design spike with architecture notes and recommended API shape
2. Implement Responses API WebSocket mode (server-side first)
3. Evaluate/implement Realtime WebSocket support (possibly behind experimental API)

References

• WebSocket mode guide: https://developers.openai.com/api/docs/guides/websocket-mode
• Realtime WebSocket guide: https://developers.openai.com/api/docs/guides/realtime-websocket
• Realtime overview: https://platform.openai.com/docs/guides/realtime/server-events
• Changelog (WebSocket mode launch, Feb 23, 2026): https://developers.openai.com/api/docs/changelog

[mikehostetler]

Proposed integration plan (no PR/code in this step)

This is a concrete delivery plan for OpenAI WebSocket support in ReqLLM, based on current architecture and OpenAI protocol constraints.

Scope and constraints to design around

• Responses WebSocket mode first (wss://api.openai.com/v1/responses)
• Realtime API intentionally deferred to a separate follow-up track
• Respect current OpenAI constraints:
  • one in-flight response per connection
  • no multiplexing today
  • max connection lifetime ~60 minutes
  • event ordering compatible with existing Responses streaming model

Phase 1: Design spike and API contract

• Define transport API shape for high-level streaming:
  • ReqLLM.stream_text/3 gains transport selection (default remains current HTTP/SSE path)
  • explicit option for OpenAI Responses WS mode
• Define provider callback strategy:
  • keep existing SSE callbacks stable
  • add optional WebSocket-specific callback(s) for providers that support WS
• Define session semantics for follow-up work:
  • call-scoped WS first
  • persistent session API second (separate phase)

Deliverable: Architecture note + callback and option contract posted to this issue.

Phase 2: Transport abstraction in streaming pipeline

• Refactor streaming orchestration so transport is pluggable:
  • preserve StreamServer as canonical chunk queue/backpressure/metadata component
  • split transport implementations into HTTP (existing Finch) and WS transport modules
• Ensure stream chunk and metadata behavior is transport-agnostic.

Acceptance criteria:

• Existing provider streaming tests pass unchanged on default HTTP path.
• No regression in StreamResponse behavior.

Phase 3: OpenAI Responses WebSocket MVP

• Implement OpenAI WS transport for Responses API only:
  • connect/authenticate
  • send create request frames
  • route incoming events through existing ResponsesAPI.decode_stream_event flow
• Reuse current response assembly path (StreamChunk + ResponseBuilder) to preserve parity.
• Add robust lifecycle handling:
  • close/error propagation
  • terminal metadata
  • timeout handling

Acceptance criteria:

• WS streaming returns equivalent text/tool-call/usage/finish_reason behavior vs current HTTP streaming.
• response_id propagation and multi-turn continuation remain intact.

Phase 4: Persistent session API (experimental)

• Add explicit session surface (experimental) for workloads that benefit from persistent sockets:
  • start session
  • submit prompt/context turn
  • stream response
  • close session
• Enforce single in-flight response per session.
• Handle session expiry and reconnect policy cleanly.

Acceptance criteria:

• Multi-turn conversation works over a single WS session.
• Session lifecycle and failure semantics are documented and tested.

Phase 5: Tests, fixtures, and parity gates

• Unit tests:
  • WS frame construction and option translation
  • event parsing and transport error cases
• Integration/parity tests:
  • Responses HTTP stream vs Responses WS stream parity
  • tool-calling round-trip parity
  • usage + finish_reason parity
• Fixture strategy:
  • deterministic replay format for WS events (parallel to existing stream fixtures)

Acceptance criteria:

• New WS tests are reliable in fixture mode.
• Live mode can re-record WS fixtures when enabled.

Phase 6: Docs and rollout

• Update docs:
  • OpenAI guide: transport options and caveats
  • Streaming guide: when to use HTTP/SSE vs WS
  • limitations and defaults
• Rollout as experimental feature first; keep HTTP default.

Acceptance criteria:

• Documentation clearly states feature status, constraints, and migration guidance.

Out of scope for this issue (proposed follow-up)

• Realtime API (voice/media/WebRTC-focused workflows)
• Browser-specific connection patterns and ephemeral-token client workflows

Suggested execution order

1. Phase 1 + Phase 2 in one design/infra PR
2. Phase 3 MVP in a focused provider/transport PR
3. Phase 4 session API in separate experimental PR
4. Phase 5/6 hardening + docs before general availability