You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Load tools from Python scripts at runtime via the `python` feature (powered by `tools-rs` 0.3 + PyO3). Decorate functions with `@tool()` and point `ToolsBuilder` at a directory of `.py` files — they register alongside any native `#[tool]`s.
188
188
189
189
```toml
190
-
chat-rs = { version = "0.4.0", features = ["gemini", "python"] }
190
+
chat-rs = { version = "0.5.0", features = ["gemini", "python"] }
chat-rs = { version = "0.4.0", features = ["gemini", "stream"] }
251
+
chat-rs = { version = "0.5.0", features = ["gemini", "stream"] }
252
252
```
253
253
254
254
```rust
@@ -277,38 +277,41 @@ while let Some(chunk) = stream.next().await {
277
277
278
278
## Input Streaming (bidirectional)
279
279
280
-
Push input *into* the chat while the model is producing output — typed text, audio chunks, tool results, anything that fits a `PartEnum`. Useful for robotics, voice assistants, or any consumer where new context arrives during generation.
280
+
Push input *into* the chat while the model is producing output — typed text, audio chunks (as `PartEnum::File`), tool results, anything that fits a `PartEnum`. Useful for robotics, voice assistants, or any consumer where new context arrives during generation.
281
281
282
-
Transition the builder into `InputStreamed<I>` via `.with_input_stream::<I>()`, then call `chat.stream(&mut messages, input)` with any `Stream<Item = PartEnum> + Send + Unpin + 'static`. On each input event the engine merges it into `Messages` per-variant (text/file/structured → push as user content; tool → resolve matching pending tool by call-id), drops the current provider stream, and re-enters with the updated state. For HTTP/SSE providers this is interrupt-and-restart; native-WS providers (planned OpenAI Realtime, Gemini Live) can hold their session open across calls in their client state — engine surface is identical either way.
282
+
Transition the builder into `InputStreamed` via `.with_input_stream()`. `chat.stream(&mut messages)` then returns a **`ChatStream`**: it *is* the output stream you iterate with `.next()`, and it carries an input side you push to with `.send()`. `.send()` is the inverse of `.next()` — one verb for every input. Pushed input merges into `Messages` (coalescing into the trailing user turn), drops the current provider stream, and re-enters with the updated state. For HTTP/SSE providers this is interrupt-and-restart; native-WS providers (planned OpenAI Realtime, Gemini Live) can hold their session open — the surface is identical either way.
283
+
284
+
Completed work is never lost on an interrupt: every finished tool call and result stays in `Messages` and is re-sent; only the in-flight partial generation is discarded (tools execute *between* steps, never mid-stream, so an interrupt can't sever a running tool).
See `examples/openai/input_stream.rs` for a complete runnable example.
314
+
For concurrent input and output, `split()` the stream into independent `(InputStream, OutputStream)` halves; `cancel()` tears the exchange down. See `examples/openai/input_stream.rs` for the minimal form and `examples/openai/interactive.rs` for an interactive CLI with mid-reply barge-in.
312
315
313
316
## Human in the Loop
314
317
@@ -498,7 +501,7 @@ let client = OpenAIBuilder::new()
498
501
To use WebSocket transport (e.g. for OpenAI's Responses API over WS):
499
502
500
503
```toml
501
-
chat-rs = { version = "0.4.0", features = ["openai", "stream", "tokio-tungstenite"] }
504
+
chat-rs = { version = "0.5.0", features = ["openai", "stream", "tokio-tungstenite"] }
Copy file name to clipboardExpand all lines: ROADMAP.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -60,7 +60,7 @@ Tracking upcoming providers and features for chat-rs.
60
60
-[x]**OpenAI WebSocket streaming** — `AsyncWsTransport` with `.with_message_type("response.create")` connects to `wss://api.openai.com/v1/responses`, authenticates once on handshake, streams events. Connection reuse across calls, terminal event detection, error frame handling.
61
61
-[x]**Image generation** — `File` split into kind/source (`#[non_exhaustive]`). OpenAI `image_generation_call` and Gemini `inlineData` image parts decode into `PartEnum::File(File { kind: Image, .. })`. Claude has no image output upstream.
62
62
-[x]**Mid-stream structured events** — `StreamEvent::Structured(Value)` variant for providers that emit complete typed objects mid-stream (robotics action steps, etc.). Engine accumulates into `ChatResponse.content.parts` as `PartEnum::Structured` for non-streaming-equivalent semantics. (chat-core 0.3.0)
63
-
-[x]**Input-stream type-state** — `Chat<CP, InputStreamed<I>>::stream(&mut messages, input)`consumes a caller-supplied `Stream<Item = PartEnum>` alongside the model output. Audio/text/tool results merge case-by-case into `Messages`; engine restarts the provider stream on each input event. Same interrupt-and-restart pattern as HITL. Native-WS providers (planned Realtime/Live) keep their session open in client state; trait contract stays unchanged. (chat-core 0.3.0)
63
+
-[x]**Input-stream type-state** — `Chat<CP, InputStreamed>::stream(&mut messages)` returns a `ChatStream`: the output stream you iterate with `.next()`, carrying an input side you push to with `.send()`(the inverse of `.next()`), with `split()` into independent `(InputStream, OutputStream)` halves and `cancel()` to tear down. Pushed input rides as `PartEnum` (audio = `File`, mapped caller-side before `send`), coalesces into the trailing user turn, and restarts the provider stream — same interrupt-and-restart pattern as HITL, now push-driven. The producer handle is `Clone + Send + 'static`, so it drops into a task; completed tool work survives interrupts (only the in-flight partial is discarded). Native-WS providers (planned Realtime/Live) keep their session open in client state; trait contract stays unchanged. (redesigned in chat-core 0.4.0)
-**Bidirectional streaming, redesigned.**`Chat<CP, InputStreamed>::stream(&mut messages)` now returns a **`ChatStream`** instead of taking a caller-supplied input stream. It *is* the output stream you iterate with `.next()`, and it carries an input side you push to with `.send()` — the inverse of `.next()`, one verb for every input. `split()` peels it into independent `(InputStream, OutputStream)` halves (the `InputStream` is `Clone + Send + 'static`, so it drops into a task and clones into multiple producers); `cancel()` tears the exchange down. Builder transition: `ChatBuilder::with_input_stream()` (no longer generic).
26
+
27
+
Pushed input rides as `PartEnum` (audio = `File`, text = `Text`, tool result = `Tool`), mapped caller-side before `send`. It coalesces into the trailing user turn via `Messages::push` and restarts the provider stream — the same interrupt-and-restart pattern HITL uses, now push-driven. Completed tool work survives interrupts (tools run *between* steps, never mid-stream); only the in-flight partial generation is discarded.
28
+
23
29
## What's new in 0.3
24
30
25
31
-**`StreamEvent::Structured(Value)`** — providers can yield complete structured objects mid-stream (each event is a whole `serde_json::Value`, not a fragment). The engine accumulates them into `ChatResponse.content.parts` as `PartEnum::Structured` so non-streaming consumers see them too. Drop-in for robotics consumers that produce a stream of typed action steps.
26
-
-**`InputStreamed<I>` type-state** — `Chat<CP, InputStreamed<I>>::stream(&mut messages, input: I)` interleaves the model's output stream with a caller-supplied `Stream<Item = PartEnum>` input source. Audio bytes ride as `PartEnum::File`, text as `PartEnum::Text`, tool results as `PartEnum::Tool`. Each input event triggers a case-by-case merge into `Messages` and re-opens the provider stream with the updated state — same interrupt-and-restart pattern HITL already uses, just automated. Builder transition: `ChatBuilder::with_input_stream::<I>()`. Bounds: `I: Stream<Item = PartEnum> + Send + Unpin + 'static`.
27
32
28
-
Both are additive — existing `Chat<CP, Unstructured>::stream` callers see no behavior change.
33
+
Both are additive over `Chat<CP, Unstructured>::stream`, which is unchanged.
0 commit comments