-
Notifications
You must be signed in to change notification settings - Fork 563
feat(runnable-rails): stream metadata in RunnableRails output #1370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
78d5038 to
2ddc4f3
Compare
8e1033c to
1c5837e
Compare
2ddc4f3 to
22b4d89
Compare
1c5837e to
6a6050b
Compare
22b4d89 to
8238039
Compare
6a6050b to
ce4d6c1
Compare
8238039 to
eb2d49c
Compare
ce4d6c1 to
0b114ba
Compare
eb2d49c to
f8004f8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR enhances the RunnableRails streaming functionality to include generation metadata in streamed chunks and improves compatibility with consumers expecting metadata in streaming responses. The changes focus on filtering out END_OF_STREAM markers and updating chunk formatting to properly support metadata for AIMessageChunk outputs.
- Updates streaming methods to include generation metadata in chunks
- Adds logic to skip END_OF_STREAM markers during streaming
- Enhances chunk formatting to handle metadata alongside text content
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/runnable_rails/test_streaming.py | Removes outdated comments and adds new tests for metadata preservation and chunk type validation |
| tests/runnable_rails/test_metadata.py | Adds comprehensive test coverage for streaming metadata preservation in both sync and async scenarios |
| nemoguardrails/integrations/langchain/runnable_rails.py | Updates streaming implementation to handle metadata and filter END_OF_STREAM markers |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
0b114ba to
0e0ea4f
Compare
a0c0e24 to
d256fe3
Compare
tgasser-nv
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved, just a couple of nits
0e0ea4f to
00589c2
Compare
d256fe3 to
3ac1414
Compare
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
3ac1414 to
1e2f0eb
Compare
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.
Enhance streaming in RunnableRails to include generation metadata in streamed chunks. Skips END_OF_STREAM markers and updates chunk formatting to support metadata for AIMessageChunk outputs. This improves compatibility with consumers expecting metadata in streaming responses.