Repository navigation
fix(a11y): announce a reply once it has finished streaming - #6752
Open
opensource-joe wants to merge 1 commit into
Open
opensource-joe wants to merge 1 commit into
opensource-joe wants to merge 1 commit into
Conversation
A screen reader user hears that a reply is generating (Mintplex-Labs#6135) but nothing when it finishes, and the reply itself is never read out unless they go looking for it. ChatHistory now listens for the existing assistant-message-complete event and puts the reply's visible text, without the thought segment or markdown syntax, into a polite live region once. The region stays mounted so screen readers are already watching it. Skipped when TTS auto-play is on, since that already speaks the reply.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #6135, which announces that a reply is generating and that one failed. That PR left out the finished reply on purpose, because the completion point isn't visible from inside
PromptReply, and said it belonged inChatHistory. This is that change.What it does
When a reply finishes, a screen reader user currently hears nothing, and the reply is never read out unless they go and find it.
ChatHistorynow listens for the existingASSISTANT_MESSAGE_COMPLETE_EVENT(the one TTS auto-play already uses), finds the finished message bychatId, and puts its visible text into ansr-onlyrole="status"region once.<think>segment is dropped and the markdown is rendered and reduced to plain text, so a screen reader doesn't read out**or list dashes.The only change outside
ChatHistoryis exporting the event name fromTTSProvider.Verification
I ran this branch for real: server and frontend from source, with a small local OpenAI-compatible stub as the LLM provider streaming a reply that contains a
<think>block and some markdown. I drove it in Chromium with Playwright and watched the live region with a MutationObserver.Here is the answer: first point second point Done.(no thought text, no markdown).autoPlayAssistantTtsResponseon, the region never changes.eslintandprettier --checkclean on both files,vite buildpasses.Not tested: agent (websocket) replies.
agent.jsfires the same event, so it should work if those messages carry achatId, and if they don't, nothing is announced, which is today's behaviour. I also haven't listened to it with VoiceOver or NVDA, only checked the DOM.Made with Claude Code.