Skip to content

fix(a11y): announce a reply once it has finished streaming - #6752

Open
opensource-joe wants to merge 1 commit into
Mintplex-Labs:masterfrom
opensource-joe:a11y/announce-completed-reply
Open

opensource-joe wants to merge 1 commit into
Mintplex-Labs:masterfrom
opensource-joe:a11y/announce-completed-reply

Conversation

@opensource-joe

Copy link
Copy Markdown
Contributor

Follow-up to #6135, which announces that a reply is generating and that one failed. That PR left out the finished reply on purpose, because the completion point isn't visible from inside PromptReply, and said it belonged in ChatHistory. This is that change.

What it does

When a reply finishes, a screen reader user currently hears nothing, and the reply is never read out unless they go and find it.

ChatHistory now listens for the existing ASSISTANT_MESSAGE_COMPLETE_EVENT (the one TTS auto-play already uses), finds the finished message by chatId, and puts its visible text into an sr-only role="status" region once.

  • The region is always mounted, so screen readers are already watching it when the text arrives.
  • The text is what a sighted user sees: the <think> segment is dropped and the markdown is rendered and reduced to plain text, so a screen reader doesn't read out ** or list dashes.
  • It is skipped when TTS auto-play is on, since that already speaks the reply and the two would talk over each other.
  • The region is emptied before each announcement, so a reply identical to the previous one is still read.
  • No new strings, so no locale changes.

The only change outside ChatHistory is exporting the event name from TTSProvider.

Verification

I ran this branch for real: server and frontend from source, with a small local OpenAI-compatible stub as the LLM provider streaming a reply that contains a <think> block and some markdown. I drove it in Chromium with Playwright and watched the live region with a MutationObserver.

  • The region mounts empty and changes once per reply, about 30 to 40ms after the reply finishes on screen. Nothing is announced while it streams.
  • Announced text: Here is the answer: first point second point Done. (no thought text, no markdown).
  • A second, identical reply removes and re-adds the text node, so it is announced again.
  • With autoPlayAssistantTtsResponse on, the region never changes.
  • eslint and prettier --check clean on both files, vite build passes.

Not tested: agent (websocket) replies. agent.js fires the same event, so it should work if those messages carry a chatId, and if they don't, nothing is announced, which is today's behaviour. I also haven't listened to it with VoiceOver or NVDA, only checked the DOM.

Made with Claude Code.

A screen reader user hears that a reply is generating (Mintplex-Labs#6135) but
nothing when it finishes, and the reply itself is never read out
unless they go looking for it.

ChatHistory now listens for the existing assistant-message-complete
event and puts the reply's visible text, without the thought segment
or markdown syntax, into a polite live region once. The region stays
mounted so screen readers are already watching it. Skipped when TTS
auto-play is on, since that already speaks the reply.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant