Introduce start session algorithm #138

beaufortfrancois · 2025-02-05T14:38:45Z

This PR addresses concerns raised in #135 (comment) and #126 (comment):

It makes sure start() and start(MediaStreamTrack audioTrack) throw if document is not fully active:
Request permission to use microphone is called only for start()
A new "start session algorithm" is introduced to formalize how things work when speech recognition starts.

The following tasks have been completed:

Updated web-platform-tests: https://chromium-review.googlesource.com/c/chromium/src/+/6237075

Implementation commitment:

Blink: (link to issue)
Gecko: (link to issue)
WebKit: (link to issue)

Preview | Diff

beaufortfrancois · 2025-02-06T08:38:50Z

@evanbliu @padenot @youennf Let me know if that looks good to you

beaufortfrancois · 2025-02-06T10:02:31Z

index.bs

@@ -339,6 +351,16 @@ See <a href="https://lists.w3.org/Archives/Public/public-speech-api/2012Sep/0072

 </dl>

+<p>When the <dfn>start session algorithm</dfn> with <var>requestMicrophonePermission</var> is invoked, the user agent MUST run the following steps:
+
+1. If the [=current settings object=]'s [=relevant global object=]'s [=associated Document=] is NOT [=fully active=], throw an {{UnknownError}} and abort these steps.


Note that I used UnknownError to match WebKit that already shipped Web Speech.

FYI Firefox uses InvalidStateError instead but did not ship Web Speech. So even though, I'd personally be in favor of InvalidStateError, I picked UnknownError so that web developers already catching this properly based on Safari don't have to update their code.

I think webkit could align to whatever deemed appropriate, this is an edge case so this is likely not a web compat issue.
I also prefer InvalidStateError.

Since you and @evanbliu also prefer InvalidStateError, I've updated this PR to use it.

@youennf This PR has landed and web-platform-tests have been updated. Check out https://wpt.fyi/results/speech-api/SpeechRecognition-detached-iframe.window.html?label=experimental&label=master&aligned
It would be nice to update Safari's implementation as well.

index.bs

beaufortfrancois · 2025-02-10T07:38:14Z

Quick question for you folks!

When adding web platform tests for this in https://chromium-review.googlesource.com/c/chromium/src/+/6237075, I used frameWindow.SpeechRecognition = frameWindow.SpeechRecognition || frameWindow.webkitSpeechRecognition; so that it works in Safari and Chrome BUT technically, it's not a spec-compliant as webkitSpeechRecognition is not a thing. I still think there's value testing implementation with webkitSpeechRecognition but I'd love your thoughts on this.

evanbliu · 2025-02-10T20:00:07Z

Blink owners have requested that we drop the "webkit" prefix from the Web Speech API implementation in Chrome. We'll have to maintain backwards compatibility indefinitely, but eventually we'll probably pretend like it doesn't exist.

index.bs

beaufortfrancois · 2025-02-13T08:05:55Z

I've addressed @evanbliu's feedback and resolved conflicts. Let me know if there's more to address or if we can merge it.

beaufortfrancois · 2025-02-17T07:39:56Z

@evanbliu Please merge.

This CL updates the new installOnDevice and availableOnDevice APIs of the Web Speech API to check for detached context. Spec changes: WebAudio/web-speech-api#138 Bug: 40286514 Change-Id: I024a3b906e4d3838fb9c77578a67c57e0249dc2b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6259599 Commit-Queue: Evan Liu <[email protected]> Reviewed-by: Fr <[email protected]> Cr-Commit-Position: refs/heads/main@{#1421672}

Spec PR: WebAudio/web-speech-api#138 Bug: 384797834 Change-Id: I553842e4427071dacc7693ea6f1f2503cb0cf0dc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6237075 Reviewed-by: Evan Liu <[email protected]> Commit-Queue: Fr <[email protected]> Reviewed-by: Mason Freed <[email protected]> Cr-Commit-Position: refs/heads/main@{#1422104}

SHA: 6356249 Reason: push, by evanbliu Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

SHA: 6356249 Reason: push, by hoch Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

…hed context, a=testonly Automatic update from web-platform-tests Update Web Speech API to check for detached context This CL updates the new installOnDevice and availableOnDevice APIs of the Web Speech API to check for detached context. Spec changes: WebAudio/web-speech-api#138 Bug: 40286514 Change-Id: I024a3b906e4d3838fb9c77578a67c57e0249dc2b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6259599 Commit-Queue: Evan Liu <[email protected]> Reviewed-by: Fr <[email protected]> Cr-Commit-Position: refs/heads/main@{#1421672} -- wpt-commits: a2f4f58ea0a30fe00db46db03af3a14ea548d763 wpt-pr: 50786

…rame, a=testonly Automatic update from web-platform-tests WebSpeech: start() throws if detached iframe Spec PR: WebAudio/web-speech-api#138 Bug: 384797834 Change-Id: I553842e4427071dacc7693ea6f1f2503cb0cf0dc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6237075 Reviewed-by: Evan Liu <[email protected]> Commit-Queue: Fr <[email protected]> Reviewed-by: Mason Freed <[email protected]> Cr-Commit-Position: refs/heads/main@{#1422104} -- wpt-commits: 247db68758a8b03474678dcc6ecfc1fc955227f6 wpt-pr: 50810

…hed context, a=testonly Automatic update from web-platform-tests Update Web Speech API to check for detached context This CL updates the new installOnDevice and availableOnDevice APIs of the Web Speech API to check for detached context. Spec changes: WebAudio/web-speech-api#138 Bug: 40286514 Change-Id: I024a3b906e4d3838fb9c77578a67c57e0249dc2b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6259599 Commit-Queue: Evan Liu <[email protected]> Reviewed-by: Fr <[email protected]> Cr-Commit-Position: refs/heads/main@{#1421672} -- wpt-commits: a2f4f58ea0a30fe00db46db03af3a14ea548d763 wpt-pr: 50786

…rame, a=testonly Automatic update from web-platform-tests WebSpeech: start() throws if detached iframe Spec PR: WebAudio/web-speech-api#138 Bug: 384797834 Change-Id: I553842e4427071dacc7693ea6f1f2503cb0cf0dc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6237075 Reviewed-by: Evan Liu <[email protected]> Commit-Queue: Fr <[email protected]> Reviewed-by: Mason Freed <[email protected]> Cr-Commit-Position: refs/heads/main@{#1422104} -- wpt-commits: 247db68758a8b03474678dcc6ecfc1fc955227f6 wpt-pr: 50810

…hed context, a=testonly Automatic update from web-platform-tests Update Web Speech API to check for detached context This CL updates the new installOnDevice and availableOnDevice APIs of the Web Speech API to check for detached context. Spec changes: WebAudio/web-speech-api#138 Bug: 40286514 Change-Id: I024a3b906e4d3838fb9c77578a67c57e0249dc2b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6259599 Commit-Queue: Evan Liu <evliugoogle.com> Reviewed-by: Fr <beaufort.francoisgmail.com> Cr-Commit-Position: refs/heads/main{#1421672} -- wpt-commits: a2f4f58ea0a30fe00db46db03af3a14ea548d763 wpt-pr: 50786 UltraBlame original commit: ea0d8b3b9d972a904ea745ca709cf396c67a92ec

…rame, a=testonly Automatic update from web-platform-tests WebSpeech: start() throws if detached iframe Spec PR: WebAudio/web-speech-api#138 Bug: 384797834 Change-Id: I553842e4427071dacc7693ea6f1f2503cb0cf0dc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6237075 Reviewed-by: Evan Liu <evliugoogle.com> Commit-Queue: Fr <beaufort.francoisgmail.com> Reviewed-by: Mason Freed <masonfchromium.org> Cr-Commit-Position: refs/heads/main{#1422104} -- wpt-commits: 247db68758a8b03474678dcc6ecfc1fc955227f6 wpt-pr: 50810 UltraBlame original commit: ea97ba98ce71585e440f2067c143288cea64378b

…hed context, a=testonly Automatic update from web-platform-tests Update Web Speech API to check for detached context This CL updates the new installOnDevice and availableOnDevice APIs of the Web Speech API to check for detached context. Spec changes: WebAudio/web-speech-api#138 Bug: 40286514 Change-Id: I024a3b906e4d3838fb9c77578a67c57e0249dc2b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6259599 Commit-Queue: Evan Liu <evliugoogle.com> Reviewed-by: Fr <beaufort.francoisgmail.com> Cr-Commit-Position: refs/heads/main{#1421672} -- wpt-commits: a2f4f58ea0a30fe00db46db03af3a14ea548d763 wpt-pr: 50786 UltraBlame original commit: ea0d8b3b9d972a904ea745ca709cf396c67a92ec

…rame, a=testonly Automatic update from web-platform-tests WebSpeech: start() throws if detached iframe Spec PR: WebAudio/web-speech-api#138 Bug: 384797834 Change-Id: I553842e4427071dacc7693ea6f1f2503cb0cf0dc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6237075 Reviewed-by: Evan Liu <evliugoogle.com> Commit-Queue: Fr <beaufort.francoisgmail.com> Reviewed-by: Mason Freed <masonfchromium.org> Cr-Commit-Position: refs/heads/main{#1422104} -- wpt-commits: 247db68758a8b03474678dcc6ecfc1fc955227f6 wpt-pr: 50810 UltraBlame original commit: ea97ba98ce71585e440f2067c143288cea64378b

…hed context, a=testonly Automatic update from web-platform-tests Update Web Speech API to check for detached context This CL updates the new installOnDevice and availableOnDevice APIs of the Web Speech API to check for detached context. Spec changes: WebAudio/web-speech-api#138 Bug: 40286514 Change-Id: I024a3b906e4d3838fb9c77578a67c57e0249dc2b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6259599 Commit-Queue: Evan Liu <evliugoogle.com> Reviewed-by: Fr <beaufort.francoisgmail.com> Cr-Commit-Position: refs/heads/main{#1421672} -- wpt-commits: a2f4f58ea0a30fe00db46db03af3a14ea548d763 wpt-pr: 50786 UltraBlame original commit: ea0d8b3b9d972a904ea745ca709cf396c67a92ec

…rame, a=testonly Automatic update from web-platform-tests WebSpeech: start() throws if detached iframe Spec PR: WebAudio/web-speech-api#138 Bug: 384797834 Change-Id: I553842e4427071dacc7693ea6f1f2503cb0cf0dc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6237075 Reviewed-by: Evan Liu <evliugoogle.com> Commit-Queue: Fr <beaufort.francoisgmail.com> Reviewed-by: Mason Freed <masonfchromium.org> Cr-Commit-Position: refs/heads/main{#1422104} -- wpt-commits: 247db68758a8b03474678dcc6ecfc1fc955227f6 wpt-pr: 50810 UltraBlame original commit: ea97ba98ce71585e440f2067c143288cea64378b

…hed context, a=testonly Automatic update from web-platform-tests Update Web Speech API to check for detached context This CL updates the new installOnDevice and availableOnDevice APIs of the Web Speech API to check for detached context. Spec changes: WebAudio/web-speech-api#138 Bug: 40286514 Change-Id: I024a3b906e4d3838fb9c77578a67c57e0249dc2b Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6259599 Commit-Queue: Evan Liu <[email protected]> Reviewed-by: Fr <[email protected]> Cr-Commit-Position: refs/heads/main@{#1421672} -- wpt-commits: a2f4f58ea0a30fe00db46db03af3a14ea548d763 wpt-pr: 50786

…rame, a=testonly Automatic update from web-platform-tests WebSpeech: start() throws if detached iframe Spec PR: WebAudio/web-speech-api#138 Bug: 384797834 Change-Id: I553842e4427071dacc7693ea6f1f2503cb0cf0dc Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6237075 Reviewed-by: Evan Liu <[email protected]> Commit-Queue: Fr <[email protected]> Reviewed-by: Mason Freed <[email protected]> Cr-Commit-Position: refs/heads/main@{#1422104} -- wpt-commits: 247db68758a8b03474678dcc6ecfc1fc955227f6 wpt-pr: 50810

beaufortfrancois mentioned this pull request Feb 5, 2025

start(audioTrack) does not check microphone permissions policy #135

Merged

4 tasks

beaufortfrancois force-pushed the refactor branch from aa8dbf1 to cbcd9f8 Compare February 5, 2025 14:42

Introduce start session algorithm

0ad5c8f

beaufortfrancois force-pushed the refactor branch from cbcd9f8 to 0ad5c8f Compare February 5, 2025 15:57

beaufortfrancois added 2 commits February 6, 2025 10:17

Nits

b7f87b7

Nit WebAudio#2

2817121

beaufortfrancois commented Feb 6, 2025

View reviewed changes

evanbliu reviewed Feb 6, 2025

View reviewed changes

index.bs Show resolved Hide resolved

Use InvalidStateError instead of UnknownError

b7b978e

beaufortfrancois requested review from youennf and evanbliu February 12, 2025 07:57

evanbliu reviewed Feb 12, 2025

View reviewed changes

index.bs Outdated Show resolved Hide resolved

evanbliu approved these changes Feb 12, 2025

View reviewed changes

beaufortfrancois added 2 commits February 13, 2025 09:01

Fix typo

ae466db

Merge branch 'main' of github.com:WebAudio/web-speech-api into refactor

42bfc5a

Merge branch 'main' into refactor

9fe3ac7

hoch merged commit 6356249 into WebAudio:main Feb 18, 2025
1 check passed

chromium-wpt-export-bot mentioned this pull request Feb 18, 2025

Update Web Speech API to check for detached context web-platform-tests/wpt#50786

Merged

chromium-wpt-export-bot mentioned this pull request Feb 19, 2025

WebSpeech: start() throws if detached iframe web-platform-tests/wpt#50810

Merged

github-actions bot added a commit to evanbliu/speech-api that referenced this pull request Feb 20, 2025

Introduce start session algorithm (WebAudio#138)

8216e12

SHA: 6356249 Reason: push, by evanbliu Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions bot added a commit that referenced this pull request Feb 20, 2025

Introduce start session algorithm (#138)

62df0cc

SHA: 6356249 Reason: push, by hoch Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce start session algorithm #138

Introduce start session algorithm #138

beaufortfrancois commented Feb 5, 2025 •

edited by pr-preview bot

Loading

beaufortfrancois commented Feb 6, 2025

beaufortfrancois Feb 6, 2025

youennf Feb 6, 2025

beaufortfrancois Feb 7, 2025

beaufortfrancois Feb 20, 2025

beaufortfrancois commented Feb 10, 2025

evanbliu commented Feb 10, 2025

beaufortfrancois commented Feb 13, 2025

beaufortfrancois commented Feb 17, 2025

Introduce start session algorithm #138

Introduce start session algorithm #138

Conversation

beaufortfrancois commented Feb 5, 2025 • edited by pr-preview bot Loading

beaufortfrancois commented Feb 6, 2025

beaufortfrancois Feb 6, 2025

Choose a reason for hiding this comment

youennf Feb 6, 2025

Choose a reason for hiding this comment

beaufortfrancois Feb 7, 2025

Choose a reason for hiding this comment

beaufortfrancois Feb 20, 2025

Choose a reason for hiding this comment

beaufortfrancois commented Feb 10, 2025

evanbliu commented Feb 10, 2025

beaufortfrancois commented Feb 13, 2025

beaufortfrancois commented Feb 17, 2025

beaufortfrancois commented Feb 5, 2025 •

edited by pr-preview bot

Loading