Undesirable prompt from selectAudioOutput({deviceId}) if valid device removed

[jan-ivar]

Use case: A website has two buttons: PLAY and SELECT SPEAKERS. If the latter is used, the site persists the resulting deviceId to localStorage. On subsequent visits, the user just hits PLAY.

Problem: if the user removes their chosen device, they get a picker prompt when they click PLAY, which is confusing since they only associate that prompt with the SELECT SPEAKERS button.

Instead, the expected behavior would be to play out over the default audio output in this case.

STRs (requires Firefox Nightly 117):

1. Using e.g. Airpods, open https://jan-ivar.github.io/dummy/speaker_output.html
2. Choose Select Speakers: Other... and pick "Airpods".
3. Put Airpods in their case
4. Hit Play!

Expected: audio over laptop speakers. Actual: the following prompt:

I propose we consider having selectAudioOutput reject in this case instead of prompt, provided the user agent recognizes the deviceId as one it used to satisfy. This should still deter trackers.

Implementing this should be possible by tracking recently removed devices.

[karlt]

If the deviceId is not one that was previously granted, then it has no meaning and is not useful to a tracker, and so behaviour could be the same as for a recently removed device.

If the deviceId corresponds to a previously permitted device, for which permission has since been revoked, then the site does have the corresponding device name. Rejecting also in this case would add ambiguity about the usefulness of a reject for cross-site correlation.

[karlt]

Transient activation is necessary for selectAudioOutput(), which makes time-precise cross-site correlations difficult. Fingerprinting entropy added through a rejection would be no more than that exposed when the device is removed during playback, for example. I'm not sure how concerned to be about this.

[karlt]

A related factor to consider is whether a site with permission for a particular removed device should receive a devicechange event and info from enumerateDevices() when that device is reconnected. i.e. could it be added to [[explicitlyGrantedAudioOutputDevices]] on selectAudioOutput() rejection for a removed permitted device?

[alvestrand]

Discussion at TPAC Sept 2023: Some worry about the "report once" language, more comfort with "MAY reject as long as the device has been known".
General approach supported.

[karlt]

What is the motivation for "as long as the device has been known"?
Is there a good reason to prompt when the deviceId is meaningless, or one which now has no meaning because site data has been cleared?

"MAY reject" might be somewhat consistent with the current "MAY decide, based on its previous decision of whether to persist this id or not for this set of origins," to resolve without prompt, but I don't know that "matches an id previously exposed by selectAudioOutput in an earlier browsing session" need be required for the reject case.

[dontcallmedom-bot]

This issue was discussed in WebRTC TPAC 2023 meeting – 12 September 2023 (Issue 137 Undesirable prompt from selectAudioOutput({deviceId}) if valid device removed)

[youennf]

The ability to resetup audio as the last visit would be nice to do, I am not sure selectAudioOutput is the full solution here.

The test page API is making design mistakes I think, it should not show AirPods if it does not know AirPods are available.
Maybe it is just me and I am too focused on the testing web page, but I do not see how rejecting here would make the user experience all that better. Which exact flow do we want to solve here by rejecting?

Getting back to the test page, AirPods are shown and selectAudioOutput is rejecting. The user expects audio to flow on AirPods.
But what will happen is that either audio will not flow at all or will flow on the default output (say headphones).

I'd like to see a more compelling flow where reject would be more useful.

With current selectAudioOutput, a typical flow would be AirPods UI being shown as greyed/inactive (or maybe AirPods - not selected) on reload and user would need to click on it to make it active via selectAudioOuput. There might be no prompt (AirPods - selected then) or there might be a prompt, in which case UI is updated depending on the result (default in case of reject, headphones if selected...).

[jan-ivar]

The test page API is making design mistakes I think, it should not show AirPods if it does not know AirPods are available.

Absolutely. A superior test page would vet choices before presenting them by filtering out unavailable speakers. But the only way to do so would be through calling selectAudioOutput even earlier, underscoring the undesirability of a prompt.¹

Maybe it is just me and I am too focused on the testing web page, but I do not see how rejecting here would make the user experience all that better. Which exact flow do we want to solve here by rejecting?

Selection is a distraction. The flow is hitting PLAY, step 4 in the OP:

go.onclick = async () => {
  try {
    if (localStorage.speakers) {
      const deviceId = localStorage.speakers;
      await audio.setSinkId((await selectAudioOutput({deviceId})).deviceId);
      updateSelectors();
    }
    // code producing audio

In this case, no prompt is expected because the user just hit PLAY. The ability to reuse the same device as last time seems important.

---

_{1. No speakers show up in enumerateDevices(), because the test page doesn't use microphone. Even if it did, non-mic'ed speakers wouldn't show.}