The first "audiooutput" `MediaDeviceInfo` returned from `enumerateDevices()` is not the default device when the default device is not exposed

[karlt]

#113 (comment) proposed output device ordering in enumerateDevices() results to identify the default audio output device. It also said 'This also probably means it should always be exposed since '' allows to be used by setSinkId.'

While the first position for the default device was specified in w3c/mediacapture-main#757, there was no change to always expose the default device and no discussion around this.
The result is that a client app does not know whether or not the first audio output device returned from enumerateDevices() is the default device.

[karlt]

This is causing trouble on some web conferencing sites that appear to be assuming that the first device is the default device. When the default device is not exposed, they don't offer a way to switch to the default device nor (sometimes) to switch to the first exposed device. The biggest problem is that sites present UI to (sometimes) allow switching audio output away from the default device, but there is then no way to switch back to the default device. This would be a regression on adding support for setSinkId().

[karlt]

Proposal: Always include, in enumerateDevices() results, a (single) MediaDeviceInfo object for the default device if there are any audio output devices exposed. If the physical audio output device that is currently the default device would not be exposed if it were not the default, then remove fingerprinting surface from the attributes of this MediaDeviceInfo.

[karlt]

For a MediaDeviceInfo representing a default device that would not otherwise be exposed, perhaps the simplest solution in terms of consistency with the current spec might be to initialize only the kind attribute, as for dummy "videoinput" and "audioinput" devices when their respective "information can be exposed" flags are false.
Some variations on this may also be considered:

A generic minimal-fingerprint label meaning something like "System default audio output device for this user-agent" may be useful to provide the user with some indication of what the option represents should a client app present this device in UI constructed from unfiltered set of MediaDeviceInfos. This would preferably be localized, so the precise wording would not be specified.

Possible variations might have deviceId that is empty or that of the physical device (defaultDeviceId). Exposing a deviceId not otherwise available might be seen as an advantage or disadvantage.

• A client app already may choose to add it's own UI option corresponding to setSinkId("") behavior.
• defaultDeviceId would be more similar to the other devices exposed through enumerateDevices() in that setSinkId(deviceId) would still tie output to a single physical device. When an empty deviceId is used with setSinkId(), the audio is redirected when the user or system changes the default audio output device. With defaultDeviceId, audio would no longer be heard if the default device changes and defaultDeviceId is not otherwise exposed. I expect an empty deviceId would match user expectations if the label is "System default audio output device for this user-agent", but perhaps for defaultDeviceId a label might be along the lines of "The physical audio output device that was the default device at the time this MediaDeviceInfo object was created, but this label does not describe that physical device."
• defaultDeviceId would be a small additional fingerprint, even if that is unique to the origin, because changes in the default device would be visible across origins, even for unexposed physical devices.
• Exposing defaultDeviceId would only be useful if setSinkId("") vs setSinkId(defaultDeviceId) #94 were revisited because

If sinkId is not the empty string, and the application would not be permitted to play audio through the device identified by sinkId if it weren't the current user agent default device, reject p with a new DOMException whose name is NotAllowedError and abort these substeps.

Either an empty deviceId or an empty label would be sufficient to indicate that the label does not describe a physical device.

Perhaps similar options exist for groupId.

I'm favoring empty deviceId and groupId, and allowing a non-empty label.

[dontcallmedom]

A generic minimal-fingerprint label meaning something like "System default audio output device for this user-agent" may be useful to provide the user with some indication of what the option represents should a client app present this device in UI constructed from unfiltered set of MediaDeviceInfos. This would preferably be localized, so the precise wording would not be specified.

Should this be adopted, it would need support for localization in the label attribute, which would be new input to w3c/mediacapture-main#665

[jan-ivar]

I'm favoring empty deviceId and groupId, and allowing a non-empty label.

This SGTM.

[guidou]

Chromium's solution is to use "default" as device ID.
Given that no other browser currently exposes the system default audio output device and that Chromium has been exposing it for years using this device ID, this would be a much more Web-compatible solution. There is little difference between using "default", "" or any other constant string.

Wrt to the label, I think we should let browsers decide what label to use. I don't think an empty label would be useful since it would require applications to give special treatment to such a label.

Wrt groupId I see no problem with allowing groupId to be the group ID of the physical device that the system default is currently pointing to, as long as it gets updated when the system default device starts to point to a different device. I also have no problem with also allowing it to be empty or uniquely nonempty.

[jan-ivar]

The proposal in #133 (comment) is a neutered entry instead of the physical "device that would not otherwise be exposed", and only when it is the OS default.

E.g. if the user selectAudioOutputs their AirPods AND they're the current OS default (which is common after they put them on in macOS), then no other entry is added to enumerateDevices:

 {label: "AirPods", deviceId: "234", ...}

If they then go to macOS's System Settings→Sounds and change the OS default to "MacBook Pro Speakers", you'll see:

 {label: "System default audio output device", deviceId: "", ...}
 {label: "AirPods", deviceId: "234", ...}

If they then selectAudioOutput their MacBook Pro Speakers, you'll see:

 {label: "MacBook Pro Speakers", deviceId: "123", ...}
 {label: "AirPods", deviceId: "234", ...}

@karlt is that right? This solves finding the default device in the spec:

const defSpkr = (await mediaDevices.enumerateDevices())
                                   .find(d => d.kind == “audio-output”);

Chromium's solution is to use "default" as device ID.

Unfortunately, this poses a competing model for finding the default device:

const defSpkr = (await mediaDevices.enumerateDevices())
                                   .find(d => d.kind == “audio-output” && d.deviceId == "default");

There is little difference between using "default", "" or any other constant string.

setSinkId("") is special and unsets the sinkId. What would adding a second special value accomplish?

[karlt]

E.g. if the user selectAudioOutputs their AirPods AND they're the current OS default (which is common after they put them on in macOS), then no other entry is added to enumerateDevices:

 {label: "AirPods", deviceId: "234", ...}

If they then go to macOS's System Settings→Sounds and change the OS default to "MacBook Pro Speakers", you'll see:

 {label: "System default audio output device", deviceId: "", ...}
 {label: "AirPods", deviceId: "234", ...}

If they then selectAudioOutput their MacBook Pro Speakers, you'll see:

 {label: "MacBook Pro Speakers", deviceId: "123", ...}
 {label: "AirPods", deviceId: "234", ...}

@karlt is that right? This solves finding the default device in the spec:

const defSpkr = await mediaDevices.enumerateDevices()
                                  .find(d => d.kind == “audio-output”);

That's correct, and this finds a deviceId that can be passed to setSinkId() or compared with a sinkId. To distinguish between a virtual and physical device and so whether the physical default device is exposed, the deviceId would be compared with "".

Chromium's solution is to use "default" as device ID.

Unfortunately, this poses a competing model for finding the default device:

const defSpkr = await mediaDevices.enumerateDevices().
                                  .find(d => d.kind == “audio-output” && d.deviceId == "default");

There is little difference between using "default", "" or any other constant string.

setSinkId("") is special and unsets the sinkId.

setSinkId("") and sinkId are the main reason for re-using "", to which the spec already gives meaning as the user-agent default device. This is also consistent with the empty deviceId on the single "audioinput" (or "videoinput") MediaDeviceInfo provided to indicate that at least one microphone exists before "audioinput" devices are exposed.

Chrome is using deviceId: "" and deviceId: "default" in different situations.
When no "audiooutput" devices are exposed, Chrome provides

{label: "", deviceId: "", groupId: ""}

The spec (and the proposal here, which is Gecko's behavior now) would expect no "audiooutput" devices.

When the default "audiooutput" device is exposed via getUserMedia(), Chrome provides (on Linux)

{label: "Default", deviceId: "default", groupId: "default"}
{label: "Built-in Audio Analog Stereo", deviceId: "38cf59402979c7c92a7dfe139b46a07932288d562b1dce9bca711b1e7d2097bc", groupId: "fc5673e12055618bb6840857ef5ffaf81ae28949a50d518734c167530922a3bf"}

https://jan-ivar.github.io/dummy/enumerate.html is useful for testing browser behavior.

In Chrome currently find(d => d.kind == “audio-output”) returns the MediaDeviceInfo with deviceId: "default", so find(d => d.kind == “audio-output” && d.deviceId == "default") would return the same.

AFAIK there is currently no way to create a situation where Chrome would expose a non-default "audiooutput" device without exposing the user-agent default "audiooutput" device. If there were, then Chrome's virtual deviceId: "default" device would be a solution to the problem of sites that assume that the first "audiooutput" device is the default device, because in Chrome setSinkId("default") switches output to the user-agent default if the default is exposed.

Chrome's deviceId: "default" on its virtual device is not a solution to the issue of providing a client app with a means to determine which exposed physical device, if any, is the user-agent default device.

[guidou]

That's correct, and this finds a deviceId that can be passed to setSinkId() or compared with a sinkId. To distinguish between a virtual and physical device and so whether the physical default device is exposed, the deviceId would be compared with "".

If "default" is the ID for the system default device as exposed in enumerateDevices() then you just need to compare with "default" instead of "".

Chromium's solution is to use "default" as device ID.

Unfortunately, this poses a competing model for finding the default device:

const defSpkr = await mediaDevices.enumerateDevices().
                                  .find(d => d.kind == “audio-output” && d.deviceId == "default");

What is exactly the problem with this? And the model is competing with what?
There is nothing in the spec as currently written that specifies what the deviceId field should be for a system-default device.

There is little difference between using "default", "" or any other constant string.

setSinkId("") is special and unsets the sinkId.

setSinkId("") and sinkId are the main reason for re-using "", to which the spec already gives meaning as the user-agent default device. This is also consistent with the empty deviceId on the single "audioinput" (or "videoinput") MediaDeviceInfo provided to indicate that at least one microphone exists before "audioinput" devices are exposed.

The spec uses the empty string for several different things. And the deviceId field for the system-default device in the output of enumerateDevices is not one of those things.

Chrome is using deviceId: "" and deviceId: "default" in different situations. When no "audiooutput" devices are exposed, Chrome provides

{label: "", deviceId: "", groupId: ""}

In this case "" is not the ID of the default device. It is an entry with that signals that the system has output devices, but the information about them cannot be exposed (because gUM hasn´t been called or because there are no permissions).

The spec (and the proposal here, which is Gecko's behavior now) would expect no "audiooutput" devices.

When the default "audiooutput" device is exposed via getUserMedia(), Chrome provides (on Linux)

{label: "Default", deviceId: "default", groupId: "default"}
{label: "Built-in Audio Analog Stereo", deviceId: "38cf59402979c7c92a7dfe139b46a07932288d562b1dce9bca711b1e7d2097bc", groupId: "fc5673e12055618bb6840857ef5ffaf81ae28949a50d518734c167530922a3bf"}

https://jan-ivar.github.io/dummy/enumerate.html is useful for testing browser behavior.

In Chrome currently find(d => d.kind == “audio-output”) returns the MediaDeviceInfo with deviceId: "default", so find(d => d.kind == “audio-output” && d.deviceId == "default") would return the same.

AFAIK there is currently no way to create a situation where Chrome would expose a non-default "audiooutput" device without exposing the user-agent default "audiooutput" device.

This is correct.

If there were, then Chrome's virtual deviceId: "default" device would be a solution to the problem of sites that assume that the first "audiooutput" device is the default device, because in Chrome setSinkId("default") switches output to the user-agent default if the default is exposed.

I haven´t thought about the details of what Chrome would do if it had a per-device permission model, but if the system default device is exposed by enumerateDevices, its ID (whatever it is) should be accepted by setSinkId and should send the output to the system default device.

Chrome's deviceId: "default" on its virtual device is not a solution to the issue of providing a client app with a means to determine which exposed physical device, if any, is the user-agent default device.

Why would using the empty string (which has different meanings in different contexts) as ID for the default device in enumerateDevices solve this problem and not "default"?

The way Chrome solves that problem right now is by setting the groupId to the same groupId of the physical device currently considered the default.

[jan-ivar]

const defSpkr = (await mediaDevices.enumerateDevices())
                                   .find(d => d.kind == “audio-output” && d.deviceId == "default");

What is exactly the problem with this?

It will only work in Chrome, not the spec. E.g. in the first "AirPods" example above.

And the model is competing with what?

The spec, which says the way to find the default is to assume it's the first one listed:

const defSpkr = (await mediaDevices.enumerateDevices()).find(d => d.kind == “audio-output”);

These are competing ways to learn the same thing, and we want web developers to adopt the interoperable way.

Chrome's virtual (large "D") Default device is listed first, so why do we need another way?

I haven´t thought about the details of what Chrome would do if it had a per-device permission model, but if the system default device is exposed by enumerateDevices, its ID (whatever it is) should be accepted by setSinkId and should send the output to the system default device.

There's no all-speakers permission model in https://w3c.github.io/mediacapture-output

Chrome's deviceId: "default" on its virtual device is not a solution to the issue of providing a client app with a means to determine which exposed physical device, if any, is the user-agent default device.
...
The way Chrome solves that problem right now is by setting the groupId to the same groupId of the physical device currently considered the default.

That's clever, but means web developers need to write additional code to work around Chrome's virtual Default device:

const speakers = (await mediaDevices.enumerateDevices()).filter(({kind}) => kind == "audio-output");
const defSpkr = speakers.reverse().find(({groupId}) => groupId == speakers[0].groupId);

[guidou]

const defSpkr = (await mediaDevices.enumerateDevices())
                                   .find(d => d.kind == “audio-output” && d.deviceId == "default");

What is exactly the problem with this?

It will only work in Chrome, not the spec. E.g. in the first "AirPods" example above.

What I'm saying is, what would be the problem with this if we say in the spec that the deviceId of a system default device is "default". So far, Chromium is the only browser that exposes a system default device. Safari doesn't expose output devices at all and Firefox exposes output devices, but not a system default device.

In my experience with Firefox 129, it shows the physical device currently marked as system default device as the first element in the output of enumerateDevices, but if the system default devices changes, the enumerateDevices result doesn't change, which is correct if we interpret this device to be a user-agent default (not system default).