Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate OpenAL #8924

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Deprecate OpenAL #8924

wants to merge 1 commit into from

Conversation

Filoppi
Copy link
Contributor

@Filoppi Filoppi commented Jul 1, 2020

This will also revert the setting of anyone that currently has OpenAL selected to the default, which is Cubeb.
OpenAL is only on Windows now, and it's the worst backend there by far, it crackles at latencies that Cubeb supports fine, it has no additional features to Cubeb.

There seems to be no reason to allow users to even select, and in the dev irc channel, no one was aware of any case where OpenAL might work better than Cubeb.
The plan is to see if users complain about this, and if not, just delete it from Dolphin.

@Miksel12
Copy link
Contributor

Miksel12 commented Jul 2, 2020

@LAGonauta had some objections in #6411 to removing OpenAL. I think those objections are still valid.

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 2, 2020

Citing this:
#6411 (comment)
I made a PR to update cubeb yesterday, and that implement low latency (shared) WASAPI mode on Windows, which should put it quite close to WASAPI Exclusive Mode in terms of perceived latency. I've also fixed most of the problems with WASAPI Exclusive Mode and made it compatible with more devices (PR upcoming), so the WASAPI backend will be stronger.
I doubt OpenAL can ever have latencies lower than WASAPI Exclusive with any sound card, but I'm not an audio device/driver expert. I also doubt WASAPI Exclusive can't be used with some devices where OpenAL could.

Windows has also recently implemented "Windows Sonic For Headphones" which should do a similar thing to what they referred to as "HRTF", and on the Microsoft Store, you can download "Dolby Atmos", and "DTS Sound Unbound", 2 additional options to do the same thing OpenAL does, except they might do it better because they are much more up to date? Cubeb will probably never support HRTF as it can now be done by these apps in "post".

Regarding the DSP HLE not working with DPLII, @degasus told me it is not true and it should work as LLE.

And really, even if we couldn't do HRTF anymore (we can), would that really be a loss? You start from a stereo signal with encoded 5.1 information, which is then manipulated to produce a "low quality" 5.1, and then you would convert it back to a stereo signal encoded with HRTF? Are you really going to get much more positional information from that? I doubt so. If you instead had native 5.1, whether headphones or speakers, you don't need "fake" HRTF.

I would say these small things are not worth keeping it around, as they get more and more outdated/unnecessary, and I specifically marked it as "Deprecated" to see whether any user would complain.

@LAGonauta
Copy link
Contributor

LAGonauta commented Jul 2, 2020

And really, even if we couldn't do HRTF anymore (we can), would that really be a loss?

Yes, it would. DPLII is not so bad on encoding surround information as long as there is not much correlation between the front and back channels

Regarding the DSP HLE not working with DPLII, degasus told me it is not true and it should work as LLE.

Really? That is news for me. HLE can encode DPLII or output 5.1 directly now? Cool.

I made a PR to update cubeb yesterday, and that implement low latency (shared) WASAPI mode on Windows

That is nice, people on newest Windows 10 should achieve lower latencies then :)

Last time I checked Dolby Atmos and Windows Sonic added annoying additional latency, but that is a problem with most APOs. Not sure how it is nowadays...

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 2, 2020

I really don't know what i'm talking about here but, let's list the cases:
-If your sound card/device supported HRTF at software level, then it would also add latency and you might as well use Windows Sonic.
-If your sound card/device supported HRTF at hardware level, then it should also support it without passing through OpenAL, or so I think.
-If your sound card/device did not support HRTF at any level, then OpenAL makes no difference.

@LAGonauta
Copy link
Contributor

I really don't know what i'm talking about here but, let's list the cases:
-If your sound card/device supported HRTF at software level, then it would also add latency and you might as well use Windows Sonic.
-If your sound card/device supported HRTF at hardware level, then it should also support it without passing through OpenAL, or so I think.
-If your sound card/device did not support HRTF at any level, then OpenAL makes no difference.

  1. Correct, but some might prefer native solutions instead of Sonic if they are customizable (custom ear, custom head shape...)
  2. Correct.
  3. OpenAL Soft can add HRTF to any sound sink, so any sound card.

Of course we are talking about projecting 5.1 into two speakers, this is common nowadays.
OpenAL is a must for FOSS projects that want object-based HRTF, with elevation support and the likes 😉 (this is not how Dolphin works, sadly.)

I wonder now... some APOs are disabled in exclusive mode, are Sonic/Atmos one of those? Never checked. Does the shared low-latency mode disable then?

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 2, 2020

I've tested all of them, Windows Sonic, Dolby Atmos and DTS. None of them changes the device into being 5.1, so dolphin still fails to start a 5.1 stream with these enabled on stereo headphones. In fact, I just really don't understand how they even work. They take a stereo signal and output a stereo signal, but it sounds slightly different. I don't really know... From what I read they should force games to output 5.1 and then mix to 2.0 by using the directional information, but that doesn't happen. And most games nowadays don't even need that stuff anymore, sounds are already directional.

But with WASAPI in exclusive mode, no, they are completely bypassed, as anything else, you send data to the device directly, while these fake surround effects are done in a post process after, or before, the windows mixer.
I doubt that the shared low latency mode prevents them from working because it was introduced with Windows 10, just like these OS integrated post processing plugins.

Edit: actually, I might have done the above mentioned test with WASAPI exclusive mode, before I really knew how it worked, which would explain why the device wasn't seen as 5.1.

@LAGonauta
Copy link
Contributor

LAGonauta commented Jul 2, 2020

Maybe they are just adding some reverb if you send stereo sound, and try to virtualize two front speakers? No idea.

They are not fake surround, they are virtualized surround and works really well if you use a model similar to your head/ear shape 😉
Spot on if it is your own model. With OpenAL it is really easy to change the model to one more akin to the user model 👍🏼

Anyway, some cards, such as the X-Fi and those new from Creative have two control panels: the Windows one and the driver one. You can set 7.1/5.1 in Windows CP and headphones in the driver CP, the DSP or driver software is in charge to project those channels into two channels using HRTF or similar techniques.

Maybe with Windows Sonic one would have to use those new 3D audio APIs that Microsoft added to... WASAPI, I think? Then the APO would do its magic and virtualize the sound.

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 2, 2020

I've updated the above comment. I doubt window sonic requires new APIs to be used, because it gives best results with older games. Nowadays games already output sounds that feel like they are coming from specific directions even if it's just stereo, so windows sonic is just a useless post process in these cases, from my limited experience.

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 2, 2020

I have an Asus dtx and it has two control panels, you can even select a different sample rate between windows and its own control panel... I have no idea how that would work and why it is there.

Anyway, my tests were probably wrong and DSP should allow users to use HRTF to their liking without OpenAL. These sound card you mentioned do HRTF in HW so they don't need OpenAL.
OpenAL soft can do that in software but, given there are loads of new, and possibly better, ways of doing the same thing, I would say it's good to drop it.
I don't care that much, but the implementation isn't good looking and its got high latency, even on a card that supports OpenAL like mine. Plus, there are quite a few other improvements to be made to dolphin sound which are more important than this in my opinion. Some of them are upcoming.

@LAGonauta
Copy link
Contributor

LAGonauta commented Jul 2, 2020

I wouldn't say better as they are not customizable ;)
The API I talked about: https://docs.microsoft.com/en-us/windows/win32/api/spatialaudioclient/nn-spatialaudioclient-ispatialaudioclient
If this API is required to make virtualisation work correctly the current WASAPI backend is not enough. More information: https://docs.microsoft.com/en-us/windows/win32/coreaudio/spatial-sound

Are those older games using DirectSound by any chance? I noticed they can force 5.1/7.1 even when Windows control panel is set to stereo (such as HL2)

I wouldn't call them better than OpenAL Soft as their HRTF is not customizable. PulseAudio with VirtualSurround module, sure, but not on Windows.
Did you try using OpenAL Soft's WASAPI backend, along with lower period size and buffer size set in alsoft.ini?

What kind of improvements would the OpenAL backend block? I can help with that.

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 3, 2020

The OpenAL backend is not blocking anything, I wasn't aware of its audiophile friendly settings nor was anyone in the dev channel.
I believe my sound card supports OpenAL for example, because it says in the user manual and because it's got OpenAL.dll in its installer, but the latencies required by it to sound as good as Cubeb are way larger.

It's more of a philosophical question at this point. OpenAL soft, being software, could also be implemented on Windows as an audio post process, like Windows Sonic, without requiring games to use the OpenAL dll, and Dolphin doesn't even directly use any features from it, it just renders sound. From what I've read OpenAL isn't really used by any games anymore, and it's a "deprecated" technology, which will eventually be fully replaced (guaranteed), so I personally think we should adapt to times and not drag along everything from the past.
I haven't tried OpenAL soft no.

@LAGonauta
Copy link
Contributor

Maybe I should lurk in the dev channel more, then ;)

Not using OpenAL because it is old would be the same thing as saying that one should not use OpenGL because it is old. OpenAL, just like OpenGL can have various drivers, various ICDs, installed. And OpenAL Soft is alive and well, and evolving: https://github.com/kcat/openal-soft

I am not sure, but I believe OpenAL is one of the few open technologies Apple still keeps around, too.
I admit OpenAL Soft was probably not made at first for real-time streaming with tight latencies, but for games with one-shot sounds. I bet this could be made better with some tinkering.

What could replace OpenAL? There is no FOSS with similar capabilities AFAIK. Cubeb is just an output library, right?

Some games that uses OpenAL:

  • Minecraft
  • Fez
  • Almost all, if not all, Valve games on Linux
  • Almost every Doom source port
  • Almost all Quake source port
  • Path of Exile
  • Serious Sam
  • Trackmania
  • SuperTuxKart
  • TORCS

Here is a list:
https://docs.google.com/document/d/1j7hqIk0LzMnagk4GHHHk13Bb2L-zt9KRqRRVsnGpLCE/view

You might have read that from those people that make money selling sound middleware, someone that likes/uses/prefers proprietary technologies such as Miles/FMOD/DolbyAnything. OpenAL has no marketing budget, so it will be harder to come by nowadays in modern games.

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 3, 2020

I'm not here to discuss this stuff, but if these sources are reliable, Apple has removed OpenAL from iOS and deprecated it for Mac OS:
https://developer.apple.com/forums/thread/7081
https://linustechtips.com/main/topic/1069629-apple-has-deprecated-openal-in-macos-1015-catalina/#:~:text=Apple%20has%20announced%20today%20in,release%20notes%20of%20macOS%20Mojave.

All the games you mention are either old, or based on old technology, and they actually use OpenAL features directly probably, Dolphin doesn't.
I believe Unreal Engine natively offers some of OpenAL spatialization features, but most non indie games use WWise or FMOD.
Also, yes, I wouldn't use OpenGL anymore, yes, that is my point of view.

To recap, it seems like the only advantage of keeping OpenAL around is so that users could customize their HRTF head shape setting, while they could not with, let's say, Windows Sonic. Given that we don't directly use any features from OpenAL, to me it is wrong that an application needs to use a specific library for a feature that isn't technically bound to that library. Meaning that HRTF with a customisation head shape could also be implemented by Windows Sonic or similar post processes, without awareness (nor requirements) from the application, which makes much more sense (because again, we don't directly use any OpenAL features).

@LAGonauta
Copy link
Contributor

Yes, Dolphin uses only the streaming part of the OpenAL API, however I disagree that old is automatically bad. Why one should spend unnecessary development time with Vulkan if OpenGL meets the requirements and is still supported?
Those games uses WWise and FMOD because there is more marketing and tooling for audio designers for it, not because they are inherently better.

Do you know of a FOSS software that can do custom head shape HRTF on Windows and has the same or lower latency than the OpenAL backend? If someone could do it does not mean that it exists, and that is bad.

@Filoppi
Copy link
Contributor Author

Filoppi commented Jul 3, 2020

To me, supporting OpenAL just because a software implementation of the original HW library allows to customise your head shape with HRTF is just insane.
Other sound processing plugins for Windows allow HRTF in an application agnostic way, just without a custom head shape setting for now, I'd say that's enough.

@RinMaru
Copy link

RinMaru commented Aug 2, 2020

Regarding the DSP HLE not working with DPLII, degasus told me it is not true and it should work as LLE.

Really? That is news for me. HLE can encode DPLII or output 5.1 directly now? Cool.

Not that I see DPLII is disabled for HLE

@Filoppi
Copy link
Contributor Author

Filoppi commented Aug 2, 2020

That turned out to be false, the messages are old, HLE still does not support DPLII.

@RinMaru
Copy link

RinMaru commented Aug 3, 2020

That turned out to be false, the messages are old, HLE still does not support DPLII.

yea... LLE Recompiler gives me some massive lag unless i use dual core so i had to switch back to HLE

@Filoppi
Copy link
Contributor Author

Filoppi commented Aug 3, 2020

All right. This page is not about that though, there is an issue open for HLE/DPLII but I doubt it's high priority.

@mirh
Copy link

mirh commented Jan 19, 2023

To me, supporting OpenAL just because a software implementation of the original HW library allows to customise your head shape with HRTF is just insane.

Putting aside that it's *the* implementation that everybody uses, and putting aside that it's open and I can hardly can think to other FOSS end-to-end hrtf solutions...
The very ironic thing is that dolphin with the OAL backend is already also reportedly compatible with Windows' spatial sound.

@Filoppi
Copy link
Contributor Author

Filoppi commented Jan 19, 2023

To me, supporting OpenAL just because a software implementation of the original HW library allows to customise your head shape with HRTF is just insane.

Putting aside that it's the implementation that everybody uses, and putting aside that it's open and I can hardly can think to other FOSS end-to-end hrtf solutions... The very ironic thing is that dolphin with the OAL backend is already also reportedly compatible with Windows' spatial sound.

Fair enough. This was just a proposal to add '''Deprecated''' to the backend name, because it kinda of it, it hasn't received much love and has major drawbacks compared to cubeb, so the point was also to prevent people from accidentally wasting time trying it.
Anyway, Windows Sonic and Dolby Atmos (any implementation of the Windows Spacial Sound) should also be supported on Cubeb if DPLII is enabled. If I remember correctly I had tested this with headphones.

@Pokechu22
Copy link
Contributor

I remember it being awkward to get OpenAL to work on modern dolphin (it always looks for openal32.dll, even on 64-bit and presumably ARM machines, and we don't ship it). This could probably be fixed, but it does seem like extra work for something that's not very useful.

@mirh
Copy link

mirh commented Jan 19, 2023

OpenAL32.dll is the name the 64-bit library has even in the official installer.
Granted, that never had an ARM version, and you'd have to compile openal-soft for it to satisfy the dependency.

This will also revert the setting of anyone that currently has OpenAL selected to the default, which is Cubeb.
OpenAL is only on Windows now, and it's the worst backend there by far, it crackles at latencies that Cubeb supports fine, it has no additional features to Cubeb.

There seems to be no reason to allow users to even select, and in the dev irc channel, no one was aware of any case where OpenAL might work better than Cubeb.
The plan is to see if users complain about this, and if not, just delete it from Dolphin.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

7 participants