IOS/USB: Emulate Wii Speak using cubeb #12769

sepalani · 2024-05-09T19:09:28Z

This PR is based on #12567.

I don't have much knowledge regarding the technical details of the implementation. I mainly fixed the Windows support, the heisenbug due to uninitialised members and finalised the config integration.

Ready to be reviewed and tested.

This change is

sepalani · 2024-05-12T20:27:32Z

I fixed a few more things. The Wii Speak Channel appears to work properly now (its initial test won't work though, as we don't emulate the echo reduction). I also added the Android support.

To test this PR, you'll need to enable the emulated Wii Speak under Tools > Emulated USB devices > Wii Speak and to check the two checkboxes to enable the emulation and to "connect" it. On Android, it's located under Settings > Config > Wii > Emulated USB Devices and you'll also need to enable the microphone permission for the app.

The Wii Speak Channel can be tested using a regular one (but by redirecting Nintendo domains to nothing like localhost, resulting in a skippable 20100 error) or via a Riiconnect24 patched version (AFAICT, WiiLink24 patcher doesn't seem to offer it currently, Wiimmfi WAD Patcher does). A Wii friend is required in order to be able to record Wii messages like this: https://www.youtube.com/watch?v=4kpNID36pW0

This PR can be tested on Monster Hunter 3 as well. You will need to patch the game to go online using alternative servers. You will need a friend in-game also using a Wii Speak (Dolphin or a real Wii) to be able to speak in a city.

I haven't checked this PR with other games yet, such as Animal Crossing. I'll when I'll get the time to do so. Otherwise, this PR is still ready to be reviewed & tested.

Florin9doi · 2024-05-15T10:37:20Z

Source/Core/Core/IOS/USB/Emulated/WiiSpeak.h

+  bool m_device_attached = false;
+  bool init = false;
+  std::unique_ptr<Microphone> m_microphone{};
+  const DeviceDescriptor m_device_descriptor{0x12,  0x1,    0x200,  0,   0,   0,   0x10,


Suggested change

const DeviceDescriptor m_device_descriptor{0x12, 0x1, 0x200, 0, 0, 0, 0x10,

const DeviceDescriptor m_device_descriptor{

.bLength = 0x12,

.bDescriptorType = 0x01,

.bcdUSB = 0x0200,

.bDeviceClass = 0x00,

...

AFAICT, this syntax isn't used in the current codebase for similar device classes. I wouldn't mind changing it in a follow up PR.

JosJuice

How does the permission prompt work on Android? Does cubeb automagically make the permission prompt show up?

Source/Android/app/src/main/AndroidManifest.xml

JosJuice · 2024-05-18T17:49:08Z

Source/Android/app/src/main/res/values/strings.xml

@@ -923,4 +923,6 @@ It can efficiently compress both junk data and encrypted Wii data.
    <string name="incompatible_figure_selected">Incompatible Figure Selected</string>
    <string name="select_compatible_figure">Please select a compatible figure file</string>

+    <string name="emulate_wii_speak">Wii Speak</string>
+    <string name="disconnect_wii_speak">Mute Wii Speak</string>


"Mute" doesn't match what the setting is called in the code or in DolphinQt. Is this something that was overlooked, or is it intentional?

It is intended since it is presented differently on PC and Android. On PC it has its own dedicated window (similar to the emulated Infinity base):

Whereas on Android, it's only a toggle settings below the one enabling the Wii Speak. So to make it clearer for the end-user I decided to rename it (as the effect and purpose of disconnecting it is not to record audio). I feel that having a "Wii Speak" toggle, plus a "Disconnect Wii Speak" toggle right under it sounds strange from a UI point of view.

I'm open to better ways of presenting this option to the user.

What does the setting do from the perspective of the emulated software? Does it disconnect the Wii Speak USB device, or does it just make the audio be silence? If it's the latter, I think both the Android UI and the PC UI are okay, but the internal name for the setting would make more sense as "mute" than "connect". "Connect" makes sense in the PC UI only because it's in the context of whether you want to connect a host microphone.

sepalani · 2024-06-04T19:35:04Z

@JosJuice

What does the setting do from the perspective of the emulated software? Does it disconnect the Wii Speak USB device, or does it just make the audio be silence? If it's the latter, I think both the Android UI and the PC UI are okay, but the internal name for the setting would make more sense as "mute" than "connect". "Connect" makes sense in the PC UI only because it's in the context of whether you want to connect a host microphone.

Ideally, it should have been used to disconnect the device. However, I couldn't find a way to do so and it had a side effect so I chose not to process the microphone data instead.

dolphin/Source/Core/Core/IOS/USB/Emulated/WiiSpeak.cpp

Lines 97 to 101 in 35436ac

    
           // Without a proper way to reconnect the emulated Wii Speak, 
        
           // this error after being raised prevents some games to use the microphone later. 
        
           // 
        
           // if (!IsMicrophoneConnected()) 
        
           //  return IPC_ENOENT;

dolphin/Source/Core/Core/IOS/USB/Emulated/Microphone.cpp

Lines 252 to 255 in 35436ac

    
           bool Microphone::HasData() const 
        
           { 
        
             return m_samples_avail > 0 && Config::Get(Config::MAIN_WII_SPEAK_CONNECTED); 
        
           }

Yes, I think we should. It's probably best to do it both when turning on the setting and when the C++ code tries to use it. The former because that makes the purpose of the permission the most clear to the user, and the latter in case the permission has been revoked by the user (or automatically by the OS because the app hasn't been used for a while).

Apparently, enabling the permission after cubeb was initialised doesn't seem to work. I added a warning message before requesting the permission to warn the user about this issue. This part (especially the android code) needs to be properly reviewed as it's been a while since I've programmed for Android and I'm unsure that's the proper way to do it.

Source/Android/app/src/main/res/values/strings.xml

JMC47 · 2024-07-03T02:23:00Z

Is anyone else planning to review this? Is this considered finished at this point?

sepalani · 2024-07-08T10:00:21Z

Is anyone else planning to review this? Is this considered finished at this point?

The PR is finished. Though, it might be worth testing to ensure there's no oversight. Especially, on the Android part where I'm not very confident as it's working on my end but I'm unsure that the way I'm getting the activity is correct (or should I check the other callbacks too).

JMC47 · 2024-08-07T20:12:20Z

Does anyone else want to review this? I can give this a test on Android, but I won't be reviewing the code.

MayImilae · 2024-08-16T11:58:35Z

Are you still intending to add a microphone selector? Having a window with a single toggle is quite odd, as in that case you could just put the toggle itself into the file menu.

sepalani · 2024-08-16T17:43:52Z

Are you still intending to add a microphone selector? Having a window with a single toggle is quite odd, as in that case you could just put the toggle itself into the file menu.

Doesn't the selector appear when the emulated device is enabled? I'm just using a layout similar to the one of the Infinity Base, another emulated USB device. I'm open to suggestion regarding this window design. The only constraint that it has to take into account (currently) is that it cannot enable or disable the emulated USB device when the emulation has started (i.e. it has to be chosen before launching the game).

MayImilae · 2024-08-16T17:55:46Z

When I tested this I only saw what was in the top window. What scenario does that occur?

sepalani · 2024-08-16T18:07:10Z

When I tested this I only saw what was in the top window. What scenario does that occur?

Currently, the microphone configuration groupbox is displayed only when the Emulate Wii Speak checkbox is checked. So the groupbox should (dis)appear when the checkbox state is changed. If there is a bug and it doesn't behave properly, then I'll have to investigate to see what's causing this.

JosJuice

Sorry for the delay in following up. I haven't been able to review all the USB and audio code in detail, but I've caught a few things.

Source/Core/Core/Config/MainSettings.cpp

Source/Core/Core/IOS/USB/Emulated/Microphone.cpp

Source/Core/Core/IOS/USB/Emulated/WiiSpeak.cpp

Source/Core/Core/IOS/USB/Emulated/WiiSpeak.h

Source/Core/DolphinQt/EmulatedUSB/WiiSpeakWindow.cpp

...in/java/org/dolphinemu/dolphinemu/features/settings/ui/viewholder/SwitchSettingViewHolder.kt

Source/Android/app/src/main/java/org/dolphinemu/dolphinemu/utils/ActivityTracker.kt

JMC47 · 2024-09-28T18:00:34Z

Last call for reviews on this. It's been waiting very long time and if I test it and the use cases are working on Linux and Windows, I will be merging it unless someone posts here and says they plan on reviewing it again or tells me not to merge it.

@degasus

Credits to @degasus and @shuffle2 (godisgovernment): https://github.com/degasus/dolphin/tree/wiispeak

@noahpistilli

Based on @noahpistilli (Sketch) PR: dolphin-emu#12567 Fixed the Windows support and the heisenbug caused by uninitialized members. Config system integration finalized.

The register should be 12 (i.e. 0x0c) instead of 0xc0

It seems to fix random echoes and reduce noises when nobody is speaking

Based on LibusbDevice::SubmitTransfer code

Add a volume modifier to the UI which relies on gain.

sepalani · 2024-11-16T14:24:36Z

I rebased this PR and added some kind of loudness level calculation. I'm not an audio expert so it might be worth checking that the formulas I'm using are correct.

I also switched this PR back to draft as I'm doing some more testing regarding the loudness level calculation.

sepalani force-pushed the wii-speak branch 7 times, most recently from cf3916c to 46d7831 Compare May 12, 2024 19:40

sepalani force-pushed the wii-speak branch 2 times, most recently from 97daea9 to 3693fe5 Compare May 13, 2024 23:16

Florin9doi reviewed May 15, 2024

View reviewed changes

JosJuice reviewed May 18, 2024

View reviewed changes

sepalani force-pushed the wii-speak branch from 3693fe5 to 72cb760 Compare May 22, 2024 19:36

sepalani force-pushed the wii-speak branch 2 times, most recently from dc72880 to 35436ac Compare June 4, 2024 18:44

JosJuice reviewed Jun 4, 2024

View reviewed changes

Source/Android/app/src/main/res/values/strings.xml Outdated Show resolved Hide resolved

sepalani force-pushed the wii-speak branch 2 times, most recently from 5d1ea67 to 0835422 Compare June 5, 2024 19:14

sepalani force-pushed the wii-speak branch 2 times, most recently from b21121e to 9023aa3 Compare July 8, 2024 09:07

sepalani force-pushed the wii-speak branch from 9023aa3 to d951dfa Compare July 9, 2024 14:13

JosJuice reviewed Aug 17, 2024

View reviewed changes

sepalani force-pushed the wii-speak branch 3 times, most recently from b938bcf to 987896a Compare September 22, 2024 09:54

sepalani force-pushed the wii-speak branch 2 times, most recently from 018c914 to 0352330 Compare September 28, 2024 19:16

noahpistilli and others added 11 commits November 16, 2024 16:02

IOS/USB: Emulate Wii Speak (OpenAL)

7dc1ee2

Credits to @degasus and @shuffle2 (godisgovernment): https://github.com/degasus/dolphin/tree/wiispeak

IOS/USB: Emulate Wii Speak using cubeb

ae6f5a4

Based on @noahpistilli (Sketch) PR: dolphin-emu#12567 Fixed the Windows support and the heisenbug caused by uninitialized members. Config system integration finalized.

IOS/USB: Add more logs to the Wii Speak code

c826e1f

IOS/USB: Fix Wii Speak SAMPLER_MUTE register

44e6625

The register should be 12 (i.e. 0x0c) instead of 0xc0

IOS/USB: Fix Wii Speak buffer memcpy size parameter

101e855

It seems to fix random echoes and reduce noises when nobody is speaking

IOS/USB: Try to fix the Wii Speak Channel record/playback feature

b4a7394

IOS/USB: Only sample Wii Speak data when necessary

37b2df8

Android: Add emulated Wii Speak

2e59fe6

IOS/USB: Report Wii Speak packet size properly

2b85a99

Android: Ask for RECORD_AUDIO permission

0793450

IOS/USB: Add a warning about RECORD_AUDIO permission

c735108

sepalani force-pushed the wii-speak branch from 4aac270 to 79f5bfe Compare November 16, 2024 12:23

sepalani added 9 commits November 16, 2024 16:53

CubebUtils: Add COM helper class

38af545

Android: Fix PermissionsHandler activity detection

8ed9ab0

Config: Rename MAIN_WII_SPEAK_CONNECTED to MAIN_WII_SPEAK_MUTED

2ff8708

WiiSpeakWindow: Don't hide QGroupBox

8b9a601

IOS/USB: Implement Wii Speak REQUEST_SET_INTERFACE command

086e2f4

Based on LibusbDevice::SubmitTransfer code

IOS/USB: Skip data when HLE Wii Speak is muted

26dbc6b

IOS/USB: Lock microphone buffer less frequently

7a9eaf2

IOS/USB: Add more Wii Speak log messages

2d96312

IOS/USB: Implement a bare-bones Wii Speak loudness level

0fa236a

Add a volume modifier to the UI which relies on gain.

sepalani force-pushed the wii-speak branch from 79f5bfe to 0fa236a Compare November 16, 2024 12:57

sepalani marked this pull request as draft November 16, 2024 14:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IOS/USB: Emulate Wii Speak using cubeb #12769

IOS/USB: Emulate Wii Speak using cubeb #12769

sepalani commented May 9, 2024 •

edited

Loading

sepalani commented May 12, 2024

Florin9doi May 15, 2024

sepalani May 22, 2024

JosJuice left a comment

JosJuice May 18, 2024

sepalani May 22, 2024

JosJuice May 27, 2024

sepalani commented Jun 4, 2024 •

edited

Loading

JMC47 commented Jul 3, 2024

sepalani commented Jul 8, 2024

JMC47 commented Aug 7, 2024

MayImilae commented Aug 16, 2024

sepalani commented Aug 16, 2024

MayImilae commented Aug 16, 2024

sepalani commented Aug 16, 2024

JosJuice left a comment

JMC47 commented Sep 28, 2024

sepalani commented Nov 16, 2024

-  const DeviceDescriptor m_device_descriptor{0x12,  0x1,    0x200,  0,   0,   0,   0x10,
+  const DeviceDescriptor m_device_descriptor{
+    .bLength = 0x12,
+    .bDescriptorType = 0x01,
+    .bcdUSB = 0x0200,
+    .bDeviceClass = 0x00,
+   ...

IOS/USB: Emulate Wii Speak using cubeb #12769

Are you sure you want to change the base?

IOS/USB: Emulate Wii Speak using cubeb #12769

Conversation

sepalani commented May 9, 2024 • edited Loading

sepalani commented May 12, 2024

Florin9doi May 15, 2024

Choose a reason for hiding this comment

sepalani May 22, 2024

Choose a reason for hiding this comment

JosJuice left a comment

Choose a reason for hiding this comment

JosJuice May 18, 2024

Choose a reason for hiding this comment

sepalani May 22, 2024

Choose a reason for hiding this comment

JosJuice May 27, 2024

Choose a reason for hiding this comment

sepalani commented Jun 4, 2024 • edited Loading

JMC47 commented Jul 3, 2024

sepalani commented Jul 8, 2024

JMC47 commented Aug 7, 2024

MayImilae commented Aug 16, 2024

sepalani commented Aug 16, 2024

MayImilae commented Aug 16, 2024

sepalani commented Aug 16, 2024

JosJuice left a comment

Choose a reason for hiding this comment

JMC47 commented Sep 28, 2024

sepalani commented Nov 16, 2024

sepalani commented May 9, 2024 •

edited

Loading

sepalani commented Jun 4, 2024 •

edited

Loading