[Request for comments] Possible launch UI changes and future expantion #123
Description
Hey @wunnr , I am still going to add my previous pr's dependency changes in a few days when I am back home, but for now, I have a computer that I can only easily test some small things.
This isnt something I am saying we fully should do this way, just one of my ideas so we can add more stuff into the future, some of that is listed here, but some of it is just going to be easier later. I am mainly sending this as a request for comments on the design of a proposed launch screen, as well as some of the new ideas I have included.
Main thing is to separate into screen selection and device selection, allowing settings for each screen, and thus allowing multiple instances of river, kde, or just displaying on physical display with the sdl stuff in gamescope.
New ideas:
Remote play (I already have a poc to display stuff, but sending key events is not done yet) - Using pipewire, we capture the screen as output from gamescope, then forward it over webrtc to a remote client in a web browser, so it is accessible to anyone who wants to use it. This would allow us to connect and stream games to a remote host, where they could have keyboard, mice, or controllers connected to be able to play in a different room or something for games like lethal company that benefit from not seeing everyone's screens.
"Hidden" screen: Just launches gamescope with --backend headless so we can utilize streaming to a remote client without it being shown, while still allowing the local device to optionally display remote feeds on a specified screen. May become a option on the remote play session, but I feel it make more sense for users (despite truly being a instance option).
Audio forwarding:
We can forward audio using pulse audio to a output (or input) automatically, useful for separating games into different speakers or earbuds, again for games like lethal company.
I am only looking for feedback on the ideas because they may or may not be used, I just think changing the layout and adding thees can be helpful and I would want them, if I have time to make them.
If anything isnt to your liking, please let me know, and if I forgot to mention anything, or you are confused, or want something difrent, just message me.
This is the 2nd time I am writing this after my computer crashed, so sorry if anything is missing. :P
Here is the big green text blob that I wrote:
The main things I hope to add with this are a way to allow users to have multiple screens, as well as allow for remote play or other forms in the future. Remote play is an idea based on "moonlight" streaming, that would esencialy use gamescope's pipewire api to copy the output of the display to a small client who would allow anyone to join on a phone, tablet, or other computer. Here they can use controllers or keyboard input / mouse input to control the game. This is implementedvia webrtc, and I would have to make it my self. Controller input is handled by the gamepad api, and on the host side with a uinput device (requires root once to setup, but is already setup on any device with steam installed, including the steamdeck.) Embeded mode is my idea of using the above pipewire steaming and playing back inside our own created partydeck window. This will be slightly slower than gamescope, only a few ms if we do it properly, but would allow for completely custom layout algorithms, like river without the loss of window control we currently have. Adds extra code base to maintain though.
Remote is shown with 2 different options here to allow either playback on any screen, including a "hidden screen" to avoid cheating in certain games while still allowing a remote session to be seen on the main screen, for games in a group setting. Using it as a "remote screen" instead, will make that screen only able to contain one window, and kind is more complex to implement, while being more restrictive, I dont suggest it.
Similarly to the remote play, I am also thinking of writing an external tool (so it can be used by other programs if anyone wants to expand on this easier) to forward audio from pipewire inside of this userns to a certain sink, so we can have certain games audios forced to one set of headphones, or similarly, forwarding audio to a remote sink. This would be accomplished via looping over all open sources in pipewire, getting if the program with that open is inside the namespace, and if so, changing its output to a specified one. This can be implemented into partydeck similarly to how inputs are setup, allowing routing of audio to a different output, so each game can have a display, keyboard, mouse, and headset, fully isolated gaming