DolphinQt: Add support for a Steam Runtime build #10462

OatmealDome · 2022-02-18T19:21:02Z

This PR adds an easy flag to disable/enable things necessary for a Steam build, and makes changes to DolphinQt to create a build that will run in the Steam Runtime.

I built and tested this PR with Steam Runtime version 1, scout, using the revision currently being used in general availability with the Steam client (which is what Valve recommends for games).

Building

Building Dolphin for the Steam Runtime is just like building for a regular Linux system. However, it must must be done within the Steam Runtime environment to prevent system libraries leaking into the application. A chroot environment or Docker container can be used, as outlined here.

Some extra CMake flags are required to produce a Steam Runtime build, as some libraries included in the runtime are too old. The recommended cmake configuration command is:

cmake .. -G Ninja -DCMAKE_C_COMPILER="gcc-9" -DCMAKE_CXX_COMPILER="g++-9" -DENABLE_EVDEV=OFF -DENABLE_LLVM=OFF -DUSE_SHARED_LIBPNG=OFF -DSTEAM=ON -DENABLE_SDL=ON -DCMAKE_PREFIX_PATH=/path/to/qt

Running

Running a build can occur on any Linux machine. However, it is not possible to run Dolphin within the chroot environment or the Docker container used for building, due to libusb initialization on startup.

First, make sure the scout runtime is installed. This can be done by copying and pasting the following into the address bar

Then, run the build:

./steam-runtime/run.sh ./dolphin-emu

Acknowledgements

Thanks to delroth, Techjar, and Warepire for their help getting DolphinQt working.

Source/Core/DolphinQt/CMakeLists.txt

leoetlino · 2022-03-02T12:05:17Z

This is going to massively bloat Externals/ -- currently the Qt submodule already wastes 0.5GB of space on macOS and Linux.

Would it be possible to pull in the Qt libs at build time using CMake rather than a submodule (similar to how MoltenVK is downloaded only when necessary)?

OatmealDome · 2022-03-03T21:07:23Z

@leoetlino I think it should be possible. A separate git repository would probably need to be created for this, though (I can set one up on my account for now)

OatmealDome · 2022-03-15T22:33:24Z

@leoetlino I had a look at turning the submodule into an ExternalProject, but it seems that the git clone step is executed only on build time, so finding Qt fails when CMake runs. Not sure there's a workaround for this, but maybe I missed something.

leoetlino · 2022-03-16T15:05:25Z

Looks like there are alternatives to ExternalProject if you just want to clone a repo at configure time (rather than at build time): https://stackoverflow.com/a/51311151/1636285

Forgot that CMake would need to find Qt at configure time and so downloading Qt at build time would be too late, my bad.

AdmiralCurtiss · 2022-03-16T15:44:33Z

As far as I know, the modern CMake way to do this is FetchContent.

OatmealDome · 2022-03-16T18:32:44Z

@leoetlino @AdmiralCurtiss I had no idea this existed - thanks! I've updated the PR to use FetchContent.

shuffle2 · 2022-03-16T19:46:56Z

What is the story with SteamOS and packaging Qt (is it based on debian/ubuntu, idk)? If you are building Qt from source, maybe just start with a Qt6 ?
edit: oh, from the PR comment, I take it SteamOS is based on Ubuntu 12.04? lol...

Source/Core/DolphinQt/CMakeLists.txt

OatmealDome · 2022-03-16T19:59:02Z

@shuffle2 Not SteamOS, but the Steam Runtime. The Steam Runtime provides a stable environment with common libraries and tools which can be built against. When a native Linux application is downloaded from Steam, the client downloads the Steam Runtime and runs the application in it. It's not distro-specific.

More info here: https://partner.steamgames.com/doc/store/application/platforms/linux and https://github.com/ValveSoftware/steam-runtime#introduction.

The new SteamOS on the Deck is Arch-based, but I think the older ones are Debian or Ubuntu based.

(Also, isn't Dolphin incompatible with Qt6? I have no experience with creating Qt applications)

shuffle2 · 2022-03-16T20:11:15Z

@shuffle2 Not SteamOS, but the Steam Runtime. The Steam Runtime provides a stable environment with common libraries and tools which can be built against. When a native Linux application is downloaded from Steam, the client downloads the Steam Runtime and runs the application in it. It's not distro-specific.

OK, but for this discussion it basically means we cannot rely on Steam OS and/or Runtime to provide a recent version of Qt, so we must build it ourselves, right?

(Also, isn't Dolphin incompatible with Qt6? I have no experience with creating Qt applications)

see #9669 (specifically #9669 (comment) ) - it's not hard to enable qt6 compat, but we are currently blocked on debian packaging of qt6 taking a long time.

I'm just wondering if:

some trouble for Steam port can be saved by directly going to Qt6 there (presumably won't have to worry update it for a long time / reduce git size of submodule)
Qt build for Steam could be reused for other *nix targets. Maybe this can be a workaround for distros being so late to ship Qt6.

Not saying Qt6 stuff needs to be tied to Steam support in any way, just suggesting that it might make things easier in the long run, but idk 🤷‍♂️

OatmealDome · 2022-03-16T22:41:31Z

@shuffle2

OK, but for this discussion it basically means we cannot rely on Steam OS and/or Runtime to provide a recent version of Qt, so we must build it ourselves, right?

Yes, I had to build it myself. The Runtime doesn't ship with a copy of Qt.

Qt build for Steam could be reused for other *nix targets. Maybe this can be a workaround for distros being so late to ship Qt6.

During testing for this PR, I at one point ran Dolphin outside of the Runtime (with the Runtime-compiled Qt and Manjaro system libraries) and it crashed.

shuffle2 · 2022-03-16T22:56:47Z

During testing for this PR, I at one point ran Dolphin outside of the Runtime (with the Runtime-compiled Qt and Manjaro system libraries) and it crashed.

Sounds funky!

delroth · 2022-03-18T07:35:00Z

Similar to my comment on #10516, let's try to avoid hiding too much magic/implicit behavior behind a STEAM_RUNTIME flag. Stuff like "not linking with libpng" should be separate options that we happen to enable on the SteamRT builder by default.

We probably still want some flag to have cmake copy the right files to generate an "app bundle" for SteamRT, but that flag should be additional behavior, not controlling behavior re: dependencies and such. Does this make sense?

Source/Core/DolphinQt/CMakeLists.txt

OatmealDome · 2022-05-15T08:09:39Z

I solved the FileSystemTests failure while under the steamrt in cd6f27d1, but I'm honestly not sure why reshuffling the code like that fixes it.

The problem appears to involve accessing free'd memory in FstEntry.name during SaveFst(). A valgrind output can be found here.

OatmealDome · 2022-05-22T05:49:35Z

OK, I think this is about ready now. Addressed all comments, cleaned up the commits, and the unit tests no longer fail when running them within the runtime.

I've been uploading builds made with this PR to Steam and they seem to work fine.

JMC47 · 2022-05-28T10:00:40Z

@dolphin-emu-bot rebuild

iwubcode · 2022-05-28T19:06:42Z

Source/Core/DolphinQt/CMakeLists.txt

@@ -607,6 +607,12 @@ if (CMAKE_SYSTEM_NAME STREQUAL "Linux" AND STEAM)
    BUILD_WITH_INSTALL_RPATH true
    INSTALL_RPATH "\$ORIGIN/lib"
  )
+
+  add_custom_command(TARGET dolphin-emu POST_BUILD


Is there a reason this isn't done all the time? Is Qt managed by the system or do we have a special build like on Windows?

There is no Qt present in the steamrt, so yes, we provide our own build. It's not in a submodule, though. I believe we will just have it on the builder.

iwubcode · 2022-05-28T19:08:31Z

Source/Core/DolphinQt/CMakeLists.txt

@@ -593,6 +593,22 @@ else()
  install(TARGETS dolphin-emu RUNTIME DESTINATION ${bindir})
 endif()

+if (CMAKE_SYSTEM_NAME STREQUAL "Linux" AND STEAM)
+  # Set that we want ORIGIN in FLAGS.


Could possibly use BUILD_RPATH_USE_ORIGIN though that wouldn't allow you to disable the tags...

I think I tried that earlier and it used RUNPATH?

iwubcode · 2022-05-28T19:10:48Z

Source/Core/Core/IOS/FS/HostBackend/FS.cpp

  const auto it = std::find_if(old_parent->children.begin(), old_parent->children.end(),
                               GetNamePredicate(split_old_path.file_name));
  if (it != old_parent->children.end())
  {
-    *new_entry = *it;
+    new_entry->data = it->data;


Maybe it doesn't matter but this is different behavior than before because all of the FstEntry isn't copied. It'd be nice to keep the copy constructor behavior. What was the ut failure?

data, children, and name (which we don't care about because it's the old name) are all the members of this struct.

Any test that attempted using Rename would crash when running it within the steamrt. I posted a valgrind output in a comment above.

The Sys folder should be included along with the executable.

This type is already declared in glxext.h in the Steam Runtime.

…n Steam builds for Linux We will store our Qt shared libraries here.

… builds for Linux

…inux

…or Linux

JMC47 · 2022-06-07T00:17:40Z

Looks like reviews have essentially finished. I asked a couple of times if anyone else had any qualms and/or concerns, and no one has spoken up. Let us get the ball rolling.

Source/Core/DolphinQt/CMakeLists.txt

OatmealDome marked this pull request as draft February 18, 2022 19:22

OatmealDome marked this pull request as ready for review February 20, 2022 23:36

OatmealDome force-pushed the steam-runtime branch from 07c9ade to cf84489 Compare February 21, 2022 03:37

OatmealDome changed the title ~~CMakeLists: Add support for a Steam Runtime build~~ DolphinQt: Add support for a Steam Runtime build Feb 21, 2022

Warepire reviewed Feb 21, 2022

View reviewed changes

Source/Core/DolphinQt/CMakeLists.txt Outdated Show resolved Hide resolved

OatmealDome force-pushed the steam-runtime branch from cf84489 to 083f761 Compare February 21, 2022 20:25

Warepire approved these changes Feb 21, 2022

View reviewed changes

OatmealDome force-pushed the steam-runtime branch from 083f761 to c5cf28c Compare March 16, 2022 18:26

OatmealDome force-pushed the steam-runtime branch from c5cf28c to db1e4a7 Compare March 16, 2022 19:37

neobrain reviewed Mar 16, 2022

View reviewed changes

Source/Core/DolphinQt/CMakeLists.txt Outdated Show resolved Hide resolved

OatmealDome force-pushed the steam-runtime branch from db1e4a7 to 0ee3c12 Compare March 18, 2022 07:54

neobrain reviewed Mar 18, 2022

View reviewed changes

Source/Core/DolphinQt/CMakeLists.txt Outdated Show resolved Hide resolved

degasus reviewed Mar 18, 2022

View reviewed changes

Source/Core/DolphinQt/CMakeLists.txt Show resolved Hide resolved

OatmealDome force-pushed the steam-runtime branch from 0ee3c12 to 20b01db Compare March 18, 2022 20:31

OatmealDome force-pushed the steam-runtime branch from 20b01db to 8c59cf6 Compare May 14, 2022 19:07

OatmealDome force-pushed the steam-runtime branch from cd6f27d to aeeb58d Compare May 22, 2022 05:36

iwubcode reviewed May 28, 2022

View reviewed changes

OatmealDome added 9 commits May 29, 2022 13:50

CMakeLists: Add option to produce a Steam build

abb7b35

CMakeLists: Enable LINUX_LOCAL_DEV on Steam builds for Linux

52d7a6f

The Sys folder should be included along with the executable.

GLX: Guard against redefinition of PFNGLXSWAPINTERVALEXTPROC

e679502

This type is already declared in glxext.h in the Steam Runtime.

DolphinQt: Set the executable's rpath to a lib directory in $ORIGIN o…

6b05f5c

…n Steam builds for Linux We will store our Qt shared libraries here.

DolphinQt: Copy all Qt libraries into Binaries output folder on Steam…

77fb5fa

… builds for Linux

DolphinQt: Copy Sys into Binaries output folder on Steam builds for L…

5c4122e

…inux

DolphinQt: Copy qt.conf into Binaries output folder on Steam builds f…

09875c6

…or Linux

FS: Reshuffle Rename to fix steamrt unit test failure

df22439

CMakeLists: Add flag to disable usage of shared libpng

2648837

OatmealDome force-pushed the steam-runtime branch from aeeb58d to 2648837 Compare May 29, 2022 17:50

JMC47 merged commit daf8e3d into dolphin-emu:master Jun 7, 2022

qurious-pixel reviewed Jun 8, 2022

View reviewed changes

Source/Core/DolphinQt/CMakeLists.txt Show resolved Hide resolved

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DolphinQt: Add support for a Steam Runtime build #10462

DolphinQt: Add support for a Steam Runtime build #10462

OatmealDome commented Feb 18, 2022 •

edited

Loading

leoetlino commented Mar 2, 2022

OatmealDome commented Mar 3, 2022

OatmealDome commented Mar 15, 2022

leoetlino commented Mar 16, 2022

AdmiralCurtiss commented Mar 16, 2022

OatmealDome commented Mar 16, 2022

shuffle2 commented Mar 16, 2022 •

edited

Loading

OatmealDome commented Mar 16, 2022

shuffle2 commented Mar 16, 2022 •

edited

Loading

OatmealDome commented Mar 16, 2022

shuffle2 commented Mar 16, 2022

delroth commented Mar 18, 2022

OatmealDome commented May 15, 2022 •

edited

Loading

OatmealDome commented May 22, 2022

JMC47 commented May 28, 2022

iwubcode May 28, 2022

OatmealDome May 28, 2022

iwubcode May 28, 2022

OatmealDome May 28, 2022

iwubcode May 28, 2022

OatmealDome May 28, 2022

JMC47 commented Jun 7, 2022

DolphinQt: Add support for a Steam Runtime build #10462

DolphinQt: Add support for a Steam Runtime build #10462

Conversation

OatmealDome commented Feb 18, 2022 • edited Loading

Building

Running

Acknowledgements

leoetlino commented Mar 2, 2022

OatmealDome commented Mar 3, 2022

OatmealDome commented Mar 15, 2022

leoetlino commented Mar 16, 2022

AdmiralCurtiss commented Mar 16, 2022

OatmealDome commented Mar 16, 2022

shuffle2 commented Mar 16, 2022 • edited Loading

OatmealDome commented Mar 16, 2022

shuffle2 commented Mar 16, 2022 • edited Loading

OatmealDome commented Mar 16, 2022

shuffle2 commented Mar 16, 2022

delroth commented Mar 18, 2022

OatmealDome commented May 15, 2022 • edited Loading

OatmealDome commented May 22, 2022

JMC47 commented May 28, 2022

iwubcode May 28, 2022

Choose a reason for hiding this comment

OatmealDome May 28, 2022

Choose a reason for hiding this comment

iwubcode May 28, 2022

Choose a reason for hiding this comment

OatmealDome May 28, 2022

Choose a reason for hiding this comment

iwubcode May 28, 2022

Choose a reason for hiding this comment

OatmealDome May 28, 2022

Choose a reason for hiding this comment

JMC47 commented Jun 7, 2022

OatmealDome commented Feb 18, 2022 •

edited

Loading

shuffle2 commented Mar 16, 2022 •

edited

Loading

shuffle2 commented Mar 16, 2022 •

edited

Loading

OatmealDome commented May 15, 2022 •

edited

Loading