Generate appimage for any upstream tag

[lestcape]

Hi is there any chance you can generate a build for the official releases/tags?

Maybe that helps to know what tag you not generate yet:

toGenerate=() #The tags that should be generate
wget -q "https://api.github.com/repos/dolphin-emu/dolphin/git/refs/tags" -O dolphin-emu.json
wget -q "https://api.github.com/repos/pkgforge-dev/Dolphin-emu-AppImage/git/refs/tags" -O dolphin-emu-AppImage.json
upstreamTags=($(jq .[].ref dolphin-emu.json))
tags=($(jq .[].ref dolphin-emu-AppImage.json))
for index in "${!upstreamTags[@]}"; do
    tag=${upstreamTags[$index]}
    if ! [[ ${tags[@]} =~ \<$tag\> ]]; then
        tag=${tag/refs\/tags\//}
        toGenerate+=($tag)
    fi
done;

[Samueru-sama]

I was on the middle setting up versioned releases in this branch.

Is this what you mean?

[lestcape]

No, I'm not talking about your release versions, but about the upstream release versions. You are currently generate your releases using the master branch. I mean instead you should use of the upstream tags as your source. So, you will have the upstream releases (https://github.com/dolphin-emu/dolphin/tags):

"nJoy"
"2.0"
"3.0"
"3.5"
"4.0"
"4.0.1"
"4.0.2"
"5.0-rc"
"5.0"
"2407"
"2409"
"2412"

and any other new release that upstream will create in the future.

[Samueru-sama]

Yeah the thing is that the AppImage is no longer being built and we just take the binaries from the archlinux package to make the AppImage.

I guess we could start building again though

You are currently generate your releases using the master branch

I just made a release with the versioned releases and also a PR adding this to the main branch.

[lestcape]

Yeah the thing is that the AppImage is no longer being built

Oh what I was seeing was this, which made me think you were using the source:
https://github.com/pkgforge-dev/Dolphin-emu-AppImage/blob/main/.github/workflows/blank.yml

Specially because of this:
git clone --filter="blob:none" --depth=1 -b master https://github.com/dolphin-emu/dolphin.git dolphin-emu

The script i shared will tell you what tags are missing and then you can generate your build for that upstream tags:

for index in "${!toGenerate[@]}"; do
    tag=${toGenerate[$index]}
    echo "Missing tag: $tag"
    git clone --depth=1 -b $tag https://github.com/dolphin-emu/dolphin.git dolphin-emu
    ....
done;

I actually don't know how arch repositories work, but if using arch you can generate an appimage for each official release, then for me that would work too.

[Samueru-sama]

Oh what I was seeing was this, which made me think you were using the source:
https://github.com/pkgforge-dev/Dolphin-emu-AppImage/blob/main/.github/workflows/blank.yml

Yeah that's old, right now this is being used instead to get the dolphin binary and this makes the AppImage.

The repo was recently added to pkgforge-dev less than 24 hours ago as well xd

I actually don't know how arch repositories work, but if using arch you can generate an appimage for each official release, then for me that would work too.

Right now the CI setup to build every 6 days and every 6 days there will be a new release, even if it is the same dolphin version.

This is done because we bundle all the libs in the AppImage to make it able to really run on any linux system, be it ubuntu 14.04 or alpine linux. This way me make sure the dependencies we bundle are always up to date and have security fixes from upstream.

So all good? Once #11 gets merged this can be closed?

[lestcape]

This issue is resolved when the appimage that you generate, will correspond to one of the official release. So, we have a way to know what version we really have, to be possible report issues in the upstream.

Since I cannot answer for what the arch maintainers do, nor do I really know how they work, i have no idea then if #11 will fix the issue. If you think it will fix it, just close this issue for me.

Ultimately, this issue is more of a recommendation to generate images comparable to the official releases. It's not that it necessarily has to be how I think it's best.

[Samueru-sama]

This issue is resolved when the appimage that you generate, will correspond to one of the official release.

The AppImages made here will contain a version that correspond to the version of the Archlinux package. maybe in the future we start building dolphin again and that way we use the official release tags.

So I will mark the PR closing this.

So, we have a way to know what version we really have, to be possible report issues in the upstream.

Always come here first instead when reporting issues instead of going upstream please. Developers of projects actually hate this and it is one of the reasons why the duckstation developer recently changed the license of his project a non free one because it seems he was fed up with people having issues with the Aur package lol.

Ultimately, this issue is more of a recommendation to generate images comparable to the official releases. It's not that it necessarily has to be how I think it's best.

Worth adding that the ideal solution is for dolphin to start release the AppImage officially themselves, there is actually a pending PR here.

With that sad the PR would make an AppImage that uses linuxdeploy, so it really wouldn't work on all linux systems like this one does instead.

[lestcape]

Ok, also, can you generate an appimage for the older upstream versions? Emulators unfortunately sometimes tend to increase their basic requirements as time goes by and sometimes for an older PC, the only solution is to use an older version of the emulator that runs in a new OS.

[Samueru-sama]

Ok, also, can you generate an appimage for the older upstream versions? Emulators unfortunately sometimes tend to increase their basic requirements as time goes by and sometimes for an older PC, the only solution is to use an older version of the emulator that runs in a new OS.

Are you having a problem similar to this?

It is highly unlikely that we can do anything, because if it is that the opengl that we bundle comes from archlinux, would need to go back to using an old distro and then either hope there is already some dolphin binary or build it again which is pain.

[lestcape]

Are you having a problem similar to...?

Not with Dolphin yet, but with Yuzu. Yuzu dropped the support on Linux for hardware without the SSE4.2 instruction set. But this is something that is not only Yuzu's, but it is not just me, it will happen to someone else, that is why it would be highly recommended to have an appimage of all the previous official versions. And that is part of the reason for this issue also. With the method to generate the previous images the risk of something like this happening is reduced.

[Samueru-sama]

Not with Dolphin yet, but with Yuzu. Yuzu dropped the support on Linux for hardware without the SSE4.2 instruction set. But this is something that is not only Yuzu's, but it is not just me, it will happen to someone else, that is why it would be highly recommended to have an appimage of all the previous official versions. And that is part of the reason for this issue also. With the method to generate the previous images the risk of something like this happening is reduced.

Oh, I actually know a few things about yuzu since I used to hang out at their discord.

The thing is that yuzu changed their appimage to be built for x86_64 v2 CPUs, this was done because it had a +8% boost on fps.

Funny enough they told everyone who had a CPU that didn't support that to move to the flatpak which was built with generic flags due to flathub policy or something.

Note that suyu has official AppImage releases but I think they also target x86_64 v2, give it a shot just in case.

You can still build new versions of yuzu/suyu/sudachi as long as you use generic flags, I have plans on making an AppImage of sudachi btw.

Also note that Ryujinx has official appimage releases and far as I know they use generic flags. But I get performance isn't great on Ryujinx, it actually drove me to make this bug report on mesa to fix an issue I was having with suyu and they fixed it!

[lestcape]

Well, that's why an older version is useful. I have yuzu-20211224-03bd85abd.AppImage that works on my PC, it just needed an extraction and re-image to get it working. The latest version of Ryujinx runs at 30fps on my PC, while with the same game and the same PC, the old version of Yuzu from 2021 hits 60fps which is the maximum the game supports.

[Samueru-sama]

Well, that's why an older version is useful.

No, this is bad, what needs to be done is release generic and x86_64-v3 builds. yuzu went middleway hoping that x86_64v2 would be good enough but this just didn't make the people that had v3 capable harware happy and cutoff the people that could not use it anymore.

I have plans on making sudachi appimages, will release them as generic buids and x86_64 v3, this is what Zen browser was doing for a while until they gave up making x86_64 v3 builds due to having some issues lol.

[lestcape]

Yes look, what should ideally happen is that at the code/runtime level (not at the compilation level) it could be decided based on the user's hardware what type of code should be executed on that hardware. In this way the same compiled code could be then used on different architectures without requiring any changes. However, there are always those in the middle who do not want to maintain several architectures and get rid of the old ones which then stop working on current software. And that put the people below them who use that code in a bind, because now they have to decide what they are going to do and no solution seems perfect now.

Unfortunately that is the reality and it is not going to change. What can change is that we prepare for when this happens and preparing means having an old version that we can fall back on when the new one no longer works.

[Samueru-sama]

Yes look, what should ideally happen is that at the code/runtime level (not at the compilation level) it could be decided based on the user's hardware what type of code should be executed on that hardware. In this way the same compiled code could be then used on different architectures without requiring any changes. However, there are always those in the middle who do not want to maintain several architectures and get rid of the old ones which then stop working on current software. And that put the people below them who use that code in a bind, because now they have to decide what they are going to do and no solution seems perfect now.

Unfortunately that is the reality and it is not going to change. What can change is that we prepare for when this happens and preparing means having an old version that we can fall back on when the new one no longer works.

I think what will end up happening (and it is happening) is that distros ship packages for old and new hardware, this is already what gentoo does now that they ship binaries iirc, you can either get them as generic or x86_64 v3.

I disagree with making binaries of old versions of software for people that no longer are supported, this has to be fixed in a different way. With that said almost all the AppImages we make here do have versioned releases and there will always be old versions to fall back to. I intended this originally just to avoid pushing breaking bugs into people and not about unsupported hardware though 😅

I know a lot of bug fixes that yuzu has had since they switched to x86_64 v2, and some a very important and include very important optimizations, this is why I don't think it is a good idea to explicitly build old versions.

[lestcape]

It is because you think the way you think that hardware lose support. That, for example, does not happen with the kernel, nor does it happen with the shell, which today works similar of how a game work. In Linux this only happens with games and it happens precisely because there are a lot of people that think that way, like you (with or without reason). In short, it is possible to introduce important optimizations without results in the loss of support for others, but of course it is hard work to do it that way and not everyone is willing to make that hard work.

this has to be fixed in a different way

Saying another way without saying which one, is like saying nothing, is just justifying yourself and pass the problem on to someone else and is exactly what I meant from the beginning.

And yet, we must be grateful for the work you are doing, taking risks and freely giving your work. It's just real life.

[Samueru-sama]

Saying another way without saying which one, is like saying nothing

I just told you the method! Release generic and x86_64 v3 builds, right now this appimage only targets generic so there is no problem to begin with.

That, for example, does not happen with the kernel, nor does it happen with the shell

Yes it does, maybe not the kernel but the shell and all your distro packages are starting to come with optimization levels.

In Linux this only happens with games

It happens with all sort of binaries, just that more often you see it with "Games" because it is where it usually matters the most.

it is possible to introduce important optimizations without results in the loss of support for others, but of course it is hard work to do it that way and not everyone is willing to make that hard work.

Yes but I'm not the developer of the application, all I can do is offer to compile binaries for x86_64 generic for people like you, which I have no problem in doing.

Other developers don't even bother to make binary releases of their products at all for linux, like mpv.

[Samueru-sama]

That, for example, does not happen with the kernel, nor does it happen with the shell

And just to give examples:

https://www.phoronix.com/news/Gentoo-x86-64-v3-Binaries

https://www.phoronix.com/review/ubuntu-x86-64-v3-benchmark

https://www.phoronix.com/review/cachyos-x86-64-v3-v4

https://www.phoronix.com/news/RedHat-RHEL10-x86-64-v3-Explore

https://www.phoronix.com/news/Fedora-42-Optimized-Executables

And yet, we must be grateful for the work you are doing, taking risks and freely giving your work. It's just real life.

What am I doing wrong????

[lestcape]

That hardware improves every year is a fact, no one can deny that. That old architectures have to be abandoned just because of that is what TODAY in Linux only happens with games and of course it can happen with all software, because it is not something that the software determines, it is determined by the humans who make the software.

I just told you the method! Release generic and x86_64 v3 builds,

That's not a solution, it's the opposite. If you release more than one different build for everything, support will become chaotic on Linux, because some developers will choose to support only one branch of their software and others only the other. Instead of solving the problem, by fragmenting it, you make it worse. My opinion, of course.

[Samueru-sama]

That's not a solution, it's the opposite

It's either that or just not having generic buidls at all.

I know you can check the CPU at runtime to see if it supports the instructions you want for better performance, but no developer bothers to do this, I 100% agree this is bad but can I do here???

All I can do to help you is compile the software you need for generic builds and responded with:

"And yet, we must be grateful for the work you are doing, taking risks and freely giving your work. It's just real life." ???

[lestcape]

I 100% agree this is bad but can I do here???

We can do nothing.

All I can do to help you is compile the software you need for generic builds

It's not about you, I used you as an example to make my point clear. Obviously it's clear, because now we agree 100%.

[Samueru-sama]

@lestcape Ok, if you need to talk about anything else you can check our discord which you can find the link if you check the organization.

You will actually find me sending a message to another dev here 11 hours ago of this message of yours with the caption "another example as to why it is a bad idea to use cachyos repos" because a while ago he was considering making optimized releases of his software using that, and I had told them before that was a bad idea.

[lestcape]

@Samueru-sama I think language support (spanish) is missing on appimage.

Have a good day.

[Samueru-sama]

@Samueru-sama I think language support (spanish) is missing on appimage.

Have a good day.

I have the same issue with the native dolphin-emu package on my PC.

It also seems the flatpak has the same issue: https://bugs.dolphin-emu.org/issues/13709

I also tried to debug with strace to see if dolphin isn't finding a file and I can't see a difference when I change the language in the settings, so I don't know how dolphin loads another language.

[lestcape]

Yes well, it's strange that arch packages also have that problem. They reallocate the package? I actually think the problem is because the package is compiled with support for a relocatable linux builds https://github.com/dolphin-emu/dolphin/blob/master/CMakeLists.txt#L150

This is because there are a problem between where is supposed to be set the linux build (https://github.com/dolphin-emu/dolphin/blob/3ea870ef8c5f4d4b86bd3ce0c2e3ddba6f1f25aa/Source/Core/DolphinQt/Translation.cpp#L284-L288) and where is compiled de language files (https://github.com/dolphin-emu/dolphin/blob/3ea870ef8c5f4d4b86bd3ce0c2e3ddba6f1f25aa/Source/Core/DolphinQt/CMakeLists.txt#L554).

If LINUX_LOCAL_DEV is enabled and you are looking for the mo files in Source/Core/DolphinQt, you should also compile them there and not in share/locale/.

[Samueru-sama]

I don't think the dolphin binaries of arch are compiled with LINUX_LOCAL_DEV because we had to use this library to overwrite a hardcoded path in dolphin.

When the appimage was compiled manually on ubuntu, all that was needed was add a Sys dir next to the dolphin binary. You can see it if you extract this AppImage

However even with that AppImage I still can't switch language.

The native arch package comes with dolphin-emu.mo files in /usr/share/locale however it is not working either.

So just to make sure, I should drop the .mo fiels in Source/Core/DolphinQt relative to the dolphin binary location?

[Samueru-sama]

Found a solution 👀

[Samueru-sama]

fixed by #13

[lestcape]

Yes, I had already extracted the image and I had seen that the mo were not anywhere. I tried to put them everywhere where I thought they should be, but that didn't work. What did work was to give you the little push you needed to fix that :).

[Samueru-sama]

@lestcape I ended up making an AppImage of Torzu instead: https://github.com/Samueru-sama/Torzu-AppImage/releases/tag/r27198.02cfee3f1

It should work on your PC since I patched out that line that makes it build for x86-64-v2

However there is a big issue, on my PC when using vulkan it just crashes with this error:

yuzu: /build/spirv-tools/src/SPIRV-Tools-2024.4.rc2/source/opt/propagator.cpp:265: spvtools::opt::SSAPropagator::Run(spvtools::opt::Function*)::<lambda(spvtools::opt::Instruction*)>: Assertion `(!HasStatus(inst) || Status(inst) != SSAPropagator::kNotInteresting) && "Unsettled value"' failed.

This is an error that I have if I build the Aur package on my PC, it even happens with these pre-compiled builds, so there isn't anything I can do to fix the issue.

It is likely that sudachi doesn't have this bug, but the reason I want with Torzu is because there is no PKGBUILD for sudachi and building this project manually isn't as easy as building dolphin.

EDIT: I think I have solved the issue, by default the aur package uses the spirv headers of the host but it can also use some external ones setting this flag.