libwow64fex.dll FTBS with llvm-mingw

[AndreRH]

Used toolchain:
llvm-mingw-20240518-ucrt-ubuntu-20.04-aarch64

How I tried to build:

mkdir build_pe
cd build_pe
export PATH=/path/to/llvm-mingw/bin:$PATH
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchain_mingw.cmake -DENABLE_JEMALLOC=0 -DENABLE_JEMALLOC_GLIBC_ALLOC=0 -DMINGW_TRIPLE=aarch64-w64-mingw32 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTS=False -DENABLE_ASSERTIONS=False ..
make -j$(nproc) wow64fex

Problems:

/path/to/fex/Source/Windows/Common/CRT/String.cpp:169:21: error: redefinition of '_sscanf_l'
169 | DLLEXPORT_FUNC(int, _sscanf_l, (const char* buffer, const char* format, _locale_t locale, ...)) {
| ^
/path/to/llvm-mingw-20240518-ucrt-ubuntu-20.04-aarch64/aarch64-w64-mingw32/include/sec_api/stdio_s.h:160:27: note: previous definition is here
160 | __mingw_ovr int __cdecl _sscanf_l(const char _Src, const char _Format, _locale_t _Locale, ...)
| ^
/path/to/fex/Source/Windows/Common/CRT/String.cpp:177:39: error: functions that differ only in their return type cannot be overloaded
177 | DLLEXPORT_FUNC(const unsigned short, __pctype_func, (void)) {
| ~~~~~~~~~~~~~~~ ^
/path/to/fex/Source/Windows/Common/CRT/../Priv.h:62:7: note: expanded from macro 'DLLEXPORT_FUNC'
62 | Ret Name Args;
| ~~~ ^
/path/to/llvm-mingw-20240518-ucrt-ubuntu-20.04-aarch64/aarch64-w64-mingw32/include/ctype.h:29:27: note: previous declaration is here
29 | _CRTIMP unsigned short __pctype_func(void);
| ~~~~~~~~~~~~~~~ ^
/path/to/fex/Source/Windows/Common/CRT/String.cpp:177:1: error: cannot initialize a variable of type 'const unsigned short ()()' with an lvalue of type 'unsigned short *()': different return type ('const unsigned short ' vs 'unsigned short ')
177 | DLLEXPORT_FUNC(const unsigned short, __pctype_func, (void)) {
| ^ ~~~~~~~~~~~~~
/path/to/fex/Source/Windows/Common/CRT/../Priv.h:63:8: note: expanded from macro 'DLLEXPORT_FUNC'
63 | Ret(_imp##Name) Args = Name;
| ^ ~~~~
:199:1: note: expanded from here
199 | __imp___pctype_func
| ^

... and

/path/to/fex/Source/Windows/Common/CRT/IO.cpp:281:5: error: redefinition of 'fseeko64'
281 | int fseeko64(FILE* File, _off64_t Offset, int Origin) {
| ^
/path/to/llvm-mingw-20240518-ucrt-ubuntu-20.04-aarch64/aarch64-w64-mingw32/include/stdio.h:667:19: note: previous definition is here
667 | __mingw_ovr int fseeko64(FILE _File, _off64_t _Offset, int _Origin) {
| ^
/path/to/fex/Source/Windows/Common/CRT/IO.cpp:286:5: error: redefinition of 'fseeko'
286 | int fseeko(FILE File, _off_t Offset, int Origin) {
| ^
/path/to/llvm-mingw-20240518-ucrt-ubuntu-20.04-aarch64/aarch64-w64-mingw32/include/stdio.h:664:19: note: previous definition is here
664 | __mingw_ovr int fseeko(FILE _File, _off_t _Offset, int _Origin) {
| ^
/path/to/fex/Source/Windows/Common/CRT/IO.cpp:294:10: error: redefinition of 'ftello64'
294 | _off64_t ftello64(FILE File) {
| ^
/path/to/llvm-mingw-20240518-ucrt-ubuntu-20.04-aarch64/aarch64-w64-mingw32/include/stdio.h:673:24: note: previous definition is here
673 | __mingw_ovr _off64_t ftello64(FILE _File) {
| ^
/path/to/fex/Source/Windows/Common/CRT/IO.cpp:300:8: error: redefinition of 'ftello'
300 | _off_t ftello(FILE File) {
| ^
/path/to/llvm-mingw-20240518-ucrt-ubuntu-20.04-aarch64/aarch64-w64-mingw32/include/stdio.h:670:22: note: previous definition is here
670 | __mingw_ovr _off_t ftello(FILE *_File) {
| ^
4 errors generated.

[AndreRH]

I worked around it by reverting FEX CRT related commits:
https://github.com/AndreRH/FEX/commits/hangover-9.15

@bylaws did you build with "regular" clang, or why didn't you get those errors?

[bylaws]

Oh i am building with a msvcrt based toolchain rather than a ucrt one so I never ran into this. (Or see the msvcrt tagged ones on official repo)
https://github.com/bylaws/llvm-mingw/releases/tag/20240630

I think most of these errors should be fixable with just some UCRT ifdefs to implement e.g. _fseeki64 instead of fseeko64. Note that with new crt fex you can drop all of the wine patches it used to need (just need the dll path one (upstream as of yesterday), suspend one and CPU register one for now (new, see wine MRs))

[bylaws]

If we want to support ucrt based toolchains ideally they can be set up for CI as well, though I'm not sure it makes sense to support both over than just msvcrt though

[AndreRH]

I understand msvcrt as legacy, especially for aarch64. Your toolchains also say ucrt btw, likely unintentionally I guess.

Ifdefs won't be enough it seems. I removed some functions like fseeko64 and friends and it compiled, but the linker complained about a lot of missing functions like for example fprintf...

[bylaws]

In this case fex isn't even using the crt so legacy doesn't really matter, it's just there's a different set of funcs implemented in headers between the two. Originally we used msvcrt because games can ship their own ucrt and that broke back when fex didn't ship it's own.

If things are more complex with ucrt though I'd be inclined to just support msvcrt, perhaps I can see some way to force msvcrt usage on an UCRT toolchain for FEX.

[AndreRH]

Good plan, so I prepared to do my builds with msvcrt based toolchains instead and got:

[100%] Linking CXX shared library ../../../Bin/libwow64fex.dll
clang: warning: argument unused during compilation: '-static-libgcc' [-Wunused-command-line-argument]
clang: warning: argument unused during compilation: '-static-libstdc++' [-Wunused-command-line-argument]
ld.lld: error: undefined symbol: __guard_check_icall_fptr

referenced by libc++.a(stdlib_new_delete.cpp.obj):(.weak._Znwy.default.__clang_call_terminate)
referenced by libc++.a(stdlib_new_delete.cpp.obj):(.weak._Znwy.default.__clang_call_terminate)
referenced by libc++.a(stdlib_new_delete.cpp.obj):(.weak._ZnwySt11align_val_t.default.__clang_call_terminate)
referenced 2242 more times

This was with:
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchain_mingw.cmake -DMINGW_TRIPLE=aarch64-w64-mingw32 -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTS=False -DENABLE_LTO=False -DENABLE_ASSERTIONS=True ..
as my original cmake settings failed even harder with missing jemalloc symbols

Toolchain:
llvm-mingw-20240619-msvcrt-ubuntu-20.04-x86_64

[bylaws]

Ah... I also disable cfguard for my toolchain because it's broken for arm64ec. Can fix this easy enough though will do.

If you wanna fix yourself just need to copy the load config from the arm64ec module and add back the cfguard sums - see mingw crt

[AndreRH]

#4060 is in, thanks
Currently it fails with upstream llvm-mingw at link time:

ld.lld: error: undefined symbol: FlushInstructionCache
>>> referenced by libclang_rt.builtins-aarch64.a(clear_cache.c.obj):(__clear_cache)
clang: error: linker command failed with exit code 1 (use -v to see invocation)