Document how to get a MSVCRTD-based debug-build LIB on Windows

[uazu]

This took us a while to figure out. A Rust "staticlib" debug-build outputs a LIB which references MSVCRT. This is not suitable for linking with C++ code built using MSVC debug settings (/MDd), which requires everything to use only MSVCRTD. To get a Rust LIB which references MSVCRTD appears to require CFLAGS=-MDd and CXXFLAGS=-MDd set in the Windows environment (e.g. set CFLAGS=-MDd etc at CMD prompt). This works because cc crate picks these up when it calls cl.exe. In our use-case this caused all the mentions of MSVCRT in the dumpbin /all output for the generated LIB to be replaced with MSVCRTD. Then the link of the LIB to debug C++ code succeeds.

So the question is where this should be documented. It is no use documenting this in cc crate, because that is deep down the crate tree and no-one is going to look there. Perhaps people don't even realize that cc crate is being used. Since it's often someone directly using cxx crate to interface to C++ who would hit this problem, it would likely be helpful to document it in the cxx.rs pages.

(Thanks for cxx crate, by the way -- it helped us a lot!)

[razaqq]

Ran into the same issue (windows, clang-cl), in debug mode you either have to compile the c++ part with /MT or the rust part with /MDd, otherwise they arent compatible for linking, this should honestly be documented somewhere

[Jashwanth537]

@uazu Were you able to get a /Mtd build by setting the CFLAGS ? It might be simple but how to set such CFLAGS in cargo?

[uazu]

@uazu Were you able to get a /Mtd build by setting the CFLAGS ? It might be simple but how to set such CFLAGS in cargo?

Yes, we're still successfully building that way. Since this is a C++ project that calls Rust, we're setting CFLAGS in CMake, since that's the top-level tool that is building both the C++ code and the Rust code. (Don't ask me about CMake though.)

[spangaer]

First

♥ cxx.rs !!!

Disclaimer

I'm a noob on this tech stack and certainly don't have the audacity to claim I can properly document this for CXX.

This ticket was instrumental in solving my problems, though I had to scrape the solution together from many places and want to help out those that come after me.

Sorry for the lengthy text.

Context

I'm attempting to build a Rust cdylib (.so or .dll). I'm only defining Rust-ffi's and "to keep things simple" I chose to compile the Cpp generated wrapper code on the consuming side, which means the dylib boundary becomes the C-ABI.

It will deploy on Linux but it will be hard to pull Cpp devs away from their Visual Studio, so debugging needs to work on Windows & MSVC.

FYI: only generating the Cpp files with Cxx without compiling them along is done with this in build.rs

cxx_build::bridge("src/lib.rs").expand();

Symptoms

I first started out with using a release build on the Rust side and unknowingly combined it with a Debug build on the MSVC side.

When using function CxxString::to_string_lossy() it resulted in panic:

thread '<unnamed>' panicked at alloc\src\raw_vec.rs:24:5:
capacity overflow

I thought I was running in to this mixed release/debug bug, but running it with a debug build only changed the panic:

thread '<unnamed>' panicked at core\src\panicking.rs:221:5:
unsafe precondition(s) violated: slice::from_raw_parts requires the pointer to be aligned and non-null, and the total size of the slice not to exceed `isize::MAX

Because of this.

Analysis

Took me a while to figure out that statement requires the pointer to be aligned and non-null was a red herring. What's actually happening is that CxxString::len() was returning garbage (either 0 or 14757395258967641292), depending a bit on how the string was initialized on Cpp side.

Eventually I suspected that this was some mix of release and debug Microsoft runtimes and when running it in a debugger with module load logging that indeed pop-up.

Loaded 'C:\my-examples\my-examples-cpp\build\Debug\my-examples-cpp.exe'. Symbols loaded.
Loaded 'C:\Windows\System32\ntdll.dll'. 
Loaded 'C:\Windows\System32\kernel32.dll'. 
Loaded 'C:\Windows\System32\KernelBase.dll'. 
Loaded 'C:\Windows\System32\msvcp140d.dll'. 
Loaded 'C:\Windows\System32\vcruntime140d.dll'. 
Loaded 'C:\Windows\System32\vcruntime140_1d.dll'. 
3
Loaded 'C:\Windows\System32\ucrtbased.dll'. 
2
Unloaded 'C:\Windows\System32\ucrtbased.dll'.
Loaded 'C:\my-rs\target\x86_64-pc-windows-msvc\release\my_cxx.dll'. Symbols loaded.
Loaded 'C:\Windows\System32\bcryptprimitives.dll'. 
Loaded 'C:\Windows\System32\ws2_32.dll'. 
Loaded 'C:\Windows\System32\rpcrt4.dll'. 
Loaded 'C:\Windows\System32\crypt32.dll'. 
Loaded 'C:\Windows\System32\ucrtbase.dll'. 
Loaded 'C:\Windows\System32\advapi32.dll'. 
Loaded 'C:\Windows\System32\msvcrt.dll'. 
Loaded 'C:\Windows\System32\sechost.dll'. 
Loaded 'C:\Windows\System32\bcrypt.dll'. 
Loaded 'C:\Windows\System32\vcruntime140.dll'. 
Loaded 'C:\Windows\System32\cryptbase.dll'. 

Notice vcruntime140 first being loaded with and without a d suffix.

So it turns out that trying to read an std::string created with the debug runtime using a release runtime produces this garbage.

Wait a minute, where does this Cpp runtime come from? Didn't you say you put the contract boundary on the C-ABI.

Yes, but it turns out that Cxx internally still uses some Cpp to read the length from that std::string. But if that runtime happens to be the same one used by the consuming application, it's loaded only once and shared. Here both debug vs. release runtimes pretend to be separate libs.

Solution

Building Cxx with debug runtimes

This is where this ticket comes in. So setting those env vars seems to be the (only) way to achieve this.

The most clean vanilla cargo solution I could come up with drew inspiration from here.

.cargo/config.toml

[alias]
bwindbg = "--config .cargo/bwindbg-config.toml build --target x86_64-pc-windows-msvc"
bwinrel = "build --release --target x86_64-pc-windows-msvc"
blindbg = "build --target x86_64-unknown-linux-gnu"
blinrel = "build --release --target x86_64-unknown-linux-gnu"

.cargo/bwindbg-config.toml

[env]
CFLAGS = "/MDd"
CXXFLAGS = "/MDd"

Note that I'm using C++17 below so should do the same here.

cargo.toml

[dependencies]
cxx = { version = "1.0", default-features = false, features = ["c++17", "std"] }

Which then hits a variant of this issue during cdylib linking:

LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library

Linking the debug runtime

So this fix should have solved it, but it kept on failing.

What I did notice is that cargo/rustc kept on inserting msvcrt.lib as linker argument. So while /NODEFAULTLIB:msvcrt would overrule /DEFAULTLIB:msvcrt. The explicit insertion of the .lib was not, which seemed to cause this problem.

I was running rust 1.79. I tried downgrading and upgrading and Rust. 1.83 ended up working with that fix.

Here's my working build.rs construct.

if env::var("TARGET").is_ok_and(|s| s.contains("windows-msvc")) {
    // MSVC compiler suite
    if env::var("CFLAGS").is_ok_and(|s| s.contains("/MDd")) {
        // debug runtime flag is set

        // Don't link the default CRT
        println!("cargo::rustc-link-arg=/nodefaultlib:msvcrt");
        // Link the debug CRT instead
        println!("cargo::rustc-link-arg=/defaultlib:msvcrtd");
    }
}

CMake Cpp project

Given that my CMake project is essentially a collage it seems only fair to share it too.

cmake_minimum_required(VERSION 3.10)
project(my-examples-cpp)

# init - Linux Make GCC: Debug; Windows MSVC: Debug + Release
# cmake -S . -B build

# init - Windows Ninja Clang: Debug
# first run Visual Studio 2022 Developer Command Prompt 
# "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvars64.bat"
# then
# cmake -S . -G "Ninja" -B build-clang  -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl

# init - Make|Ninja: Release
# <init debug> -D CMAKE_BUILD_TYPE=Release

# build - Make: as inited; Windows MSVC: Debug
# cmake --build build
# build - Ninja: as inited
# cmake --build build-clang
# build - Windows MSVC: Release
# cmake --build build --config Release

set(CMAKE_CXX_STANDARD 17)

if (WIN32)
    set(RUST_TARGET ${CMAKE_CURRENT_SOURCE_DIR}/lib/x86_64-pc-windows-msvc)
    set(MY_LIB_NAME my_cxx.dll.lib)
    if (MSVC)
        # using UTF-8 source
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /utf-8")
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /utf-8")
    endif()
else()
    set(RUST_TARGET ${CMAKE_CURRENT_SOURCE_DIR}/lib/x86_64-unknown-linux-gnu)
    set(MY_LIB_NAME libmy_cxx.so)
endif()

include_directories(${RUST_TARGET}/cxxbridge/my-cxx/src)
add_executable(my-examples-cpp src/main.cpp ${RUST_TARGET}/cxxbridge/my-cxx/src/lib.rs.cc)

# Link the dynamic library
if(CMAKE_CONFIGURATION_TYPES)
    # https://stackoverflow.com/a/2224732/574370
    target_link_libraries(my-examples-cpp debug ${RUST_TARGET}/debug/${MY_LIB_NAME})
    target_link_libraries(my-examples-cpp optimized ${RUST_TARGET}/release/${MY_LIB_NAME})
elseif(CMAKE_BUILD_TYPE STREQUAL "Release")
    target_link_libraries(my-examples-cpp ${RUST_TARGET}/release/${MY_LIB_NAME})
else()
    target_link_libraries(my-examples-cpp ${RUST_TARGET}/debug/${MY_LIB_NAME})
endif()

Obviously you have to figure out your own paths there. I lost quite a bit of time over the .so having a lib prefix while the .dll(.lib) variants don't...

Don't forget

To make sure your custom built .dll is also available on the PATH when debugging/running!

Conclusion

If all those pieces of the puzzle fall in to place, now std::string/CxxString struct are handled by the same Cpp-runtime and not a debug/release mix, string length reports correct values and all panincs disappear.