forked from microsoft/onnxruntime
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix delay load for WebGPU EP and DML EP (microsoft#23111)
### Description This change fixes the DLL delay load problem for the WebGPU EP and DirectML EP. See detailed explanation below. ### Problem When onnxruntime.dll uses delay loading for its dependencies, the dependencies are loaded using `LoadLibraryEx()`, which search the directory of process (.exe) instead of this library (onnxruntime.dll). This is a problem for usages of Node.js binding and python binding, because Windows will try to find the dependencies in the directory of node.exe or python.exe, which is not the directory of onnxruntime.dll. There was previous attempt to fix this by loading DirectML.dll in the initialization of onnxruntime nodejs binding, which works for DML EP but is not a good solution because it does not really "delay" the load. For WebGPU, the situation became worse because webgpu_dawn.dll depends on dxil.dll and dxcompiler.dll, which are explicitly dynamically loaded in the code using `LoadLibraryA()`. This has the same problem of the DLL search. ### Solutions For onnxruntime.dll loading its direct dependencies, it can be resolved by set the [`__pfnDliNotifyHook2` hook](https://learn.microsoft.com/en-us/cpp/build/reference/understanding-the-helper-function?view=msvc-170#structure-and-constant-definitions) to load from an absolute path that constructed from the onnxruntime.dll folder and the DLL name. For webgpu_dawn.dll loading dxil.dll and dxcompiler.dll, since they are explicitly loaded in the code, the hook does not work. Instead, it can be resolved by ~~using WIN32 API `SetDllDirectory()` to add the onnxruntime.dll folder to the search path.~~ preloading the 2 DLLs from the onnxruntime.dll folder .
- Loading branch information
Showing
16 changed files
with
324 additions
and
66 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| // Copyright (c) Microsoft Corporation. All rights reserved. | ||
| // Licensed under the MIT License. | ||
|
|
||
| // == workaround for delay loading of dependencies of onnxruntime.dll == | ||
| // | ||
| // Problem: | ||
| // | ||
| // When onnxruntime.dll uses delay loading for its dependencies, the dependencies are loaded using LoadLibraryEx, | ||
| // which search the directory of process (.exe) instead of this library (onnxruntime.dll). This is a problem for | ||
| // usages of Node.js binding and python binding, because Windows will try to find the dependencies in the directory | ||
| // of node.exe or python.exe, which is not the directory of onnxruntime.dll. | ||
| // | ||
| // Solution: | ||
| // | ||
| // By using the delay load hook `__pfnDliNotifyHook2`, we can intervene the loading procedure by loading from an | ||
| // absolute path. The absolute path is constructed by appending the name of the DLL to load to the directory of | ||
| // onnxruntime.dll. This way, we can ensure that the dependencies are loaded from the same directory as onnxruntime.dll. | ||
| // | ||
| // See also: | ||
| // - https://learn.microsoft.com/en-us/cpp/build/reference/understanding-the-helper-function?view=msvc-170#structure-and-constant-definitions | ||
| // - https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#alternate-search-order-for-unpackaged-apps | ||
| // | ||
| // The DLL DelayLoad hook is only enabled when the compiler is MSVC and at least one of the following is True: | ||
| // - both USE_WEBGPU and BUILD_DAWN_MONOLITHIC_LIBRARY are defined | ||
| // - USE_DML is defined | ||
| // | ||
| #define ORT_DELAY_LOAD_WEBGPU_DAWN_DLL (defined(USE_WEBGPU) && defined(BUILD_DAWN_MONOLITHIC_LIBRARY)) | ||
| #define ORT_DELAY_LOAD_DIRECTML_DLL defined(USE_DML) | ||
| #if defined(_MSC_VER) && (ORT_DELAY_LOAD_WEBGPU_DAWN_DLL || ORT_DELAY_LOAD_DIRECTML_DLL) | ||
|
|
||
| #include <Windows.h> | ||
| #include <delayimp.h> | ||
| #include <stdlib.h> | ||
| #include <string> | ||
|
|
||
| #include "core/platform/env.h" | ||
|
|
||
| namespace { | ||
|
|
||
| #define DEFINE_KNOWN_DLL(name) {#name ".dll", L#name L".dll"} | ||
|
|
||
| constexpr struct { | ||
| const char* str; | ||
| const wchar_t* wstr; | ||
| } known_dlls[] = { | ||
| #if ORT_DELAY_LOAD_WEBGPU_DAWN_DLL | ||
| DEFINE_KNOWN_DLL(webgpu_dawn), | ||
| #endif | ||
| #if ORT_DELAY_LOAD_DIRECTML_DLL | ||
| DEFINE_KNOWN_DLL(DirectML), | ||
| #endif | ||
| }; | ||
| } // namespace | ||
|
|
||
| FARPROC WINAPI delay_load_hook(unsigned dliNotify, PDelayLoadInfo pdli) { | ||
| if (dliNotify == dliNotePreLoadLibrary) { | ||
| for (size_t i = 0; i < _countof(known_dlls); ++i) { | ||
| if (_stricmp(pdli->szDll, known_dlls[i].str) == 0) { | ||
| // Try to load the DLL from the same directory as onnxruntime.dll | ||
|
|
||
| // First, get the path to onnxruntime.dll | ||
| auto path = Env::Default().GetRuntimePath(); | ||
| if (path.empty()) { | ||
| // Failed to get the path to onnxruntime.dll. In this case, we will just return NULL and let the system | ||
| // search for the DLL in the default search order. | ||
| return NULL; | ||
| } | ||
|
|
||
| // Append the name of the DLL. Now `path` is the absolute path to the DLL to load. | ||
| path.append(known_dlls[i].wstr); | ||
|
|
||
| // Load the DLL | ||
| return FARPROC(LoadLibraryExW(path.c_str(), NULL, | ||
| LOAD_LIBRARY_SEARCH_DEFAULT_DIRS | LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR)); | ||
| } | ||
| } | ||
| } | ||
| return NULL; | ||
| } | ||
|
|
||
| extern "C" const PfnDliHook __pfnDliNotifyHook2 = delay_load_hook; | ||
|
|
||
| #endif |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.