Skip to content

Mempool memory resource - IPC #446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 29 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
247af8c
add IPC support to default async mempool
ksimpson-work Feb 24, 2025
4b52151
revert utils
ksimpson-work Feb 24, 2025
050e4d6
revert utils
ksimpson-work Feb 24, 2025
1da30fa
rever utils
ksimpson-work Feb 24, 2025
ebef438
fix typo
ksimpson-work Feb 25, 2025
5023339
fix typo
ksimpson-work Feb 25, 2025
d0f0bbc
merge main
ksimpson-work Feb 25, 2025
e4da633
another typo
ksimpson-work Feb 25, 2025
77678e9
make IPC buffer a descriptor not a buffer
ksimpson-work Feb 25, 2025
1711f78
support windows
ksimpson-work Feb 25, 2025
4f25290
remove super call
ksimpson-work Feb 26, 2025
24a5652
add security handle with defualt no security for now
ksimpson-work Feb 26, 2025
da08473
push for local test
ksimpson-work Feb 26, 2025
9e1d546
switch dll call
ksimpson-work Feb 26, 2025
7deb681
use library
ksimpson-work Feb 26, 2025
c4e59f6
use library
ksimpson-work Feb 26, 2025
2294632
use library
ksimpson-work Feb 26, 2025
62f1e93
use library
ksimpson-work Feb 26, 2025
2e5013a
use library
ksimpson-work Feb 26, 2025
0c82df8
use library
ksimpson-work Feb 26, 2025
1c6ee59
use library
ksimpson-work Feb 26, 2025
3074742
use library
ksimpson-work Feb 26, 2025
f35cf01
use library
ksimpson-work Feb 26, 2025
bff995a
use library
ksimpson-work Feb 26, 2025
5fd7f46
remove windows stuff from async mempool until it is supported
ksimpson-work Mar 5, 2025
d486fcb
Merge branch 'main' into IPC
ksimpson-work Mar 5, 2025
c39d9a9
remove windows stuff from async mempool until it is supported
ksimpson-work Mar 5, 2025
140ba3f
Merge remote-tracking branch 'origin/IPC' into IPC
ksimpson-work Mar 5, 2025
1bcebd5
Merge branch 'main' into IPC
ksimpson-work Mar 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 48 additions & 37 deletions cuda_core/cuda/core/experimental/_memory.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,21 @@
from cuda.core.experimental._stream import default_stream
from cuda.core.experimental._utils import driver, handle_return

# Check if pywin32 is available on Windows
_PYWIN32_AVAILABLE = False
if platform.system() == "Windows":
try:
import win32security

_PYWIN32_AVAILABLE = True
except ImportError:
import warnings

warnings.warn(
"pywin32 module not found. For better IPC support on Windows, " "install it with: pip install pywin32",
stacklevel=2,
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How hard would it be to recreate the functionality we need from here to avoid this optional dependency?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'll need to discuss our next move first and then clean this up based on the decision. Keenan struggled for quite a long time and it turns out the IPC support for mempools is not enabled on Windows at all. There is a number of doc and sample bugs we'll create later (the memory team created one).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created 2 bugs to address the documentation issues.

yeah the review is in limbo right now. I'm not sure if we should push the change for IPC mempools and have it only support linux for now, or if we should shelve it and add a cuMem and VMM API IPC memory_resource in the meantime.

All of the Windows specific changes should be treated as draft. That optional dependency is from when I was debugging windows errors and operating under the assumption that I was improperly creating the windows security descriptor for the mempool handle. In hindsight it was a (temporary) lack of suport in driver layer as Leo said.


PyCapsule = TypeVar("PyCapsule")


Expand Down Expand Up @@ -287,8 +302,8 @@ def _get_platform_handle_type() -> int:
def _create_win32_security_attributes():
"""Creates a Windows SECURITY_ATTRIBUTES structure with default settings.

The security descriptor is configured with a DACL that allows access to everyone,
which is appropriate for shared memory that needs to be accessible across processes.
The security descriptor is configured to allow access across processes,
which is appropriate for shared memory.

Returns:
A pointer to a SECURITY_ATTRIBUTES structure or None if not on Windows.
Expand All @@ -304,54 +319,50 @@ class SECURITY_ATTRIBUTES(ctypes.Structure):
("bInheritHandle", ctypes.c_int),
]

# Constants for security descriptor creation
SECURITY_DESCRIPTOR_REVISION = 1
SECURITY_DESCRIPTOR_MIN_LENGTH = 1024
if _PYWIN32_AVAILABLE:
# Create a security descriptor using pywin32
sd = win32security.SECURITY_DESCRIPTOR()

# Create a new security descriptor - use kernel32 for memory allocation
# LPTR = 0x0040 (LMEM_ZEROINIT | LMEM_FIXED)
LPTR = 0x0040
security_descriptor = ctypes.windll.kernel32.LocalAlloc(LPTR, SECURITY_DESCRIPTOR_MIN_LENGTH)
# Create a blank DACL (this allows all access)
dacl = win32security.ACL()

if not security_descriptor:
return None
# Set the DACL to the security descriptor
sd.SetSecurityDescriptorDacl(1, dacl, 0)

# Initialize the security descriptor
if not ctypes.windll.advapi32.InitializeSecurityDescriptor(security_descriptor, SECURITY_DESCRIPTOR_REVISION):
ctypes.windll.kernel32.LocalFree(security_descriptor)
return None
# Create and initialize the security attributes structure
sa = SECURITY_ATTRIBUTES()
sa.nLength = ctypes.sizeof(SECURITY_ATTRIBUTES)
sa.lpSecurityDescriptor = ctypes.c_void_p(int(sd.SECURITY_DESCRIPTOR))
sa.bInheritHandle = False

# Set a NULL DACL which allows all access to everyone
# 3rd parameter is a BOOL that specifies whether to set a DACL (TRUE) or not (FALSE)
# 4th parameter is the DACL pointer (NULL for unrestricted access)
# 5th parameter is a BOOL that specifies whether the DACL was explicitly provided (TRUE) or defaulted (FALSE)
if not ctypes.windll.advapi32.SetSecurityDescriptorDacl(security_descriptor, True, None, False):
ctypes.windll.kernel32.LocalFree(security_descriptor)
return None
# Store the security descriptor to prevent garbage collection
if not hasattr(_create_win32_security_attributes, "_security_descriptors"):
_create_win32_security_attributes._security_descriptors = []
_create_win32_security_attributes._security_descriptors.append(sd)

# Create and initialize the security attributes structure
sa = SECURITY_ATTRIBUTES()
sa.nLength = ctypes.sizeof(SECURITY_ATTRIBUTES)
sa.lpSecurityDescriptor = security_descriptor
sa.bInheritHandle = False # Don't inherit handle
return ctypes.addressof(sa)
else:
# If pywin32 is not available, use a NULL security descriptor
# This is less secure but should work for testing
try:
sa = SECURITY_ATTRIBUTES()
sa.nLength = ctypes.sizeof(SECURITY_ATTRIBUTES)
sa.lpSecurityDescriptor = ctypes.c_void_p(0) # NULL security descriptor
sa.bInheritHandle = False

# Store the security descriptor in a global variable to prevent it from being garbage collected
# and to allow cleanup when the module is unloaded
if not hasattr(_create_win32_security_attributes, "_security_descriptors"):
_create_win32_security_attributes._security_descriptors = []
_create_win32_security_attributes._security_descriptors.append(security_descriptor)
return ctypes.addressof(sa)

# Return a pointer that can be passed to the CUDA API
return ctypes.addressof(sa)
except Exception as e:
print(f"Warning: Failed to create security attributes: {e}")
return 0 # Return 0 as a fallback


# Add cleanup function for security descriptors
def _cleanup_security_descriptors():
"""Free any allocated security descriptors when the module is unloaded."""
if hasattr(_create_win32_security_attributes, "_security_descriptors"):
for sd in _create_win32_security_attributes._security_descriptors:
if sd:
ctypes.windll.kernel32.LocalFree(sd)
# The security descriptors are now pywin32 objects that will be garbage collected
# or simple ctypes structures, so we just need to clear the list
_create_win32_security_attributes._security_descriptors.clear()


Expand Down