Skip to content

Conversation

@MatthiasValvekens
Copy link
Collaborator

This is a big refactor intended to clean up the way the library handles internal state a bit (library handles, running operations, etc.).

The main goals were the following:

  • Remove the limitation that python-pkcs11 can only be used with one PKCS#11 library at once (see e.g. Unload a pkcs11.lib ? #130)
  • Make sure that all multi-step operations (search, chunked cryptographic ops, etc.) run in context managers that cancel the underlying operation when they exit prematurely (see e.g. "FIXME: cancel the operation when we exit the generator early." #75).
  • Automatically deal with buffer size increases requested by the token wherever possible (preferably without introducing a ton of boilerplate).

Challenges faced:

  • The multi-token support necessitated getting rid of the global _funclist pointer and passing it around between classes.
  • Introducing the token operation management uniformly without duplicating a bunch of code was also surprisingly tricky.

I ended up rewriting most of the internal Cython types as cdef class classes to achieve this, since those classes can have C-level values (including raw pointers) as attributes. I handled the token operation management code similarly.

@MatthiasValvekens MatthiasValvekens force-pushed the feature/internal-state-systematisation branch from 918499b to f339867 Compare June 24, 2025 00:53
cdef CK_VERSION fw_version

@staticmethod
cdef Slot make(CK_FUNCTION_LIST *funclist, CK_SLOT_ID slot_id, CK_SLOT_INFO info, tuple cryptoki_version):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using Cython extension types (cdef classes) pretty much forces moving a bunch of state-touching logic from types.py to the Cython layer. It's not too bad, though. API-wise, there's no real breakage for a user that doesn't access any of the Cython internals directly.

)


cdef class OperationContext:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main context manager class that monitors operations. There are subtypes for search & crypto ops.

# All other APIs are taken from the CK_FUNCTION_LIST table
ctypedef CK_RV (*C_GetFunctionList_ptr) (CK_FUNCTION_LIST **) nogil

ctypedef CK_RV (*KeyOperationInit) (
Copy link
Collaborator Author

@MatthiasValvekens MatthiasValvekens Jun 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some function pointer typedefs for cryptographic operations. Used in the context manager that deals with sizing buffers for the module, since the logic cares fairly little about what the underlying operations are for, so these typedefs allow for some poor mans' generics...

def _encrypt(self, data,
mechanism=None, mechanism_param=None):

cdef class KeyOperation(OperationContext):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This context manager deals with cryptographic operations and the associated buffer management etc.

# cancel the operation if still active
# This is a PKCS#11 3.x feature
with nogil:
retval = self.op_init(self.session.handle, NULL, self.key)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit of a shame that this very elegant way of cancelling operations is only available in modules that support version 3.x...

@requires(pkcs11.Mechanism.AES_KEY_GEN, pkcs11.Mechanism.AES_CBC_PAD)
# Ideally deleting iterator #1 would terminate the operation, but it
# currently does not.
@unittest.expectedFailure
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test actually works now :)

@MatthiasValvekens MatthiasValvekens force-pushed the feature/internal-state-systematisation branch from f339867 to 89f11a9 Compare June 24, 2025 17:29
 - Needed to get rid of global funclist pointer to make this work.
 - This required the ability to pass around a C-level
   pointer between classes (can't do that with Python-level
   attrs), so a significant part of the Cython classes
   were reworked into Cython extension types.
 - Skeletons in types.py no longer hold any attributes or
   init logic.
 - Logic to reuse existing loaded libs when possible was preserved.
@MatthiasValvekens MatthiasValvekens force-pushed the feature/internal-state-systematisation branch from 89f11a9 to cd1aac3 Compare June 26, 2025 20:21
Covers search, encryption/decryption, and sign/verify.
@MatthiasValvekens MatthiasValvekens force-pushed the feature/internal-state-systematisation branch from cd1aac3 to 09e363d Compare June 28, 2025 00:30
@MatthiasValvekens MatthiasValvekens merged commit 09e363d into pyauth:master Jun 28, 2025
16 checks passed
@MatthiasValvekens MatthiasValvekens deleted the feature/internal-state-systematisation branch July 31, 2025 17:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants