Add Affinity support for datasource #122
Draft
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds affinity support for datasource.
What is affinity
When multiple threads access the datasource, each thread will get the last returned connection to the pool.
It might happen, if two threads do some load and the connection pool has min-size of 2 that only the first connection will be used.
in other words, the first connection "jumps" from thread 1 to thread 2 and the second connection is idle.
It is very likely that thread 1 and thread 2 are doing completely different workload
From the perspective of the prepared statement caches (maybe even cpu caches), it would be better to serve connection1 to thread 1 and connection 2 to thread 2.
In short words, this is what can be achived with this PR when using Thread::currentThread as affinity provider. There is a test case, that simulates such a workload, which results in a massive increase of the hit rate in PSC.
There are other use cases (in my case DB2 trusted context - PR will follow) where a connection in the pool has to be switched to the correct tenant. So this can be also used to return the best matching connection from the freelist.
How is this implemented
I tried to structure this PR in several commits:
First, I've refactored the java linked list for a custom linked list implementation: 49bfd31
After some renames (The FreeConnectionBuffer should become a ConnectionBuffer, that holds free and busy list) I added this busyList here 84dc7d1
Up to this point I've replaced the slot-based BusyConnectionBuffer also by a linked list implementation.
The "Node" can now "jump" from one list to the other, that means, we have no object creation/gc when retrieveing/returning connections. It does only pointer arithmetic and no findFreeSlot, so all operations are in constant time. Also no "grow" is needed.
Note: The BusyConnectionBuffer was quite good, I tried to figure out, how much this is faster (as we can reuse the node and save the object creation in java.util.linkedList) and I would say it is nearly not measurable. (Profiler shows, most time is spent in locking, but maybe the linked list can now be replaced with a lock free version 😉)
As we have no disadvantages in performance, I hope this is a good replacement for Busy/FreeList
As mentioned above, always returning the first connection in the freeList may not be the best strategy. We have to find the "best" connection in the freeList.
A simple walkthroug may be inefficient, so there is a kind of hashmap of affinity-lists (by default 257)
Each affinity-list is a linkedList that holds the nodes for a certain affinity hash value.
So if currentThread.hashCode() % 257 = 42, the pool will look up in affinity-list 42 and ideally finds the first connection that was served last to this thread. If no connection is found, the last connection in the freeList is used (this can be discussed, as it may prevent the pool from shrinking?)
I've added a schematic explanation in bd1abfe#diff-a5149a78e256760bba8b61d0384dbc3cd5d2ab0ce3c8a72921b02e60c34fa149
Performance
When not using affinity, this implementation is as fast as BusyConnectionBuffer/FreeConnectionBuffer and always returns the first free connection. (it may be theoretical faster, as there are only pointer operations, no grow, no findFreeeSlot and no new objects, but this is not measurable and we are talking here about a few nanoseconds)
When using affinity and you have a workload like in the sample test, you can get a better hit rate in prepared statements.
When you have special use cases like DB2 connection pool, the benefit could be even more