Support Client Migration (Client Side) #4218

masa-koz · 2024-03-29T10:21:26Z

Description

This PR adds the support of Client Migration (Client SIde). Fix #1946.

This pull request includes significant changes to the QUIC connection ID management and introduces a new feature for manual connection ID generation for testing purposes. The most important changes include the replacement of QUIC_CID_HASH_ENTRY with QUIC_CID_SLIST_ENTRY to register connection IDs into more than one bindings, the addition of new APIs to add/remove local address in client, and the introduction of a conditional compilation feature for manual connection ID generation.

Added QuicConnOpenNewPath and QuicConnOpenNewPaths are to manage opening new paths for a connection. These functions will be called when Destination Connection ID is available for sending PATH_CHALLENGE.

Testing

I've added two test.

Basic/WithProbePathArgs.ProbePath/*
Basic/WithMigrationArgs.Migration/*

Documentation

In this PR, the following two Connection level Parameters will be added.

QUIC_PARAM_CONN_ADD_LOCAL_ADDRESS
QUIC_PARAM_CONN_REMOVE_LOCAL_ADDRESS

masa-koz · 2024-03-29T10:22:58Z

@microsoft-github-policy-service agree

src/inc/msquic.h

src/inc/quic_platform.h

nibanks · 2024-03-29T12:35:48Z

src/platform/datapath_winkernel.c

@@ -2523,7 +2523,6 @@ CxPlatRecvDataReturn(
        DATAPATH_RX_IO_BLOCK* IoBlock =
            CXPLAT_CONTAINING_RECORD(Datagram, DATAPATH_RX_PACKET, Data)->IoBlock;

-        CXPLAT_DBG_ASSERT(Binding == NULL || Binding == IoBlock->Binding);


When a connection binds more than one bindings (i.e. after adding a new local address on the client side), we cannot assure that the Datagrams in RecvDataChain come from the same binding.

src/test/lib/PathTest.cpp

nibanks · 2024-03-29T12:42:15Z

src/test/lib/PathTest.cpp

+            QUIC_PARAM_CONN_STATISTICS_V2_PLAT,
+            &Size,
+            &Stats));
+    TEST_EQUAL(Stats.RecvDroppedPackets, 0);


How does this line help? Why would the client drop a packet? Also, are there other test conditions we can add here to determine if the new path worked?

The intention is that I would like to check whether the client succeeded in adding the source connection IDs to a newly opened bindings (if it failed, it will drop a packet whose destination is a newly added local address).
And I think that an event notification about the validation of a path is helpful for both testing and real application.

src/test/lib/PathTest.cpp

src/core/binding.h

src/core/path.h

nibanks · 2024-03-29T12:55:08Z

src/core/lookup.c

@@ -746,8 +768,24 @@ QuicLookupAddLocalCid(
            Hash);

    if (ExistingConnection == NULL) {
-        Result =
-            QuicLookupInsertLocalCid(Lookup, Hash, SourceCid, TRUE);
+        QUIC_CID_HASH_ENTRY* CID =


Just a thought, but we might need to start using pools for allocating and freeing all these entries so they don't hurt perf.

src/core/connection.c

nibanks · 2024-03-29T12:58:40Z

src/core/connection.c

+
+        BOOLEAN Collision = FALSE;
+
+        CxPlatDispatchLockAcquire(&MsQuicLib.DatapathLock);


Why do we have to grab a global lock here? That's not great, and definitely could cause perf issues.

I agree with you. I've tried to remove a global lock in masa-koz#5.
In this change, we will replace all the existing source connection IDs if we find the collision in adding a source connection ID to a newly opened bindings.

I`ve committed it in 5da50d7

src/core/connection.c

nibanks · 2024-03-29T13:05:26Z

src/core/connection.c

+        if (!Connection->State.Started) {
+            Status = QUIC_STATUS_INVALID_STATE;
+            break;
+        }


Why not? It seems kind of intuitive to me that a client could create a connection, queue all paths it wants to use and then start it. No?

I agree with you. So, I've also implemented this. masa-koz#4

Now, we can add local address before connection start. And I've added tests for adding/removing address.
2b3c6fd

src/core/connection.c

nibanks · 2024-03-29T13:08:49Z

src/core/connection.c

+
+        QUIC_CID_LIST_ENTRY* NewDestCid = QuicConnGetUnusedDestCid(Connection);
+        if (NewDestCid == NULL) {
+            Status = QUIC_STATUS_INVALID_STATE;


I wonder if we can support a model where we always queue the path (somehow) even if we don't have a CID to use for it, yet...

I've implemented this.
https://github.com/masa-koz/msquic/pull/3/files

Thanks. The hardest part with this change overall will likely be that we need to make sure we have lots of tests for all the different scenarios.

Now, we defer sending a path challange if there is no unused ConnID. So, I've removed QUIC_PARAM_CONN_LOCAL_UNUSED_DEST_CID_COUNT.
And I've modified Basic/WithProbePathArgs.ProbePath for testing wheter a path challenge will be sent when the peer sends a new ConnID.

c6f7e3d

src/core/connection.c

codecov · 2024-03-29T13:14:20Z

Codecov Report

Attention: Patch coverage is 80.97252% with 90 lines in your changes missing coverage. Please review.

Project coverage is 86.71%. Comparing base (bffa118) to head (dda0b00).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
src/core/connection.c	78.76%	69 Missing ⚠️
src/core/configuration.c	28.57%	10 Missing ⚠️
src/core/loss_detection.c	40.00%	3 Missing ⚠️
src/core/binding.c	93.75%	2 Missing ⚠️
src/core/lookup.c	95.12%	2 Missing ⚠️
src/core/path.c	86.66%	2 Missing ⚠️
src/core/crypto.c	88.88%	1 Missing ⚠️
src/core/settings.c	92.85%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4218      +/-   ##
==========================================
- Coverage   87.10%   86.71%   -0.40%     
==========================================
  Files          56       56              
  Lines       17400    17758     +358     
==========================================
+ Hits        15156    15398     +242     
- Misses       2244     2360     +116

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…on in adding a source connection ID to a newly opened bindings

masa-koz · 2024-12-18T14:12:04Z

@anrossi I'm sorry for my TOO late response.
I've added some tests for SetParam(QUIC_CONN_ADD_LOCAL_ADDRESS)/SetParam(QUIC_CONN_REMOVE_LOCAL_ADDRESS) in
2b3c6fd. Currently, not all the cases are covered. I will add them later.

src/core/connection.c

nibanks

A few comments on the test code

src/test/bin/quic_gtest.h

nibanks · 2024-12-20T14:43:43Z

src/test/lib/ApiTest.cpp

+            QuicAddr ClientLocalAddr;
+            TEST_QUIC_SUCCEEDED(Connection.GetLocalAddr(ClientLocalAddr));
+            TEST_QUIC_STATUS(
+                QUIC_STATUS_INVALID_STATE,


Can/should we really block removal of current active path? In the future, removal can/should be in response to the address being removed from the system itself (i.e. Wi-Fi gets disconnected). You can't say 'no, you're not allowed because I'm using it'. So, really, I think we need to allow removal, but if that's the last one, the connection just dies.

I agree with you. How should we handle the challenge? Separate PR?

Ideally I think we need to have it in this PR.

OK. I'll try it.

Now, we can remove a local address that belongs to the active path.

src/test/lib/PathTest.cpp

src/inc/msquic.h

… enabled

…ress collisions

nibanks · 2024-12-27T15:53:07Z

This PR is big, so would it be possible to add more detailed description of what exactly is changed and why?

Copilot reviewed 17 out of 37 changed files in this pull request and generated no comments.

Files not reviewed (20)

src/core/binding.c: Language not supported
src/core/binding.h: Language not supported
src/core/cid.h: Language not supported
src/core/configuration.c: Language not supported
src/core/connection.h: Language not supported
src/core/crypto.c: Language not supported
src/core/inline.c: Language not supported
src/core/library.h: Language not supported
src/core/lookup.c: Language not supported
src/core/lookup.h: Language not supported
src/core/loss_detection.c: Language not supported
src/core/packet_builder.c: Language not supported
src/core/packet_builder.h: Language not supported
src/core/path.c: Language not supported
src/core/path.h: Language not supported
src/core/quicdef.h: Language not supported
src/core/send.c: Language not supported
src/core/settings.c: Language not supported
src/core/settings.h: Language not supported
src/inc/msquic.h: Language not supported

masa-koz · 2024-12-31T07:31:06Z

@nibanks I've updated the description. If any missing, please notify me.

src/core/binding.c

src/core/cid.h

src/core/binding.c

src/core/connection.c

src/core/lookup.c

nibanks · 2025-01-04T19:59:04Z

src/test/lib/ApiTest.cpp

+            QuicAddr ClientLocalAddr;
+            TEST_QUIC_SUCCEEDED(Connection.GetLocalAddr(ClientLocalAddr));
+            TEST_QUIC_STATUS(
+                QUIC_STATUS_INVALID_STATE,


Ideally I think we need to have it in this PR.

…hSetActive

nibanks · 2025-01-09T12:27:25Z

src/core/connection.c

-        Status = QuicConnAddLocalAddress(Connection, (QUIC_ADDR*)Buffer);
+        QUIC_ADDR* LocalAddress = (QUIC_ADDR*)Buffer;
+
+        if (!QuicAddrIsValid(LocalAddress)) {


Personally, I'd have combined this with the other invalid parameter check (on the same line) as the one above. Looks a bit cleaner IMO.

Now, all the checks are done on the same line.

…anagement logic

…arameter

masa-koz · 2025-01-14T01:23:06Z

@nibanks ping

nibanks · 2025-01-14T12:18:05Z

@nibanks ping

I will try to review this week. Thanks for your patience.

…_MAX_IOCTL_FUNC_CODE

…quic into add_active_migration

Add active migration support.

3ca34bb

masa-koz requested a review from a team as a code owner March 29, 2024 10:21