test: Update cross-client testing matrix based on compatibility checkpoints

CompatibilityCheckpoints.md

@@ -28,7 +28,7 @@ checkpoints are designated.
 | CC-1 | `>=1.4.0 <2.0.0 \| 2.0.0-internal* \| 2.0.0-rc*` | 2024-04-09 | CC-1, CC-2, CC-3, CC-4 | `>=1.4.0 <2.140.0`(estimated)` \| 2.0.0-internal* \| 2.0.0-rc*` |
 | CC-2 | `>=2.0.0 <2.60.0` | 2024-06-26 | CC-1, CC-2, CC-3, CC-4, CC-5 | `>=1.4.0 <2.180.0`(estimated)` \| 2.0.0-internal* \| 2.0.0-rc*` |
 | CC-3 | `>=2.60.0 <2.100.0` | 2025-09-02 | CC-1, CC-2, CC-3, CC-4, CC-5, CC-6 | `>=1.4.0 <2.220.0`(estimated)` \| 2.0.0-internal* \| 2.0.0-rc*` |
-| CC-4 | `>=2.100.0 <2.140.0` (limit TBD) | ~2026-04-27 | CC-1, CC-2, CC-3, CC-4, CC-5, CC-6, CC-7 | `>=1.4.0 <2.260.0`(estimated)` \| 2.0.0-internal* \| 2.0.0-rc*` |
+| CC-4 | `>=2.100.0 <2.140.0` (limit TBD) | 2026-04-28 | CC-1, CC-2, CC-3, CC-4, CC-5, CC-6, CC-7 | `>=1.4.0 <2.260.0`(estimated)` \| 2.0.0-internal* \| 2.0.0-rc*` |
 | CC-5 (TBD) | `>=2.140.0 <2.180.0` | ~2026-12-07 | CC-2, CC-3, CC-4, CC-5, CC-6, CC-7, CC-8 | `>=2.0.0 <2.300.0`(estimated) |
 | CC-6 (TBD) | `>=2.180.0 <2.220.0` | ~2027-07-19 | CC-3, CC-4, CC-5, CC-6, CC-7, CC-8, CC-9 | `>=2.70.0 <2.340.0`(estimated) |
 | CC-7 (TBD) | `>=2.220.0 <2.260.0` | ~2028-02-28 | CC-4, CC-5, CC-6, CC-7, CC-8, CC-9, CC-10 | `>=2.100.0 <2.380.0`(estimated) |

CrossClientCompatibilityDevGuide.md

@@ -197,7 +197,12 @@ Designation is the act of officially marking a Fluid Framework release as a new
 
 A new checkpoint should be designated no less than 6 months after the previous one. It should also land on a new major or beta boundary (e.g., `3.0.0`, `2.100.0`), so the prior checkpoint's range can extend cleanly to the new boundary.
 
-**To designate a new checkpoint:** update the [Compatibility Checkpoints](./CompatibilityCheckpoints.md) document and include a changeset noting the new boundary so it appears in the release notes.
+**To designate a new checkpoint:**
+
+1. Update the table in [`CompatibilityCheckpoints.md`](./CompatibilityCheckpoints.md).
+2. Update `checkpoints` in [`packages/test/test-version-utils/src/checkpoints.ts`](./packages/test/test-version-utils/src/checkpoints.ts).
+3. From `packages/test/test-version-utils`, run `pnpm run update-compat-versions` to update the installed versions used by e2e tests.
+4. Include a changeset noting the new boundary so it appears in the release notes.
 
 ## Tightening Runtime Enforcement
 
@@ -219,9 +224,10 @@ To tighten runtime enforcement:
    (e.g., `2.x` → `3.x`), also narrow the `MinimumVersionForCollab` type in
    [compatibilityDefinitions.ts](./packages/runtime/runtime-definitions/src/compatibilityDefinitions.ts)
    to drop the now-unsupported major from its definition.
-3. **Update the e2e test matrix:** The `FullCompat` version matrix is derived
-   from the currently supported checkpoints — update it so tests only run
-   against versions within the new window.
+3. **Update the e2e test matrix:** The `FullCompat` and cross-client matrices
+   are derived from the in-repo Compatibility Checkpoints data and follow
+   whichever checkpoints are currently in the supported window automatically;
+   see [Designating a New Compatibility Checkpoint](#designating-a-new-compatibility-checkpoint).
 
 ## Testing
 
@@ -230,25 +236,22 @@ variations using `describeCompat()` with `"FullCompat"`. The variations test
 cross-client compatibility scenarios by using one version of the Fluid runtime for
 creating containers and a different version for loading containers.
 
-**Example:** A test may generate the following variations for cross-client
-compatibility scenarios:
-
-> **Note:** The version labels below (e.g., "N-1 fast train") reflect the current
-> test infrastructure naming. These labels will be updated to reflect
-> checkpoint-based versioning as part of the checkpoint adoption work.
+**Example:** With current build `2.101.0` (N) and in-window prior Compatibility
+Checkpoints `2.60.0` (CC-3), `2.0.9` (CC-2), and `1.4.0` (CC-1), a SharedCell
+test generates these cross-client variations:
 
 ```
-compat cross-client - create with 2.43.0 (N) + load with 2.33.2 (N-1 fast train)
+compat cross-client - create with 2.101.0 (N) + load with 2.60.0 (CC-3)
   ✔ Example test
-compat cross-client - create with 2.43.0 (N) + load with 2.23.0 (N-2 fast train)
+compat cross-client - create with 2.101.0 (N) + load with 2.0.9 (CC-2)
   ✔ Example test
-compat cross-client - create with 2.43.0 (N) + load with 1.4.0 (N-1 slow train/LTS)
+compat cross-client - create with 2.101.0 (N) + load with 1.4.0 (CC-1)
   ✔ Example test
-compat cross-client - create with 2.33.2 (N-1 fast train) + load with 2.43.0 (N)
+compat cross-client - create with 2.60.0 (CC-3) + load with 2.101.0 (N)
   ✔ Example test
-compat cross-client - create with 2.23.0 (N-2 fast train) + load with 2.43.0 (N)
+compat cross-client - create with 2.0.9 (CC-2) + load with 2.101.0 (N)
   ✔ Example test
-compat cross-client - create with 1.4.0 (N-1 slow train/LTS) + load with 2.43.0 (N)
+compat cross-client - create with 1.4.0 (CC-1) + load with 2.101.0 (N)
   ✔ Example test
 ```
 

packages/test/test-version-utils/README.md

@@ -45,14 +45,11 @@ are generated (empty entries are current versions):
 In addition to the layer version combinations seen above, this package also provides functions to generate variations
 intended to test all layers of one version against all layers of another version in tests that feature more than one client.
 The intention is to simulate scenarios where the client that created a document was using a different version than the client
-loading the document. These variations are applied in our cross-client tests where we test the current version against the
-most recent **public** release.
+loading the document. The matrix pairs the current build against every in-window prior **Compatibility Checkpoint**
+(see [`CompatibilityCheckpoints.md`](../../../CompatibilityCheckpoints.md)) in both directions, with each prior
+checkpoint resolved to the **earliest minor** in its range (e.g. CC-3 → latest patch of `2.60.x`).
 
-For example, at the time of writing, main is on version `2.0.0-internal.7.3.0` and the latest **public** release is `1.3.7`.
-Therefore, we would test the following combinations:
-
--   Client A is running `2.0.0-internal.7.3.0` across **all** layers and Client B is running `1.3.7` across **all** layers.
--   Client A is running `1.3.7` across **all** layers and Client B is running `2.0.0-internal.7.3.0` across **all** layers.
+The data driving the matrix lives in [`src/checkpoints.ts`](./src/checkpoints.ts).
 
 ### Mocha test setup with layer version combinations
 
@@ -74,8 +71,9 @@ to enable compat testing easily in the future just by changing the compatVersion
 
 ### Legacy version defaults and installation
 
-By default, N-1 (public release), N-1 (internal release), N-2 (internal release), and LTS (hard coded) test variants are
-generated. The versions can be specified using command line (see below) to run the test against any two versions. This
+By default, the cross-client checkpoint matrix described above is generated, a layer-compat permutation against
+the most recent prior in-window checkpoint, and a hard-coded "Oldest Compatible Version" (OCV) for Loader / Driver
+layer-compat. The versions can be specified using command line (see below) to run the test against any two versions. This
 package includes a `mocha` global hook that will install legacy packages at the beginning of the package based on the
 `compatVersion` settings.
 
@@ -148,7 +146,8 @@ default set of legacy versions whether it is necessary or not.
 
 ## Updating compat versions
 
-After a Fluid Framework version bump, run from this package's directory:
+After a Fluid Framework version bump or after a new compatibility checkpoint is designated, run from this package's
+directory:
 
 ```
 pnpm run update-compat-versions
@@ -157,7 +156,8 @@ pnpm run update-compat-versions
 The script (`scripts/updateCompatVersions.ts`) does the following:
 
 1. Reads the current package version from `src/packageVersion.ts`.
-2. Queries the npm registry to resolve N-1, N-2, OCV, cross-client, and full back-compat versions.
+2. Maps that version to a checkpoint via [`src/checkpoints.ts`](./src/checkpoints.ts) and queries the npm registry to
+   resolve every in-window prior checkpoint, plus the full back-compat versions, to exact versions.
 3. Writes `compat-workspaces/generated-versions.cjs` with the resolved exact versions.
 4. Creates or updates per-version `package.json` files in `compat-workspaces/full/`.
 5. Removes version directories that are no longer needed.

packages/test/test-version-utils/compat-workspaces/full/2.0.9/package.json

@@ -0,0 +1,33 @@
+{
+  "name": "compat-2.0.9",
+  "version": "1.0.0",
+  "private": true,
+  "homepage": "https://fluidframework.com",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/microsoft/FluidFramework.git",
+    "directory": "packages/test/test-version-utils/compat-workspaces/full/2.0.9"
+  },
+  "license": "MIT",
+  "author": "Microsoft and contributors",
+  "dependencies": {
+    "@fluid-experimental/sequence-deprecated": "2.0.9",
+    "@fluidframework/agent-scheduler": "2.0.9",
+    "@fluidframework/aqueduct": "2.0.9",
+    "@fluidframework/cell": "2.0.9",
+    "@fluidframework/container-loader": "2.0.9",
+    "@fluidframework/container-runtime": "2.0.9",
+    "@fluidframework/counter": "2.0.9",
+    "@fluidframework/datastore": "2.0.9",
+    "@fluidframework/local-driver": "2.0.9",
+    "@fluidframework/map": "2.0.9",
+    "@fluidframework/matrix": "2.0.9",
+    "@fluidframework/odsp-driver": "2.0.9",
+    "@fluidframework/ordered-collection": "2.0.9",
+    "@fluidframework/register-collection": "2.0.9",
+    "@fluidframework/routerlicious-driver": "2.0.9",
+    "@fluidframework/sequence": "2.0.9",
+    "@fluidframework/test-utils": "2.0.9",
+    "@fluidframework/tree": "2.0.9"
+  }
+}

packages/test/test-version-utils/compat-workspaces/full/2.60.0/package.json

@@ -0,0 +1,33 @@
+{
+  "name": "compat-2.60.0",
+  "version": "1.0.0",
+  "private": true,
+  "homepage": "https://fluidframework.com",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/microsoft/FluidFramework.git",
+    "directory": "packages/test/test-version-utils/compat-workspaces/full/2.60.0"
+  },
+  "license": "MIT",
+  "author": "Microsoft and contributors",
+  "dependencies": {
+    "@fluid-experimental/sequence-deprecated": "2.60.0",
+    "@fluidframework/agent-scheduler": "2.60.0",
+    "@fluidframework/aqueduct": "2.60.0",
+    "@fluidframework/cell": "2.60.0",
+    "@fluidframework/container-loader": "2.60.0",
+    "@fluidframework/container-runtime": "2.60.0",
+    "@fluidframework/counter": "2.60.0",
+    "@fluidframework/datastore": "2.60.0",
+    "@fluidframework/local-driver": "2.60.0",
+    "@fluidframework/map": "2.60.0",
+    "@fluidframework/matrix": "2.60.0",
+    "@fluidframework/odsp-driver": "2.60.0",
+    "@fluidframework/ordered-collection": "2.60.0",
+    "@fluidframework/register-collection": "2.60.0",
+    "@fluidframework/routerlicious-driver": "2.60.0",
+    "@fluidframework/sequence": "2.60.0",
+    "@fluidframework/test-utils": "2.60.0",
+    "@fluidframework/tree": "2.60.0"
+  }
+}