Turbopack: add support for selective reads of keyed cell values

[sokra]

What?

Previously cells could either be unchanged or changed. This falls short when updating Maps or Sets.
We can't have a dependency on a single key of a map or set.
The workaround for this was to create an "selector" turbo-tasks which picks the value and copies it into a new cell.
This allows more granular invalidation.

But it also creates a lot of extra tasks. And changing the map/set does invalidate all these selector tasks, which causes a lot of invalidation work.

This change adds support for dependencies on certain keys of cells.

On write side a turbo-tasks value need to opt-in into that behavior via cell = "keyed".
This changes the cell method to compare the new value key-by-key and report the changed keys to the backend.

On read side the Vc need to be read via .get(key).await? or .contains_key(key).await?, which returns only the value resp. existance of the key and adds a dependency only to that key.

With this approach the "selector" tasks can be avoided, while maintaining the same level of invalidation.

How?

On implementation side we only track the hash of the key for size reasons.
A dependency has an additional field key: Option<u64> which represents the key depending on.

When updating a cell we have a new optional argument updated_key_hashes: Option<SmallVec<[u64; 2]>> where the updates keys can be provided. The new CellMode (cell = "keyed") fills that.

[sokra]

• Turbopack: refactor data storage to avoid reverse task cache #88492
• Turbopack: avoid chunk_item_id task and use selective cell access instead #88411
• Turbopack: follow tracing spans through aggregation update queue #88410
• Consistency for compute_async_module_info #88409
• Turbopack: snapshot the ModuleGraph to avoid (eventual) inconsistent graphs #88340
• Turbopack: avoid removing cells for erroring tasks #88305
• Turbopack: parallelize making dependent tasks dirty #86755
• read task cell should only need Data #88339
• Turbopack: run tasks with priority based on distance to leaf #87662
• Turbopack: replace is_reference_used with unused_references set #88304
• Turbopack: add support for selective reads of keyed cell values #88303 👈 (View in Graphite)
• canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

[nextjs-bot]

Allow CI Workflow Run

• approve CI run for commit: 1122fbd

Note: this should only be enabled once the PR is ready to go and can only be enabled by a maintainer

[nextjs-bot]

Stats from current PR

✅ No significant changes detected

📊 All Metrics

📖 Metrics Glossary

Dev Server Metrics:

• Listen = TCP port starts accepting connections
• First Request = HTTP server returns successful response
• Cold = Fresh build (no cache)
• Warm = With cached build artifacts

Build Metrics:

• Fresh = Clean build (no .next directory)
• Cached = With existing .next directory

Change Thresholds:

• Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
• Size: Changes < 1KB AND < 1% are insignificant
• All other changes are flagged to catch regressions

⚡ Dev Server

Metric	Canary	PR	Change	Trend
Cold (Listen)	455ms	456ms	✓	▁▁▁█▁
Cold (Ready in log)	440ms	439ms	✓	▂▁▁█▁
Cold (First Request)	1.129s	1.157s	✓	▄▁▂█▁
Warm (Listen)	456ms	457ms	✓	▁▁▁█▁
Warm (Ready in log)	449ms	445ms	✓	▁▁▁█▁
Warm (First Request)	348ms	341ms	✓	▂▁▁█▂

📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric	Canary	PR	Change	Trend
Cold (Listen)	455ms	455ms	✓	▅▁▁▁▁
Cold (Ready in log)	439ms	440ms	✓	▆▇▅▆▆
Cold (First Request)	1.833s	1.849s	✓	▇█▆▇█
Warm (Listen)	456ms	456ms	✓	█████
Warm (Ready in log)	438ms	438ms	✓	▇█▆██
Warm (First Request)	1.837s	1.855s	✓	▆▇▅▇▇

⚡ Production Builds

Metric	Canary	PR	Change	Trend
Fresh Build	4.137s	4.118s	✓	▁▁▂█▁
Cached Build	4.098s	4.110s	✓	▁▁▂█▁

📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric	Canary	PR	Change	Trend
Fresh Build	14.051s	14.069s	✓	▃▃▁▃▂
Cached Build	14.232s	14.254s	✓	▃▃▁▃▃
node_modules Size	458 MB	458 MB	✓	▁▁▁▁▁

📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles: **431 kB** → **431 kB** ✅ -7 B

82 files with content-based hashes (individual files not comparable between builds)

Server

Middleware

	Canary	PR	Change
middleware-b..fest.js gzip	789 B	792 B	✓
Total	789 B	792 B	⚠️ +3 B

Build Details

Build Manifests

	Canary	PR	Change
_buildManifest.js gzip	450 B	450 B	✓
Total	450 B	450 B	✓

📦 Webpack

Client

Main Bundles

	Canary	PR	Change
2086.HASH.js gzip	169 B	N/A	-
2161-HASH.js gzip	5.41 kB	N/A	-
2747-HASH.js gzip	4.48 kB	N/A	-
4322-HASH.js gzip	52.8 kB	N/A	-
ec793fe8-HASH.js gzip	62.3 kB	N/A	-
framework-HASH.js gzip	59.8 kB	59.8 kB	✓
main-app-HASH.js gzip	251 B	254 B	🔴 +3 B (+1%)
main-HASH.js gzip	38.6 kB	39 kB	✓
webpack-HASH.js gzip	1.68 kB	1.68 kB	✓
1596.HASH.js gzip	N/A	169 B	-
2658-HASH.js gzip	N/A	52.7 kB	-
6349-HASH.js gzip	N/A	4.46 kB	-
7019-HASH.js gzip	N/A	5.43 kB	-
b17a3386-HASH.js gzip	N/A	62.3 kB	-
Total	225 kB	226 kB	⚠️ +245 B

Polyfills

	Canary	PR	Change
polyfills-HASH.js gzip	39.4 kB	39.4 kB	✓
Total	39.4 kB	39.4 kB	✓

Pages

	Canary	PR	Change
_app-HASH.js gzip	194 B	193 B	✓
_error-HASH.js gzip	182 B	182 B	✓
css-HASH.js gzip	336 B	335 B	✓
dynamic-HASH.js gzip	1.8 kB	1.8 kB	✓
edge-ssr-HASH.js gzip	256 B	256 B	✓
head-HASH.js gzip	352 B	349 B	✓
hooks-HASH.js gzip	385 B	384 B	✓
image-HASH.js gzip	580 B	580 B	✓
index-HASH.js gzip	259 B	258 B	✓
link-HASH.js gzip	2.5 kB	2.51 kB	✓
routerDirect..HASH.js gzip	319 B	317 B	✓
script-HASH.js gzip	385 B	387 B	✓
withRouter-HASH.js gzip	316 B	315 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Total	7.97 kB	7.96 kB	✅ -8 B

Server

Edge SSR

	Canary	PR	Change
edge-ssr.js gzip	125 kB	125 kB	✓
page.js gzip	242 kB	242 kB	✓
Total	366 kB	367 kB	⚠️ +478 B

Middleware

	Canary	PR	Change
middleware-b..fest.js gzip	652 B	652 B	✓
middleware-r..fest.js gzip	155 B	156 B	✓
middleware.js gzip	33.1 kB	33.3 kB	✓
edge-runtime..pack.js gzip	842 B	842 B	✓
Total	34.7 kB	34.9 kB	⚠️ +205 B

Build Details

Build Manifests

	Canary	PR	Change
_buildManifest.js gzip	738 B	738 B	✓
Total	738 B	738 B	✓

Build Cache

	Canary	PR	Change
0.pack gzip	3.67 MB	3.67 MB	🔴 +7.95 kB (+0%)
index.pack gzip	98.1 kB	101 kB	🔴 +2.58 kB (+3%)
index.pack.old gzip	100 kB	101 kB	✓
Total	3.86 MB	3.87 MB	⚠️ +11.5 kB

🔄 Shared (bundler-independent)

Runtimes

	Canary	PR	Change
app-page-exp...dev.js gzip	304 kB	304 kB	✓
app-page-exp..prod.js gzip	158 kB	158 kB	✓
app-page-tur...dev.js gzip	304 kB	304 kB	✓
app-page-tur..prod.js gzip	158 kB	158 kB	✓
app-page-tur...dev.js gzip	300 kB	300 kB	✓
app-page-tur..prod.js gzip	156 kB	156 kB	✓
app-page.run...dev.js gzip	301 kB	301 kB	✓
app-page.run..prod.js gzip	156 kB	156 kB	✓
app-route-ex...dev.js gzip	68.8 kB	68.8 kB	✓
app-route-ex..prod.js gzip	47.6 kB	47.6 kB	✓
app-route-tu...dev.js gzip	68.8 kB	68.8 kB	✓
app-route-tu..prod.js gzip	47.6 kB	47.6 kB	✓
app-route-tu...dev.js gzip	68.4 kB	68.4 kB	✓
app-route-tu..prod.js gzip	47.4 kB	47.4 kB	✓
app-route.ru...dev.js gzip	68.4 kB	68.4 kB	✓
app-route.ru..prod.js gzip	47.3 kB	47.3 kB	✓
dist_client_...dev.js gzip	324 B	324 B	✓
dist_client_...dev.js gzip	326 B	326 B	✓
dist_client_...dev.js gzip	318 B	318 B	✓
dist_client_...dev.js gzip	317 B	317 B	✓
pages-api-tu...dev.js gzip	41.2 kB	41.2 kB	✓
pages-api-tu..prod.js gzip	31.3 kB	31.3 kB	✓
pages-api.ru...dev.js gzip	41.1 kB	41.1 kB	✓
pages-api.ru..prod.js gzip	31.2 kB	31.2 kB	✓
pages-turbo....dev.js gzip	50.8 kB	50.8 kB	✓
pages-turbo...prod.js gzip	38.2 kB	38.2 kB	✓
pages.runtim...dev.js gzip	50.7 kB	50.7 kB	✓
pages.runtim..prod.js gzip	38.2 kB	38.2 kB	✓
server.runti..prod.js gzip	62.2 kB	62.2 kB	✓
Total	2.69 MB	2.69 MB	✅ -1 B

[codspeed-hq]

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 3 skipped benchmarks¹

---

_{Comparing sokra/selective-invalidation (c151034) with sokra/eventual-consistency-no-remove-cells (0494417)}

Footnotes

1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩