Repository navigation
[Caching] Reduce the cost of computing compile job cache keys - #2229
Merged
cachemeifyoucan merged 2 commits intoOct 15, 2026
Merged
Conversation
Support the new `ArgumentIsCASID` flag from swift options, which labels the options whose argument is a CAS ID when compilation caching is enabled, and update the options that have the flag. rdar://189417668
Use `swiftscan_cache_compute_keys_from_input_indices` when available to compute the cache keys for all the output producing inputs of a compile job at once, so the command-line is only processed once for a batch job. Fall back to computing the key for each input otherwise. rdar://189417668
cachemeifyoucan
requested review from
DougGregor,
artemcm,
compnerd and
owenv
as code owners
October 8, 2026 17:20
Contributor
Author
|
@swift-ci please test |
Contributor
Author
|
@swift-ci please test windows platform |
Contributor
Author
|
Windows test doesn't work until swift change is landed due to the new makeOption added. |
owenv
approved these changes
Oct 12, 2026
Contributor
Author
|
@swift-ci please test windows platform |
cachemeifyoucan
enabled auto-merge
October 13, 2026 16:15
Contributor
Author
|
@swift-ci please test windows platform |
1 similar comment
Contributor
Author
|
@swift-ci please test windows platform |
|
@swift-ci please test windows |
1 similar comment
Contributor
Author
|
@swift-ci please test windows |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Computing the compilation cache keys was expensive for large modules in
batch mode. The base cache key stored the entire command-line, including
the content of the file list, as a single blob that was different for
every job and every build, so each key stored a large new blob in the
CAS. The driver also computed the base key again for every output
producing input of a batch job.
This changes the following in swift and swift-driver:
the jobs in a module, the clang arguments, and the input list, each
stored as a blob that can be shared between keys. The primary inputs
are stored in the base key node, and the arguments of the options
labeled with the new
ArgumentIsCASIDflag are added as referencesto the CAS objects instead of strings, so the new objects for each
job are small. A labeled CAS ID that is not in the CAS is an error.
and add
createCASFileSystemFromCacheKeyto replaceiterateCommandLine, which is used by LLDB.swiftscan_cache_compute_keys_from_input_indicesto compute thecache keys for multiple inputs in one call, and use it in the driver
when available.
ArgumentIsCASIDflag in swift-driver's makeOptions.In a synthetic module with 1000 files, the CAS growth for a one line
edit goes from 86 MB to 0.7 MB when the driver computes the keys for
all the files. In a module with 2653 files, computing the cache keys
while planning a clean build goes from about 2.5s to 0.2s.
The driver change to skip the cache keys for the jobs that are batched
later is in #2227.
rdar://189417668
swift: swiftlang/swift#93076
llvm-project: swiftlang/llvm-project#14603