Document that ThinLTO parallelism is limited to the number of bitcode files

[nh2]

The documentation on https://clang.llvm.org/docs/ThinLTO.html says

all transformations, including function importing, occur later when the modules are optimized in fully parallel backends

the ThinLTO link step will launch as many threads in parallel as there are cores

so I had high hopes that ThinLTO would allow to achieve very fast unity builds, because optimisation would be "parallel".

However, testing it, I noticed that there is no pallelism in that case. I observed that the linker only spawns as many threads as there were .o files created during the bitcode-creation steps.

This meant that, contrary to my hopes, ThinLTO can never compile faster than a normal non-ThinLTO build.

Is this expected / as designed, or could it be made faster (e.g. per-function optimisation parallelism)?

If it is as designed, it would be great to document this constraint on that page.

CC @teresajohnson

[nh2]

Similarly, I found that the ThinLTO incremental on-disk build cache has .o file granularity, meaning that it does not build faster than a non-ThinLTO build during incremental rebuilds, if some small part of a file is compiled.

This, too, would be great to document, unless it can be improved.

[Andarwinux]

Surprisingly, fulllto with --lto-partitions can achieve higher parallelism than thinlto when there are fewer object files, even though this reduces performance.

[nh2]

@Andarwinux where is that lto-partitions flag/setting documented? Does it only work on full LTO, not ThinLTO?

[teresajohnson]

Correct, ThinLTO parallelism and caching is at the input file granularity level. It was designed as a faster and lighterweight LTO for large projects linking lots of files (where merging all the IR objects for regular LTO would result in an extremely large merged IR object file). It won't help a unity build. Can I ask why you use unity builds?

I've heard people propose the idea of splitting up input modules into function granularity modules to get better LTO parallelism or caching, but I don't think anyone has done any concrete engineering work, other than this proposal to do something like that for better caching (that I don't believe was ever sent upstream for review): https://www.youtube.com/watch?v=9bQEtJHkDTw

Surprisingly, fulllto with --lto-partitions can achieve higher parallelism than thinlto when there are fewer object files, even though this reduces performance.

Yes, this is currently the only way to get parallelism for a unity build. I'm surprised it reduces performance (compared to full LTO without partitions?), but I've never used it personally as we use ThinLTO.

[teresajohnson]

@Andarwinux where is that lto-partitions flag/setting documented? Does it only work on full LTO, not ThinLTO?

It's mentioned in the lld man page (https://man.archlinux.org/man/extra/lld/ld.lld.1.en#lto-partitions). @pcc might be able to point to more documentation (having trouble finding some via google search). It is full LTO only.

[teresajohnson]

I'll plan to add some wording to https://clang.llvm.org/docs/ThinLTO.html about the parallelism/caching being at the linked object file granularity.

[nh2]

@teresajohnson Thanks for the clarification! The added wording would probably be a timesaver for many.

Can I ask why you use unity builds?

To get faster C++ builds. The nature of C++ generics (templates) is that they compile and optimise the same code 100s of times (e.g. vector<MyProjectsType> once for each .cpp file / Translation Unit) and then the linker throws away all but one of them.

Unity builds solve it because there's only one Translation Unit, so no duplicate work.

In our small (~30k LOC C++) project, the duplicate work amounts to 7x overhead of user time; for larger projects, this factor can be much larger.

The problem with unity builds is that because compiler frontends are inherently single-threaded, it loses all parallelism. Now the idea is:

If we can make the single-threaded part fast (by just generating bitcode without optimisation), have the in-linker optimisations be fully parallel even for a single Translation Unit, then the unity build can be the fastest way to to do full builds.

(Side note: Some of the parallelism can be recovered by doing unity-packing not into a single Translation Unit, but into as many as you have cores. For example, for my project the fastest full-build achievable overall is "unity-pack 4 .cpp files into 1". But that still causes significant of overhead.)

Relatedly, if ThinLTO could do said single-Translation-Unit parallelism and finer-than-Translation-Unit caching, it might also become the fastest way to do incremental non-unity builds (change 1 .cpp file, build). Arguably this use case would not necessarily have to live in the linker, but might perhaps also be implemented in e.g. clang itself, as the only really non-parallelisable thing is probably the C++ parsing that must be serial due to the C preprocessor etc. But ThinLTO currently seems to have the infrastructure already built out out for this concept of "parallise where all of the serial stuff is gone and where we have global information for embarassingly-parallel work".

Hope that makes sense.

[nh2]

this proposal to do something like that for better caching (that I don't believe was ever sent upstream for review): https://www.youtube.com/watch?v=9bQEtJHkDTw

Thanks for the link, this sounds like a good idea to get cross-Translation-Unit caching without having to run the first compile step fully serially.

I found the repo https://github.com/SNSystems/sn-llvm-project-prepo via this mailing list post but the repo has been archived 5 days ago and there's no note on the README on the state (was it discontinued?).

[teresajohnson]

Thanks for the info!

To get faster C++ builds. The nature of C++ generics (templates) is that they compile and optimise the same code 100s of times (e.g. vector once for each .cpp file / Translation Unit) and then the linker throws away all but one of them.

This is one place where ThinLTO can help a non-unity build. While these duplicate copies still need to be compiled through clang and down into bitcode, ThinLTO uses linker info to determine which copy is prevailing and early in the ThinLTO backends tosses the rest (before most of the optimization pipeline and any of code gen).

In theory full LTO should get some of this too, as all copies are merged into one at the start of the LTO link. However, because of the non-scalable nature of the full LTO link, more optimizations are performed during the pre-LTO compile in full LTO mode.

I think the idea though to do function-granularity (or at least sub-file) parallelism is interesting to explore. I personally don't have the bandwidth or sufficient motivation (my internal customers are LTO linking mongo sized targets with many thousands of input files). But I would welcome patches from any interested parties.

[teresajohnson]

I found the repo https://github.com/SNSystems/sn-llvm-project-prepo via this mailing list post but the repo has been archived 5 days ago and there's no note on the README on the state (was it discontinued?).

I believe the author stopped working on this when they left Sony / SN Systems.