Unix: Use llvm-ar by default

[finagolfin]

Now that llvm-ar is installed by default in the toolchain, swiftlang/swift#62510, and a recent SwiftPM change requires there to be an archiver in the toolchain/PATH, swiftlang/swift-package-manager#5761, use that bundled llvm-ar for all Unix platforms.

This is the Swift translation of swiftlang/swift#62800.

[finagolfin]

@artemcm, please run the CI on this.

[artemcm]

@swift-ci test

[finagolfin]

Windows CI failure is unrelated, all pulls are failing on the Windows CI.

[finagolfin]

@artemcm, now that the CI is working, would you run it again?

[artemcm]

@swift-ci test

[finagolfin]

Rebased, need another CI run.

[rauhul]

@swift-ci test

[finagolfin]

@swift-ci test

[finagolfin]

@swift-ci test macos

[finagolfin]

@swift-ci test

[finagolfin]

@artemcm, can we go ahead and get this in before the upcoming branch? @al45tair once expressed interest in testing this more last summer in the linked Driver pull but hasn't responded since, and we have plenty of data that it is unlikely to break anything as it has been the default Unix archiver for SwiftPM for the last year and a half, swiftlang/swift-package-manager#6829.

If you're okay with merging this, which requires approval from someone like you, please approve and I'll go ahead and merge this and the Driver pull, which doesn't require anything else.

[al45tair]

I'm honestly not sure this is a desirable change. As I observed previously, almost nobody uses ar via SwiftPM, because it's very rarely used to create static libraries, and it isn't true that the ar format is standardised. While most static library formats did indeed base their format on the original UNIX ar format, there are genuinely some significant differences.

I think we should be using the system librarian (ar or lib.exe). I don't think we should be using llvm-ar, except where that is the system librarian.

@compnerd, WDYT?

[finagolfin]

I think we should be using the system librarian (ar or lib.exe). I don't think we should be using llvm-ar, except where that is the system librarian.

@compnerd, WDYT?

I don't see why you think he'd agree with you, considering we switched SwiftPM to default to llvm-ar for non-Darwin Unix based on Saleem's plan and review: see the discussion at swiftlang/swift-package-manager#6829 where I made the change a year and a half ago.

I am skeptical that there are any worthwhile differences in what llvm-ar produces for major platforms today, but if you're so worried about it, what do you suggest we do instead, switch SwiftPM to default to ar then fall back to llvm-ar if no system ar is found, ie the opposite of what it does today?

[finagolfin]

@swift-ci test

[cachemeifyoucan]

I feel like using llvm-ar is totally fine but should we just symlink llvm-ar to ar on platforms what to use that as default instead?

[finagolfin]

Not sure what you mean, as we currently include llvm-ar in the Swift toolchain and SwiftPM defaults to using it for non-Windows/non-Darwin triples, falling back to looking for ar in the PATH if it can't be found somehow, which will essentially never happen since llvm-ar is in the toolchain.

This pull simply changes the compiler to do the same, ie only look for llvm-ar when archiving static libraries for Unix platforms using the Swift compiler itself.

If you want a completely different librarian when invoking SwiftPM, you can always provide an override to SwiftPM using LIBTOOL or AR, as SwiftPM completely ignores this logic in the Swift compiler driver.

As for platforms that want to use ar more directly, they can always symlink llvm-ar to it or do anything they want with their toolchains already.

[cachemeifyoucan]

I mean if we are preferring llvm-ar all the time, we should just symlink llvm-ar to ar in the swift toolchain instead of playing with the logics in swift-driver since user has no control over this. I feel that is the better approach, even for Darwin and Windows (maybe not windows since symlink is not really supported there).

I have no real objection of the function changed to use llvm-ar for creating static archive.

[al45tair]

.a files are not in a standardised format, and llvm-ar does not necessarily create archives that are compatible with the system archiver. We should use the system archiver, unless we know we're using lld. (See Wikipedia)

We should not merge this.

[finagolfin]

Don't worry, I don't try to sneak pulls by people, will look at the underlying format and let you know.

[al45tair]

To be clear, I'm not trying to be awkward here; it is genuinely the case that .a files are not a fully standardised format. Worse, the various .a (and Windows .lib files) share enough basic structure that sometimes incompatible tools will seem to work, but will result in a .a (or .lib) file that is broken and won't work correctly (or it might work for some linkers and not others).

AFAIK the most obvious problem areas involve indices (the thing that either ranlib or the -s option generates), long filenames and filenames containing spaces. That's not the whole of it, mind; AIX is very different (.a files there can hold a mix of static and shared objects, so e.g. there's only libc.a and no libc.so, and to link the C library you link against the .a file, but some things will be dynamically linked and other things statically, depending on the exact content of the .a). I think HPUX had something different again, and doubtless our old friend z/OS will be doing something completely off the wall as usual.