[Unix] Go back to only checking the runtime resource path for swiftrt.o

[finagolfin]

This is made possible by the swift-frontend now setting this instead in swiftlang/swift#79621. More changes incoming...

[finagolfin]

Ready for review along with the required linked frontend pull, which passed the linux CI when tested with this pull (this pull has no effect on Darwin and Windows). Comments should mostly go there, since this is just a consequence of that pull.

[finagolfin]

@swift-ci please test

[finagolfin]

Alright, this is ready to go, a partial revert of #1667, independent of the linked changes in swiftlang/swift#79621. This pull alone passed CI, along with a full Windows toolchain build. I'd like to get this into 6.1 also next.

Some background: the prior change checked the -sdk alone for swiftrt.o, which sometimes works if the -sdk has one and it isn't too different, but usually doesn't. Under the planned new cross-compilation model, all Swift runtime resources like libswiftCore.so will be looked for in -sdk/usr/lib/swift/<os>, but this reverted code only looks for the single file swiftrt.o, which means the runtime libraries are not looked for in the same -sdk directory! This mixing of Swift runtime resources from two different resource directories in the new model may sometimes work, but will often break.

As for the original cross-compilation model referenced in that doc, which is still the one commonly used, this change I'm reverting often breaks builds because -sdk refers to a C/C++ sysroot alone, like the Android NDK, which usually contains no Swift resources. Since the prior change is broken in both cross-compilation models, I'm reverting it and proposing swiftlang/swift#79621 instead to implement it properly for both the original and new model.

@artemcm, please review.

@compnerd, let us know what you think.

[finagolfin]

Ping @artemcm, mind approving this? I'd like to get it into 6.1 next and I don't think Saleem will respond.

Ever since I argued against this change in the linked pull last year, I have occasionally raised the problems that this caused, eg swiftlang/swift#76381, and got no response. I think they were probably experimenting with new SDK layouts at the time, tried this hack, then moved on to other things.

Let's fix this now, otherwise it is going to start breaking 6.1 SDK bundles and toolchain builds on Unix, ie the two main scenarios where an -sdk is now explicitly specified.

[finagolfin]

@etcwilde, mind reviewing this? I suspect I'm not going to get a response from those above, since they approved the original pull that broke this, and I'd like to get this fixed before it finally ships in 6.1.

[etcwilde]

swiftrt.o is associated with the SDK content though so that the sections are set up correctly for the implementation of swift_addNewDSOImage coming from the stdlib.

During the hybrid phase, this means that we should be grabbing swiftrt from the same place that we found libswiftCore (either via the sdk or resource-dir), though for expedience I'm fine with not hooking that exact logic up today.

As a compromise, can we continue to search the SDK first, and if it's not found there, fall back on the resource-dir?

[finagolfin]

"SDK" can mean different things under the different cross-compilation models, so I try to be more specific.

Most compilation does not set the -sdk flag explicitly, so this code is ignored. We primarily set it explicitly in two scenarios:

1. cross-compilation
2. Building the Swift compiler and stdlib itself from source

The problem with this previous Unix change for most cross-compilation SDK bundles is that they use the old model of -sdk having a C/C++ sysroot alone and a separate -resource-dir containing Swift modules/libraries. Despite the bundle pitch doc saying otherwise, both flags are required, because of a separate bug in this swift-driver. This code I'm changing then fails as the -sdk contains only C/C++ files and no swiftrt.o, see swiftlang/swift#76381 for a concrete example.

As for searching both, the problem is it might find a swiftrt.o in -sdk and still fail because it's the wrong version. I'm trying to fix all this more comprehensively for Swift 6.2 in swiftlang/swift#79621, but I'd like to revert this broken swift-driver code first for 6.1, as we probably won't want to rush swiftlang/swift#79621 into 6.1.

[finagolfin]

Ping @etcwilde or @artemcm, I think this should be merged before the 6.1 release, as SDK bundles and building the Swift toolchain itself with Swift 6.1 is going to start breaking on non-Darwin Unix otherwise, as shown in swiftlang/swift#76381. I get these runtime resources from an external -sdk properly in the frontend pull swiftlang/swift#79621 instead, and intend to implement this more completely that way instead.

[finagolfin]

@shahmishal, could you take a look at this, as it's been a couple weeks and I haven't really been able to get a review here? I tried building the 6.1 compiler with a prebuilt 6.1 on linux to see if the change this is reverting broke anything and it didn't, as the compiler build is careful to use clang for linking Swift executables instead of swift-driver, but the moment that changes, this previous change that I'm reverting will break the compiler build. It has already broken cross-compiling with SDKs on my Android CI with the -sdk/-resource-dir flags.

I'm asking you since you're the review manager for linux and I'd like to get this into 6.1 next.

[etcwilde]

@al45tair, I know that this will affect the static linux SDK, do you have feedback?
@compnerd, do you have feedback here? Does this break your SDKs?

As for searching both, the problem is it might find a swiftrt.o in -sdk and still fail because it's the wrong version.

This is true. But we are also moving away from the model where the SDK content is shoved in the compiler resource directory.

@compnerd, you were driving the push to split the system C sysroot flag from the SDK flag. What is missing in
swiftlang/swift#76381? At least at first glance, the compiler invocation is only using the -sdk flag and not the -sysroot flag. I would expect the situation where you're using the C runtimes from the system and a custom runtime to look something like this swiftc -sdk <path to Swift runtimes> -sysroot /.

[finagolfin]

But we are also moving away from the model where the SDK content is shoved in the compiler resource directory.

Essentially nothing has been done to make that move, other than this change looking for a single file alone, which means currently under your new model, the 6.1 compiler always looks in two different directories for Swift runtime resources like swiftrt.o or Swift.swiftmodule when a -sdk is specified, which is a recipe for compilation failures when they don't match versions.

At least at first glance, the compiler invocation is only using the -sdk flag and not the -sysroot flag. I would expect the situation where you're using the C runtimes from the system and a custom runtime to look something like this swiftc -sdk -sysroot /.

Yes, the invocation in swiftlang/swift#76381 is using the current -sdk/-resource-dir model, because the planned new -sdk/-sysroot model doesn't work yet. Particularly, a glaring hole in the new model with the current compiler implementation is that the frontend will not look in -sdk or -sysroot for the Swift stdlib modules! I detailed that for you here a couple weeks ago.

[finagolfin]

Sigh, with all the delays in reviewing this, 6.1 has now shipped with this regression, so we'll have to fix it in a patch release.

[finagolfin]

@swift-ci test

[finagolfin]

@swift-ci test windows

[finagolfin]

Forced another Windows CI run because the prior one was not registering with github, which means it would have been useless for the merge.

[finagolfin]

@etcwilde and @compnerd, this previous change I'm reverting broke compiling Unix SDK bundles using external platform C/C++ SDKs with the -sdk flag, because such platform C/C++ SDKs do not contain a swiftrt.o. Of course, SDK bundle support for using external C/C++ SDKs is half-baked- see swiftlang/swift-package-manager#8584 and the second blocker from this comment by @marcprux- but Marc is in the process of fixing those, and this is the only other major issue he found.

Can we go ahead and get this in now?

[marcprux]

This is the final blocker for being able to have the Swift Android SDK use a completely external NDK sysroot without any gruesome hacks.

Is there hope for getting this PR in? Or, at least, are there any objections or concerns that we can assuage? There were no responses from @al45tair or @compnerd to @etcwilde's inquiry about whether this might affect the existing SDKs — perhaps they could take another look and offer their thoughts?

[finagolfin]

@swift-ci test

[finagolfin]

This issue is breaking tests on the community Android CI:

/home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/bin/swiftc -target aarch64-unknown-linux-android -sdk /home/swiftci/android/ndk/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot -Xclang-linker --target=aarch64-unknown-linux-android21 -tools-directory /home/swiftci/android/ndk/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin -resource-dir /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/lib/swift -module-cache-path /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/swift-test-results/aarch64-unknown-linux-android/clang-module-cache -swift-version 4 -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.1:macOS 15.4, iOS 18.4, watchOS 11.4, tvOS 18.4, visionOS 2.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.1:macOS 15.4, iOS 18.4, watchOS 11.4, tvOS 18.4, visionOS 2.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.2:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.2:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.3:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.3:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftCompatibilitySpan 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2, visionOS 1.1' -Xfrontend -define-availability -Xfrontend 'SwiftCompatibilitySpan 6.2:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/main.swift /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/b.swift /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/c.swift -cxx-interoperability-mode=upcoming-swift -I /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/swift/test/Interop/Cxx/function/Inputs -o /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/artifacts/out
error: link command failed with exit code 1 (use -v to see invocation)
clang++: error: no such file or directory: '/home/swiftci/android/ndk/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/swift/android/aarch64/swiftrt.o

@compnerd, if you think you need this change being reverted for the Android SDKs in your Windows toolchain, can you apply this patch and let us know what error you're seeing?

@etcwilde, there is a bug in this new swift-driver, where it does not detect Swift runtime libraries and this swiftrt.o object file in a -sdk path. The way most remedy this long-standing bug is to explicitly specify the -resource-dir as sdkPath/usr/lib/swift/, but it appears Saleem does not want to do that, whether because he wants to stop using that flag or they have some other new organizational layout in their TBC SDKs.

But by hacking this in explicitly here, he has broken the build of those who do specify both -sdk/-resource-dir flags when cross-compiling to Unix platforms, particularly when done for external C/C++ SDKs that contain no Swift files, eg the Android NDK.

It may be possible to reconcile all this with additional info, as I just asked from Saleem, but we've gotten silence on the matter since it was merged last summer in #1667, despite my repeatedly raising the issue since then.

If someone needs that change, they should be able to explain why. If they cannot or just aren't using these Android SDKs anymore, we should merge this and go back to the way things were.

Pinging @shahmishal too, as this is breaking the upcoming Android SDK bundle with an external NDK, and this pull has received no real review for the last four months.