Ship Swift Atomics 1.2

[lorentey]

This release will technically not contain any API-level changes, but the fix for #88 is essentially a complete rewrite of the package, so it's risky enough to deserve a minor version bump.

I will not be going through the full test matrix for this release.

Candidate commit:
- f10f031 (current HEAD of #102, merge pending)
- 8be2ba9 (current HEAD of #103, merge pending)
- d8a5239 (main as of #102, #103 and #105)
- 1abcb95 (current HEAD of #106)
- 8189a83 (current HEAD of #106)
- 48fad5a (current HEAD of #106)
- fe0fc8b (current HEAD of #106)
- f26da16 (current HEAD of #106)
- 3431b85 (current HEAD of #106)

Diff: 1.1.0...main

• Review CMake configuration for missed source files
• Review diffs for stray changes
• Update copyright year in file headers
• Update README
• Draft release notes
• Announce new release on the forum

Testing progress: (On Mac and Linux, this is done by running Utilities/run-full-tests.sh. This script builds and tests this package using various build systems and build configurations.)

macOS/AS (Including simulator tests for iOS/watchOS/tvOS)

• macOS 14.0, Apple Silicon, Xcode 15.0, Swift main (swift.org developer snapshot as of 2023-09-24)
• macOS 14.0, Apple Silicon, Xcode 15.0, Swift 5.10 (swift.org developer snapshot as of 2023-xx-xx)
• macOS 14.0, Apple Silicon, Xcode 15.0, Swift 5.9
• macOS 14.0, Apple Silicon, Xcode 14.3.1, Swift 5.8.1
• macOS 14.0, Apple Silicon, Xcode 14.2, Swift 5.7.2

macOS/x86_64

• macOS 14.0, x86_64 (Rosetta), Xcode 15.0, Swift main (swift.org developer snapshot as of 2023-xx-xx)
• macOS 14.0, x86_64 (Rosetta), Xcode 15.0, Swift 5.10 (swift.org developer snapshot as of 2023-xx-xx)
• macOS 14.0, x86_64 (Rosetta), Xcode 15.0, Swift 5.9
• macOS 14.0, x86_64 (Rosetta), Xcode 14.3.1, Swift 5.8.1
• macOS 14.0, x86_64 (Rosetta), Xcode 14.2, Swift 5.7.2
• macOS 14.0, x86_64 (Rosetta), Xcode 13.4.1, Swift 5.6.1
• macOS 14.0, x86_64, Xcode 15.0, Swift main (swift.org developer snapshot as of 2023-xx-xx)
• macOS 14.0, x86_64, Xcode 15.0, Swift 5.10 (swift.org developer snapshot as of 2023-xx-xx)
• macOS 14.0, x86_64, Xcode 15.0, Swift 5.9
• macOS 14.0, x86_64, Xcode 14.3.1, Swift 5.8.1
• macOS 14.0, x86_64, Xcode 14.2, Swift 5.7.2

Linux/Aarch64

• Ubuntu 20.04, Aarch64, Swift main (swift.org developer snapshot as of 2023-xx-xx)
• Ubuntu 20.04, Aarch64, Swift 5.10 (swift.org developer snapshot as of 2023-xx-xx)
• Ubuntu 20.04, Aarch64, Swift 5.9 release
• Ubuntu 20.04, Aarch64, Swift 5.8.1 release
• Ubuntu 20.04, Aarch64, Swift 5.7.3 release

Linux/x86_64

• Ubuntu 20.04, x86_64, Swift main (swift.org developer snapshot as of 2023-xx-xx)
• Ubuntu 20.04, x86_64, Swift 5.10 (swift.org developer snapshot as of 2023-xx-xx)
• Ubuntu 20.04, x86_64, Swift 5.9 release
• Ubuntu 20.04, x86_64, Swift 5.8.1 release
• Ubuntu 20.04, x86_64, Swift 5.7.3 release

Windows 10/x86_64

• Windows 10, x86_64, Swift main (swift.org developer snapshot as of 2023-xx-xx)
• Windows 10, x86_64, Swift 5.10 (swift.org developer snapshot as of 2023-xx-xx)
• Windows 10, x86_64, Swift 5.8.1 release
• Windows 10, x86_64, Swift 5.7.3 release

Known failures:

• watchOS simulator fails to build tests on i386 [Xcode 15]
• SPM can't build package with library evolution enabled (can't import _AtomicShims) [Xcode 15]

[lorentey]

Gah, CMakeLists.txt references the .gyb files as sources. This doesn't seem to cause problems, but we should not list them.

[lorentey]

Xcode 15 seems to have trouble linking test targets building for the generic watchOS simulator, which seems to include i386. Ignoring.

Ld /tmp/run-full-tests.sh.A57jS/xcodebuild/Build/Intermediates.noindex/swift-atomics.build/Release-watchsimulator/AtomicsTests.build/Objects-normal/i386/Binary/AtomicsTests normal i386 (in target 'AtomicsTests'
 from project 'swift-atomics')
...
Undefined symbols for architecture i386:
  "_$s6XCTest12XCTAssertNil__4file4lineyypSgyKXK_SSyXKs12StaticStringVSutF", referenced from:
      _$s12AtomicsTests019AtomicLazyReferenceB0C26test_unsafe_create_destroyyyFTo in AtomicLazyReference.o
      _$s12AtomicsTests019AtomicLazyReferenceB0C30test_unsafe_storeIfNilThenLoadyyFTf4d_n in AtomicLazyReference.o
      _$s12AtomicsTests019AtomicLazyReferenceB0C31test_managed_storeIfNilThenLoadyyFTf4d_n in AtomicLazyReference.o
      _$s12AtomicsTests027LockFreeSingleConsumerStackB0C11test_BasicsyyFTf4d_n in LockFreeSingleConsumerStack.o
     (maybe you meant: _$s6XCTest12XCTAssertNil__4file4lineyypSgyKXK_SSyXKs12StaticStringVSutFfA0_SSycfu_)
  "_$s6XCTest13XCTAssertTrue__4file4lineySbyKXK_SSyXKs12StaticStringVSutF", referenced from:
      _$s12AtomicsTests019AtomicLazyReferenceB0C30test_unsafe_storeIfNilThenLoadyyFTf4d_n in AtomicLazyReference.o
      _$s12AtomicsTests019AtomicLazyReferenceB0C31test_managed_storeIfNilThenLoadyyFTf4d_n in AtomicLazyReference.o
      _$s12AtomicsTests015BasicAtomicBoolB0C28test_compareExchange_relaxedyyFTf4d_n in BasicAtomicBoolTests.o
      _$s12AtomicsTests015BasicAtomicBoolB0C30test_compareExchange_acquiringyyFTf4d_n in BasicAtomicBoolTests.o
      _$s12AtomicsTests015BasicAtomicBoolB0C30test_compareExchange_releasingyyFTf4d_n in BasicAtomicBoolTests.o
      _$s12AtomicsTests015BasicAtomicBoolB0C42test_compareExchange_acquiringAndReleasingyyFTf4d_n in BasicAtomicBoolTests.o
      _$s12AtomicsTests015BasicAtomicBoolB0C43test_compareExchange_sequentiallyConsistentyyFTf4d_n in BasicAtomicBoolTests.o
      ...
     (maybe you meant: _$s6XCTest13XCTAssertTrue__4file4lineySbyKXK_SSyXKs12StaticStringVSutFfA0_SSycfu_)
... etc ...

[lorentey]

The xcodebuild.library-evolution build is failing to import the _AtomicShims module on Xcode 15 (iOS). This smells like the known issue with XCBuild's dependency management for C modules.

This package isn't designed to be built with library evolution enabled, so I'm not treating this as a blocker.

[22 xcodebuild.library-evolution] xcodebuild -scheme swift-atomics -destination generic/platform=macOS -destination generic/platform=iOS BUILD_LIBRARY_FOR_DISTRIBUTION=YES

SwiftVerifyEmittedModuleInterface normal arm64 Verifying\ emitted\ module\ interface\ Atomics.private.swiftinterface /Users/klorentey/Library/Developer/Xcode/DerivedData/swift-atomics-fflkxpmrsggtlnbcxorrecorsseo/Build/Intermediates.noindex/swift-atomics.build/Debug/Atomics.build/Objects-normal/arm64/Atomics.private.swiftinterface (in target 'Atomics' from project 'swift-atomics')
...
/Users/klorentey/Library/Developer/Xcode/DerivedData/swift-atomics-fflkxpmrsggtlnbcxorrecorsseo/Build/Intermediates.noindex/swift-atomics.build/Debug/Atomics.build/Objects-normal/arm64/Atomics.private.swiftinterface:8:8: error: no such module '_AtomicsShims'
import _AtomicsShims
       ^

[lorentey]

A wild failure appeared!

[26 xcodebuild.test.macCatalyst] on macOS 14, AS:

Command line invocation:
    /Applications/Xcode-MAS.app/Contents/Developer/usr/bin/xcodebuild -scheme swift-atomics -configuration Release -destination "platform=macOS,variant=Mac Catalyst" -derivedDataPath /tmp/run-full-tests.sh.faoJS/xcodebuild test
[...]
Test Case '-[AtomicsTests.BasicAtomicPointerTests test_weakCompareExchange_acquiringAndReleasing_acquiring]' started.
/Users/klorentey/Docker/swift-atomics/Tests/AtomicsTests/Basics/autogenerated/BasicAtomicPointerTests.swift:1603: error: -[AtomicsTests.BasicAtomicPointerTests test_weakCompareExchange_acquiringAndReleasing_acquiring] : XCTAss
ertTrue failed
/Users/klorentey/Docker/swift-atomics/Tests/AtomicsTests/Basics/autogenerated/BasicAtomicPointerTests.swift:1605: error: -[AtomicsTests.BasicAtomicPointerTests test_weakCompareExchange_acquiringAndReleasing_acquiring] : XCTAss
ertEqual failed: ("0x0000600002183860") is not equal to ("0x0000600002183870")
/Users/klorentey/Docker/swift-atomics/Tests/AtomicsTests/Basics/autogenerated/BasicAtomicPointerTests.swift:1612: error: -[AtomicsTests.BasicAtomicPointerTests test_weakCompareExchange_acquiringAndReleasing_acquiring] : XCTAss
ertFalse failed
/Users/klorentey/Docker/swift-atomics/Tests/AtomicsTests/Basics/autogenerated/BasicAtomicPointerTests.swift:1613: error: -[AtomicsTests.BasicAtomicPointerTests test_weakCompareExchange_acquiringAndReleasing_acquiring] : XCTAss
ertEqual failed: ("0x0000600002183860") is not equal to ("0x0000600002183870")
Test Case '-[AtomicsTests.BasicAtomicPointerTests test_weakCompareExchange_acquiringAndReleasing_acquiring]' failed (0.999 seconds).

The same test is passing on every other configuration I've tried so far. The failure is specific to this one ordering. Huh?

[lorentey]

Oh, d'oh, the weak compare-exchange is allowed to spuriously fail, but the test expects it to unconditionally succeed. The scheduler must have decided to randomly suspend execution of the process in the middle of the LL/SC transaction.

All weakCompareExchange tests need to be adjusted to allow spurious failures. 🙈

[lorentey]

The Swift 5.11 snapshot as of 2023-09-24 doesn't interoperate well with Xcode 15 GM's package integration, so xcodebuild fails to build with "Invalid manifest" errors. This is due to the version mismatch between xcbuild and swiftc/swiftpm; it isn't specific to this codebase.

watchOS builds of the xcodeproj under ./Xcode fail to link due to what looks like a missing arch issue in the toolchain's libraries. Also filing this as due to a version mismatch unrelated to this project.

Otherwise 5.11 seems to work okay on the candidate commit.

[lorentey]

Oops, platform conditionals are broken on 5.8:

[...]/swift-atomics/Sources/Atomics/Primitives/autogenerated/Primitives.native.swift:2113:5: error: unexpected platform condition (expected 'os', 'arch', or 'swift')
#if _pointerBitWidth(_64)
    ^
[...]/swift-atomics/Sources/Atomics/Primitives/autogenerated/Primitives.native.swift:2441:5: error: unexpected platform condition (expected 'os', 'arch', or 'swift')
#if _pointerBitWidth(_64)
    ^
[...]/swift-atomics/Sources/Atomics/Primitives/autogenerated/Primitives.native.swift:2444:9: error: unexpected platform condition (expected 'os', 'arch', or 'swift')
#elseif _pointerBitWidth(_32)
        ^
[...]/swift-atomics/Sources/Atomics/Types/DoubleWord.swift:16:5: error: unexpected platform condition (expected 'os', 'arch', or 'swift')
#if _pointerBitWidth(_32)
    ^
[...]/swift-atomics/Sources/Atomics/Types/DoubleWord.swift:32:9: error: unexpected platform condition (expected 'os', 'arch', or 'swift')
#elseif _pointerBitWidth(_64)
        ^

We do no enable ATOMICS_NATIVE_BUILTINS on 5.8, so evidently this is because of Swift's uniquely frustrating parsing rules. :-P

[lorentey]

d976dd2 fixes the parsing issue on 5.8. A quick retest confirms that this does not appear to have adverse effects on 5.9 & 5.11, so not restarting the process.

[lorentey]

Swift 5.8 fails the -Werror test with valid complaints:

[2/21] Compiling Atomics IntegerOperations.swift
<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "_AtomicsShims.h"
        ^
[...]/swift-atomics/Sources/_AtomicsShims/include/_AtomicsShims.h:215:1: error: incompatible pointer types passing '_Atomic(intptr_t) *' to parameter of type 'intptr_t *' (aka 'long *')
SWIFTATOMIC_DEFINE_INTEGER_TYPE(Int, intptr_t)
^
[...]/swift-atomics/Sources/_AtomicsShims/include/_AtomicsShims.h:207:3: note: expanded from macro 'SWIFTATOMIC_DEFINE_INTEGER_TYPE'
  SWIFTATOMIC_DEFINE_TYPE(swiftType, cType)                            \
  ^
[...]/swift-atomics/Sources/_AtomicsShims/include/_AtomicsShims.h:203:3: note: expanded from macro 'SWIFTATOMIC_DEFINE_TYPE'
  SWIFTATOMIC_CMPXCHG_FNS(strong, swiftType, cType)                    \
  ^
[...]/swift-atomics/Sources/_AtomicsShims/include/_AtomicsShims.h:179:3: note: expanded from macro 'SWIFTATOMIC_CMPXCHG_FNS'
  SWIFTATOMIC_CMPXCHG_FN(kind, swiftType, cType, relaxed, relaxed)      \
  ^
[...]/swift-atomics/Sources/_AtomicsShims/include/_AtomicsShims.h:142:7: note: expanded from macro 'SWIFTATOMIC_CMPXCHG_FN'
      &expected->value,                                                 \
      ^

[lorentey]

Oh hm, it looks like we won't be able to properly unify the two low-level primitive implementations -- they will need to produce different operations. The fix is in commit 48fad5a.

[lorentey]

Swift 5.8 on Intel macOS also has trouble with library evolution enabled:

/Users/klorentey/Library/Developer/Xcode/DerivedData/swift-atomics-dlyapsytwcmzwnderwymccydfvuw/Build/Intermediates.noindex/swift-atomics.build/Debug/Atomics.build/Objects-normal/arm64/Atomics.private.swiftinterface:5:8: error: no such module '_AtomicsShims'
import _AtomicsShims
       ^
/Users/klorentey/Library/Developer/Xcode/DerivedData/swift-atomics-dlyapsytwcmzwnderwymccydfvuw/Build/Intermediates.noindex/swift-atomics.build/Debug/Atomics.build/Objects-normal/arm64/Atomics.private.swiftinterface:1:1: error: failed to verify module interface of 'Atomics' due to the errors above; the textual interface may be broken by project issues or a compiler bug
// swift-interface-format-version: 1.0
^