Add AST representation for coroutines #85372

asl · 2025-11-06T22:38:05Z

This is a proof-of-concept PR adding AST representation for coroutines

Further background at https://forums.swift.org/t/pitch-yield-once-functions-first-class-coroutines/77081

asl · 2025-11-06T22:40:19Z

@tkremenek @rjmccall Following discussions on LLVM Dev Meeting, this is a stripped-down version of #78508:

No autodiff-related parts
No silgen (besides the "standard" special path used for coroutine accessors)
Some other fixes for SIL optimizer is omitted

Essentially the PR contains AST-related bits for coroutine declarations and coroutine function types as well as required boilerplate for AST mangling, dumping, serialization, etc.

@xedin comments from the original PR were addressed as well.

asl · 2025-11-06T23:42:03Z

Please test with following pull request: swiftlang/llvm-project#11396
@swift-ci please test

asl · 2025-11-13T07:02:13Z

Please test with following pull request: swiftlang/llvm-project#11396
@swift-ci please test

asl · 2025-11-13T20:34:42Z

Please test with following pull request: swiftlang/llvm-project#11396
@swift-ci please test

asl · 2025-11-14T08:02:00Z

Please test with following pull request: swiftlang/llvm-project#11396
@swift-ci please test windows platform

rjmccall · 2025-11-17T19:18:05Z

include/swift/AST/Types.h

+  static bool classof(const TypeBase *T) {
+    return T->getKind() == TypeKind::YieldResult;
+  }
+};


Alright, this not how I would suggest designing this; I think you need function types to have a list of yields, basically like its parameter list.

Right. But this is what I wanted to avoid ... as this would require the changes over all the codebase wrt types and typechecking. Is there a way to overcome this :) ?

Well, if we were really doing this properly, we'd need basically those exact same changes anyway. Everywhere that's calling the "without yields" function is arguably wrong and is just skating by because we aren't really supporting this as a feature in most of the compiler. That's why this is a hugely invasive change that I'm really concerned about.

@slavapestov, what do you think?

I'm not a fan of introducing a YieldResultType and modeling a function's results as a tuple where some elements are yields; please let's not do that.

I think adding a list of yield results to AnyFunctionType is the way to go. There would be a fair amount of churn since we take function types apart into parameters/results/extInfo and then re-assemble them in various places as you've noted, but I think the changes should be mechanical for the most part, and it's important for the long-term health of the code base to model these fundamental concepts properly.

As a side benefit, the SIL lowering logic for constructing SILFunctionTypes for coroutine accessors could be simplified as a result of this change, because now you will have all of the relevant information in one place instead of having to "assemble" the SIL function type from two distinct pieces of information.

(There is also some precedent for this change. We already did a similar refactoring when we introduced AnyFunctionType's parameter list in the first place -- prior to Swift 5, a function type's "input type" used to be just a single type, which could then be a tuple type to simulate multiple parameters (!). Untangling this was somewhat painful, but not horribly painful.)

As a side benefit, the SIL lowering logic for constructing SILFunctionTypes for coroutine accessors could be simplified as a result of this change, because now you will have all of the relevant information in one place instead of having to "assemble" the SIL function type from two distinct pieces of information.

Yes, I already have this code in #78508 and it is generic for accessors and non-accessor coroutines.

@slavapestov @rjmccall I think I'm having the question wrt syntax. In SIL we parse yields as a part of result type:

$@yield_once @convention(method) (@inout S) -> @yields @inout Float // Single yield

$@yield_once @convention(method) (@inout S) -> (@yields @inout Float, Float) // Yield and result

And then we're having a special rule in type resolver:

If result type is a tuple type with labels it is resolved as tuple type

Otherwise it is resolved component-wise

What do we want to have at Swift level? Same syntax or something else (e.g. make yields a context-sensititve keyword and parse source after -> in a special way allowing, e.g. -> yields Foo, Float? I think multiple yields should not be tuple, but rather separate ones as potentially we might have a mixture of indirect and direct yields and something like func foo() -> yields (inout T, U) looks worse than func foo() -> yields inout T, yields U.

What are your preferences here?

That's a great question. Ultimately, we're not actually committing to a permanent syntax here — if we added this to the language for real, it would need an evolution proposal, but this is just for testing and the representational convenience of the frontend, right? So I think we should prioritize not interfering too much with normal parsing and type-checking, and if that leads us to some very hacky syntax (maybe even an attribute — @yields(inout T, U)?), that's okay.

@rjmccall I already checked how the things to be organized both in C++ and Swift parser. And it seems the cleanest solution that would not interfere much is actually to do similar to thrown error type. The reason is a bit "fun": C++ and Swift parsers do parsing of result type / clause a bit different, essentially for new Swift parser arrow is a part of syntax and I really do not want to interfere with the existing ResultClause thing and do "parsing in the middle" as this would require also changes in roundtrip text emission that I am trying to avoid. Second thing is that current parsers expect inout in front of any attributed type (so inout @yields T would parse as-is, but @yields inout T would require parser change in common place). But for inout @yields T we will later have issues around type resolution as we'd need to special case non-type @yields attribute.

That said, I will plugin-in into existing effects specifier parsing. It is already generic enough as it supports all kinds of async and rethrows and throws combination, so adding another yield there seems most straightforward in terms of amount of required parser and type resolution changes.

So, we'll have for now (with yields being another context-specific keyword):

func foo() yields T -> ()

I'm already having a prototype that could be used as a start.

Okay, that sounds fine. And this will only be accepted under an experimental flag, right?

Still needs to add this, but eventually yes :)

…ype checks

asl · 2026-01-05T22:15:42Z

swiftlang/swift-syntax#3225
@swift-ci please test

asl · 2026-01-05T22:25:12Z

@slavapestov @rjmccall

The updated PR contains the support for yields w/o dedicated yield result type (I added them on top of the previous attempts to show the approach).

Things to note:

There are no silgen changes here (as we discussed), though I have ported them separately and verified that removing special path for coroutine accessors and treating them as "ordinary coroutines" does not lead to any test breakage
Function types support multiple yields (there was some initial boilerplate for this there), but I decided to postpone decision how we'd represent multiple yields for declarations (similar to error types), so declarations only support a single yield for now.
It turned out that swift syntax parser treats function result arrow as a part of result clause, so to minimize amount of changes I decided to use for yields the syntax similar to the throws, so the yielded type uses contextual yields keyword that (if present) should be after after specifiers like throws or async
Function type construction always requires yield types to be passed (even if empty), so they won't forgotten by accident. Though, the default implementation w/o yields could be added (I left it commented out for now)
Everything is hidden behind dedicated experimental flag

Please let me know if there is something that should be changed / done differently in this approach.

asl · 2026-01-05T22:38:01Z

swiftlang/swift-syntax#3225
@swift-ci please test

asl · 2026-01-06T08:19:41Z

swiftlang/swift-syntax#3225 swiftlang/llvm-project#12064
@swift-ci please test

…unction types and declarations

asl · 2026-01-06T20:28:52Z

swiftlang/swift-syntax#3225 swiftlang/llvm-project#12064
@swift-ci please test

asl · 2026-01-07T19:23:05Z

swiftlang/swift-syntax#3225 swiftlang/llvm-project#12064
@swift-ci please test macos platform

asl · 2026-01-08T06:14:00Z

Looks like main is broken:

[2026-01-08T05:03:21.576Z] /Users/ec2-user/jenkins/workspace/swift-PR-macos/branch-main/swift/stdlib/public/core/StaticPrint.swift:866:29: error: expression uses unsafe constructs but is not marked with 'unsafe' [#]8;;https://docs.swift.org/compiler/documentation/diagnostics/strict-memory-safety\StrictMemorySafety]8;;\]
[2026-01-08T05:03:21.576Z] 864 |   let argumentClosures = message.interpolation.arguments.argumentClosures
[2026-01-08T05:03:21.576Z] 865 |   if Bool(_builtinBooleanLiteral: Builtin.ifdef_SWIFT_STDLIB_PRINT_DISABLED()) { return }
[2026-01-08T05:03:21.576Z] 866 |   let formatStringPointer = _getGlobalStringTablePointer(formatString)
[2026-01-08T05:03:21.576Z]     |                             |- error: expression uses unsafe constructs but is not marked with 'unsafe' [#]8;;https://docs.swift.org/compiler/documentation/diagnostics/strict-memory-safety\StrictMemorySafety]8;;\]
[2026-01-08T05:03:21.576Z]     |                             `- note: reference to global function '_getGlobalStringTablePointer' involves unsafe type 'UnsafePointer<CChar>' (aka 'UnsafePointer<Int8>')
[2026-01-08T05:03:21.576Z] 867 |   unsafe constant_vprintf_backend(
[2026-01-08T05:03:21.576Z] 868 |     fmt: formatStringPointer,
[2026-01-08T05:03:21.576Z] 
[2026-01-08T05:03:21.576Z] [#StrictMemorySafety]: <https://docs.swift.org/compiler/documentation/diagnostics/strict-memory-safety>
[2026-01-08T05:03:21.576Z] ninja: build stopped: subcommand failed.
[2026-01-08T05:03:21.576Z] ERROR: command `['env', '/usr/local/bin/cmake', '--build', '/Users/ec2-user/jenkins/workspace/swift-PR-macos/branch-main/build/buildbot_incremental/minimalstdlib-macosx-x86_64', '--config',

asl · 2026-01-08T06:14:09Z

swiftlang/swift-syntax#3225 swiftlang/llvm-project#12064
@swift-ci please test macos platform

rjmccall

I think just worrying about the AST part of this is fine for now, yeah. This seems like good progress.

rjmccall · 2026-01-13T00:00:35Z

include/swift/ABI/MetadataValues.h

+    HasSendingResult       = 0x00000010U,
+
+    // Values if we have any yields
+    IsCoroutine            = 0x00000020U,


Do you really need runtime metadata support? I don't think there's anything wrong with the code you're adding here, but I don't know if I'm happy about baking ABI support into the runtime for a feature we haven't actually added to the language.

Here (and many other places), I would like to avoid baking in that "coroutine" means this specific kind of coroutine. We do have other coroutine-like features, and I think we will probably add more. "YieldOnceCoroutine" is unambiguous.

So, just to clarify – in other places you'd also want to have, say FunctionType::isYieldOnceCoroutine()? This looks different from e.g. SIL level when SILFunctionType::isCoroutine() means any coroutine (being it yield_once or yield_many).

rjmccall · 2026-01-13T00:07:35Z

include/swift/AST/Types.h


 public:
  /// 'Constructor' Factory Function
+#if 0  


Refactor residue?

Oh, no. As I mentioned, I have not decided whether to keep or not yield-less ctors. As one one hand they are less verbose, but on other hand make it easy to forget about yields leading to subtle unintended differences

rjmccall · 2026-01-13T00:08:01Z

include/swift/AST/Types.h

 }

-  CanFunctionType substGenericArgs(SubstitutionMap subs) const;
+#if 0


rjmccall · 2026-01-13T00:14:10Z

lib/AST/ASTMangler.cpp

      return appendOperator("XE");
    }
+    if (fn->isCoroutine())
+      return appendOperator("Xy");


Hmm. You should think about this together with the yields — probably this should be something like a list of yields followed by a "this is a coroutine" operator, rather than being a one-off operator up here. Also, do we need to mangle the different yield_once kinds, or are we just assuming that we can use the new ABI for all such functions? (That would be best, but I'm not sure if it's ready for that.)

Oh, right. It is something left from the old approach... I need to resolve TODO in ASTMangler::appendFunctionSignature below. Ideally I'd use new ABI for non-accessors coroutines starting from the beginning. Although I do not know if some AST-level things would be required... So far it is indeed "old" one. But I guess it would be ok to break the ABI here as it is not something stable here.

rjmccall · 2026-01-13T00:14:54Z

lib/AST/ASTVerifier.cpp

        Out << "\n";
        abort();
      }
+      if (FT->hasExtInfo() && FT->isCoroutine()) {


Suggested change

if (FT->hasExtInfo() && FT->isCoroutine()) {

if (FT->isCoroutine()) {

rjmccall · 2026-01-13T00:16:43Z

lib/AST/Builtins.cpp

          _label("operation",
                 _sending(_function(_async(_throws(_thick)), _typeparam(0),
-                                    _parameters())))),
+                                    _parameters(), _yields())))),


We should be able to figure out a way to not need a builder for the yields when we don't have any.

rjmccall · 2026-01-13T00:18:37Z

lib/AST/Builtins.cpp

      FunctionType::ExtInfo info;
      auto innerFunction =
-          FunctionType::get(params, linearFnResultGen.build(builder), info);
+          FunctionType::get(params, {}, linearFnResultGen.build(builder), info);


It's unfortunate that we need this change in a million places. It was probably really nice for writing this patch that you had to fix a bunch of generic places to propagate and handle yields, but now that that's done, I wonder if we shouldn't just have a factory that defaults to not having any yields.

That's exactly what I mentioned above. I do not know :) I'm worrying that the caller might forget about yields and it might lead to dropping yields should they present during various things here and there

asl mentioned this pull request Nov 6, 2025

Add AST representation for coroutines #78508

Draft

asl force-pushed the coro-ast-no-diff branch from 8805213 to 29ab221 Compare November 13, 2025 07:01

asl force-pushed the coro-ast-no-diff branch from 29ab221 to 28dadc4 Compare November 13, 2025 20:34

asl removed request for AnthonyLatsis, susmonteiro and tshortli November 14, 2025 16:53

rjmccall reviewed Nov 17, 2025

View reviewed changes

asl added 8 commits January 4, 2026 12:29

Add basic boilerplate for AST coroutines and yields

5f17a39

First cut of making coroutine AST type

6ce87d9

Add parsing of @yield_once and @yields attributes and corresponding t…

fd5bbec

…ype checks

Properly substitute coroutines

25f6375

Few fixes related to coroutine function types.

47e8b65

Deconstruct Escapable check for coroutines with normal results

f2e76f7

Unbreak tests after read / modify accessors added

fdbec58

Add typecheck tests for @yield_once / @yields

5a4aa06

asl force-pushed the coro-ast-no-diff branch from 28dadc4 to d5b8b73 Compare January 5, 2026 22:12

asl requested a review from ktoso as a code owner January 5, 2026 22:12

asl requested a review from rjmccall January 5, 2026 22:25

asl force-pushed the coro-ast-no-diff branch from d5b8b73 to e336b6a Compare January 5, 2026 22:37

asl mentioned this pull request Jan 6, 2026

Changes to LLDB swift type system plugin due to introduction of coroutines to AST swiftlang/llvm-project#12064

Open

asl added 3 commits January 6, 2026 22:52

Remove YieldResult type, switch to direct yields representation for f…

cdb48d1

…unction types and declarations

Hide non-accessor coroutines behind experimental flag

49176c9

Fix serialization / deserialization of non-accessor coroutines

1c28062

asl force-pushed the coro-ast-no-diff branch from e336b6a to 1c28062 Compare January 6, 2026 20:28

rjmccall reviewed Jan 13, 2026

View reviewed changes

	if (FT->hasExtInfo() && FT->isCoroutine()) {
	if (FT->isCoroutine()) {

Add AST representation for coroutines #85372

Are you sure you want to change the base?

Add AST representation for coroutines #85372

Uh oh!

Conversation

asl commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asl commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asl commented Nov 6, 2025

Uh oh!

asl commented Nov 13, 2025

Uh oh!

asl commented Nov 13, 2025

Uh oh!

asl commented Nov 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asl Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asl commented Jan 5, 2026

Uh oh!

asl commented Jan 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

asl commented Jan 5, 2026

Uh oh!

asl commented Jan 6, 2026

Uh oh!

asl commented Jan 6, 2026

Uh oh!

asl commented Jan 7, 2026

Uh oh!

asl commented Jan 8, 2026

Uh oh!

asl commented Jan 8, 2026

Uh oh!

rjmccall left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

asl commented Nov 6, 2025 •

edited

Loading

asl commented Nov 6, 2025 •

edited

Loading

asl Dec 8, 2025 •

edited

Loading

asl commented Jan 5, 2026 •

edited

Loading