Introduce Task monad

[jypma]

Relates to #1801

This commit introduces Task, a data structure that represents a recipe, or program, for producing a value of type T (or failing with an exception). It is similar in semantics to RunnableGraph[T], but intended as first-class building block.

It has the following properties:

• A task can have resources associated to it, which are guaranteed to be released if the task is cancelled or fails
• Tasks can be forked so multiple ones can run concurrently
• Such forked tasks can be cancelled

A Task can be created from a RunnableGraph which has a KillSwitch, by connecting a Source and a Sink through a KillSwitch, or by direct lambda functions.

Open discussion points and TODOs (in order of highest architectural impact):

• Current cancellation of a graph through KillSwitch doesn't communicate back a trigger when the graph is actually done cancelling. However, since we also have the Future[T] that the graph's sink materializes to, that's enough to synchronize on.
• Resource safety, which can then guarantee cleanup even under cancellation signals.
• Background processing wth cancellation (fibers, forkDaemon, forkResource). No child fibers (yet).
• Add many more combinators (plain zip, andThen, before, raceAll)
• Add more combinators (asResource, ...)
• Add .delay() and general scheduling by hooking in the scheduling features of Materializer
• Sink.forEachTask and Flow.mapTask
• Resource.ofAutoCloseable
• Task.never()
• Task.runAsMain (or perhaps Application base classes / traits), which runs a task as main(), capturing Ctrl+C to interrupt the task (and, hence, clean up resources).
• Move to task module. Don't depend on stream. Streams should depend on task, so we can use tasks directly in stream combinators.
• Scala DSL

Needing an eye from reviewers:

• More test cases to exercise typical edge cases and concurrency race conditions

To be handled in later pull requests:

• TestClock (implementation of Clock for unit tests, where tests can advance the clock, so time-driven code can run deterministically and faster than realtime)
• Fun Task-friendly concurrency primitives like a queue, mutex, and a mutable ref with stream subscription support.
• (Potentially) Another monad Effect[E,A] which introduces modelled errors (in addition to exceptions). This can be easily built on top of a TaskDef[Either[E,A]].

Fixes #1801 .

[jypma]

I'm getting closer to a first well-connected base with things like resource safety, interruptability and time somewhat properly present. I think we should have a discussion on what to do with the interruptability of graphs, though:

• Pekko's KillSwitch only has a void method to stop a graph. You won't know when it's actually stopped, unless you also have a Sink with a future that completes when the graph is actually dead.
• Probably as a side result of this, pekko-connectors-kafka has its own DrainingControl concept, which does promise to shut down the graph (even cleanly).

Is this something to discuss and look at now? I suppose it could be done later, by putting interruptability into a type class when creating a GraphDef, but it'd be nicer to stick to first-level concepts (like KillSwitch).

[pjfanning]

@jypma if you thinking copying some of the pekko-connectors-kafka code into your task package helps, go ahead. Let's keep any additions specific to the task package for now and we can later discuss if it is worth taking some of reusable bits out and making them available as public APIs or to be used by other Pekko classes.

[jypma]

I've decided to just introduce a RunningGraph interface to represent a graph that will eventually complete (with a Task), and has the ability to be interrupted (with another Task). That covers both KillSwitch (where interruption is immediate, but followed by awaiting the sink's completion stage) and DrainingControl (where interruption has a clear "done" signal).

[He-Pin]

@jypma Seems there are some conflicts. I would like to learn more once this is ready

[He-Pin]

This is an ambitious and substantial addition to Pekko -- introducing a Task monad with resource safety, cancellation, and concurrency primitives. A few observations:

1. Architectural significance: This introduces a significant new abstraction layer. The TODO list in the PR body acknowledges important open items: Scala DSL completeness, moving to a dedicated task module, and Task.runAsMain. These should be resolved before merge.

2. Module placement: Currently in the stream module, but the TODO notes that streams should eventually depend on task, not the other way around. This architectural decision should be settled before merging.

3. Testing gap: The PR acknowledges needing more test cases for edge cases and concurrency race conditions. This is critical for a concurrency primitive.

4. Staleness: Open since March 2025 with activity stopping in May 2025. The concept is interesting but the PR is clearly incomplete (several unchecked TODO items). @jypma -- is this something you plan to complete? Otherwise consider closing and revisiting when there's bandwidth to finish the full implementation.

[jypma]

I don't have any plans to complete this, unfortunately. I've concluded that Pekko doesn't currently have the resource safeties needed for a proper Task monad, particularly with respect to graph cancellation and shutdown. Hence, we'd have to look at the internals of Pekko first, e.g. tightening the semantics of callbacks within a graph when a shutdown happens, to get precise control there.

For those reading along: what I've been working on a bit on instead, is to take a framework that does give these guarantees, but only in Scala (ZIO), and create a "nice" fluent Java API on top of that. Pekko streams could slot in there as well. My initial approach for that is here: JIO, although development there has also stalled a bit (but only due to shifting interests). Send me a message if anyone is curious about that direction.

For this PR on Pekko, feel free to close it. Thanks for all the thoughtful discussion and comments!