diff --git a/durable-streams/README.md b/durable-streams/README.md new file mode 100644 index 00000000..add6a748 --- /dev/null +++ b/durable-streams/README.md @@ -0,0 +1,559 @@ +# @effectionx/durable-streams + +Durable execution for [Effection](https://frontside.com/effection) — crash-safe generator workflows that survive process restarts by journaling effects to an append-only stream. + +```typescript +import { durableRun, durableCall, durableAll } from "@effectionx/durable-streams"; + +function* processOrder(orderId: string): Workflow { + const order = yield* durableCall("fetchOrder", () => fetchOrder(orderId)); + const [fraud, inventory] = yield* durableAll([ + () => durableCall("checkFraud", () => checkFraud(order)), + () => durableCall("checkInventory", () => checkInventory(order)), + ]); + yield* durableCall("chargeCard", () => chargeCard(order.payment)); + yield* durableCall("fulfillOrder", () => fulfill(order)); +} +``` + +If the process crashes between `chargeCard` and `fulfillOrder`, the workflow resumes exactly from that point. `chargeCard` is not called again. `fulfillOrder` runs once, as intended. + +--- + +## Mental model + +An Effection generator is already an **effect description machine** — it yields descriptions of what it wants to happen, and the runtime interprets them. `@effectionx/durable-streams` extends this: instead of simply executing each effect, the runtime first journals the result to an append-only stream, then resumes the generator. + +On restart, the runtime reads those journal entries back and feeds the stored results directly into the generator, replaying its execution path without re-running any side effects. When the journal runs out, execution transitions seamlessly to live mode. + +The generator itself never knows which mode it's in. It sees a sequence of values flowing from `yield*` — whether those values came from a live network call or a replay of one is invisible to it. + +This means your workflow logic is written once, with no replay-awareness code, no `if (replaying)` branches, and no explicit checkpoint calls. + +--- + +## The journal: what goes in, what doesn't + +The journal is an append-only stream of two event types: + +```typescript +type DurableEvent = Yield | Close; +``` + +**`Yield`** is written after a user-facing effect resolves. It records both the effect description (what was requested) and the result (what happened): + +```typescript +interface Yield { + type: "yield"; + coroutineId: string; // e.g. "root.0.1" + description: { + type: string; // "call", "sleep", "action", etc. + name: string; // the stable effect name + [key: string]: Json; // extra input fields, stored verbatim + }; + result: Result; // { status: "ok", value } | { status: "err" } | { status: "cancelled" } +} +``` + +**`Close`** is written when a coroutine terminates — whether it completed, threw an error, or was cancelled. Close events are load-bearing: they tell the runtime on restart which coroutines finished cleanly and which need re-execution. + +### What goes into the journal + +User-facing effects: anything that interacts with the outside world. In practice, anything you express with `durableCall`, `durableSleep`, `durableAction`, `durableEach`, or a custom `createDurableEffect`. + +```text +[0] yield root { type: "call", name: "fetchOrder" } result: { status: "ok", value: { id: "42", ... } } +[1] yield root.0 { type: "call", name: "checkFraud" } result: { status: "ok", value: true } +[2] yield root.1 { type: "call", name: "checkInventory" } result: { status: "ok", value: true } +[3] close root.0 result: { status: "ok", value: true } +[4] close root.1 result: { status: "ok", value: true } +[5] yield root { type: "call", name: "chargeCard" } result: { status: "ok" } +[6] yield root { type: "call", name: "fulfillOrder" } result: { status: "ok" } +[7] close root result: { status: "ok" } +``` + +### What doesn't go into the journal + +**Infrastructure effects** — scope setup, context reads, middleware. These run transparently during both live execution and replay. They're deterministic by construction: they depend only on the runtime's internal state, which is reconstructed identically during replay because all user-facing effects are replayed in order. + +The `ephemeral()` function is the explicit escape hatch when you need to run non-durable Effection operations inside a `Workflow`. It produces no journal entry and re-runs on replay: + +```typescript +function* myWorkflow(): Workflow { + // useScope() is infrastructure — use ephemeral() to run it in a Workflow + const signal = yield* ephemeral(useAbortSignal()); + + // durableCall is journaled + return yield* durableCall("fetchData", () => fetchData(signal)); +} +``` + +--- + +## Workflows vs. Operations + +A `Workflow` is a generator that only yields `DurableEffect` values. TypeScript enforces this at compile time — yielding a plain Effection `Operation` inside a `Workflow` generator is a type error: + +```typescript +function* safeWorkflow(): Workflow { + yield* durableSleep(1000); // ✓ DurableEffect + yield* durableCall("fetch", fn); // ✓ DurableEffect + yield* sleep(1000); // ✗ TypeError — use durableSleep + yield* call(fn); // ✗ TypeError — use durableCall +} +``` + +This is the key design guarantee: **if it compiles as a `Workflow`, it's durable** (except values explicitly wrapped with `ephemeral()`, which intentionally opt out of journaling). + +Every `Workflow` is structurally compatible with `Operation`, so you can always use a workflow where an operation is expected. + +### Core workflow effects + +| Effect | Description | +|--------|-------------| +| `durableCall(name, fn)` | Call a function returning a `Promise` or `Operation` | +| `durableSleep(ms)` | Wait for a duration | +| `durableAction(name, executor)` | Custom callback-based effect | +| `versionCheck(name, { minVersion, maxVersion })` | Version gate for code evolution | +| `durableEach(name, source)` | Durable iteration with per-item checkpointing | + +### Concurrency combinators + +Combinators return `Workflow` and delegate to Effection's native structured concurrency primitives. Children must themselves be `Workflow`. + +| Combinator | Behavior | +|------------|----------| +| `durableSpawn(workflow)` | Spawn a concurrent child, returns `Task` | +| `durableAll([...workflows])` | Run all concurrently, wait for all to complete | +| `durableRace([...workflows])` | Run all, return first winner, cancel the rest | + +### Prefer Operations over async/await + +When writing functions called from `durableCall`, prefer returning an `Operation` over a `Promise`. Operations participate fully in Effection's structured concurrency — they can be cancelled, they respect scope lifetimes, and they compose cleanly: + +```typescript +// Prefer this: +function fetchUser(id: string): Operation { + return resource(function* (provide) { + const controller = new AbortController(); + try { + const response = yield* call(() => + fetch(`/users/${id}`, { signal: controller.signal }) + ); + yield* provide(yield* call(() => response.json())); + } finally { + controller.abort(); + } + }); +} + +// Over this: +async function fetchUser(id: string): Promise { + const response = await fetch(`/users/${id}`); + return response.json(); +} + +// Both work with durableCall, but the Operation version is cancellable: +const user = yield* durableCall("fetchUser", () => fetchUser(id)); +``` + +When the parent scope is cancelled (e.g., a race loser), an `Operation`-returning function cleans up immediately. A `Promise`-returning function keeps the network request open until it settles. + +### Durable iteration + +Use `durableEach` to iterate over a source with per-item checkpointing. Each call to `durableEach.next()` produces a journal entry — if the process crashes mid-loop, it resumes at the next unprocessed item: + +```typescript +function* processQueue(): Workflow { + for (let msg of yield* durableEach("queue", queueSource)) { + yield* durableCall("process", () => processMessage(msg)); + yield* durableEach.next(); // checkpoint + pre-fetch next item + } +} +``` + +The `DurableSource` interface uses Operations: + +```typescript +interface DurableSource { + next(): Operation<{ value: T } | { done: true }>; + close?(): void; // called on cancellation or completion, must be idempotent +} +``` + +--- + +## Entry point: durableRun + +`durableRun` is itself an `Operation` — it inherits the caller's Effection scope, including any middleware installed on it: + +```typescript +function* durableRun( + workflow: () => Workflow | Operation, + options: { stream: DurableStream; coroutineId?: string } +): Operation +``` + +Typical usage from standalone async code: + +```typescript +import { run } from "effection"; +import { durableRun } from "@effectionx/durable-streams"; +import { useHttpDurableStream } from "@effectionx/durable-streams"; + +await run(function* () { + const stream = yield* useHttpDurableStream({ + baseUrl: "http://localhost:4437", + streamId: "order-42", + producerId: "worker-1", + epoch: 1, + }); + + const result = yield* durableRun( + () => processOrder("order-42"), + { stream } + ); +}); +``` + +When `durableRun` is called as a generator inside another generator, it shares the parent's scope chain — middleware installed before the `yield*` is visible inside the workflow: + +```typescript +import { useFileContentGuard } from "@effectionx/durable-effects"; + +function* supervisedRun(): Operation { + // Install middleware (see Replay Guards below) + yield* useFileContentGuard(); + + // All workflows run inside this durableRun inherit the guard + yield* durableRun(() => buildPipeline(), { stream }); +} +``` + +--- + +## DurableRuntime + +Effects that interact with the operating system (file I/O, subprocess execution, HTTP requests) use a `DurableRuntime` abstraction instead of importing Node-specific or Deno-specific APIs directly. The runtime is installed on the Effection scope as a context value before calling `durableRun`: + +```typescript +import { DurableRuntimeCtx } from "@effectionx/durable-streams"; +import { nodeRuntime } from "@effectionx/durable-effects"; + +function* main(): Operation { + const scope = yield* useScope(); + scope.set(DurableRuntimeCtx, nodeRuntime()); + + yield* durableRun(() => myWorkflow(), { stream }); +} +``` + +Effects access the runtime inside their operation callbacks via `scope.expect(DurableRuntimeCtx)`. The interface is fully Operation-native — every I/O method returns `Operation`, not `Promise`, so cancellation flows through Effection's structured concurrency automatically. + +The `@effectionx/durable-effects` package provides `nodeRuntime()` for production use and `stubRuntime()` for testing. + +--- + +## Coroutine identity + +Every generator instance running under `durableRun` gets a stable coroutine ID — a dot-delimited path that encodes its position in the scope tree: + +```text +root → "root" + first child of root → "root.0" + second child of root → "root.1" + first child of .1 → "root.1.0" +``` + +These IDs are assigned by a per-parent creation counter and are identical across runs, given the same generator code and the same resolution sequence. This determinism is what makes it possible to match journal entries to the right generator instances on replay. + +You never assign or manage coroutine IDs manually — they're derived entirely from the structure of your generator code. + +--- + +## Replay + +When `durableRun` starts, it reads the full event stream, builds an in-memory `ReplayIndex`, then starts the workflow generator. As the generator yields effects: + +1. **Replay path** — if the index has an entry for this coroutine at this position, the stored result is fed directly to the generator via `iterator.next(value)` or `iterator.throw(error)`. The effect's live executor is never called. + +2. **Live path** — if the index has no entry, the effect executes normally. Once it resolves, the result is persisted to the stream *before* the generator is resumed (`persist-before-resume`). + +The transition from replay to live happens **per-coroutine**, not globally. In a fork/join workflow where two children ran before a crash and a third didn't, the first two replay their stored results while the third executes live — all simultaneously, within the same `durableAll`. + +### Persist-before-resume + +This is the protocol's most critical invariant: **the `Yield` event must be durably written to the stream before `iterator.next()` is called**. If the process crashes between an effect resolving and the journal write completing, the effect will be re-executed on the next run — which is safe, because the generator hasn't advanced past that point yet. + +Violating this invariant (advancing the generator before the write) creates an unrecoverable gap: the journal would be missing an entry, and replay would feed the wrong result to a subsequent effect. + +--- + +## Divergence detection + +During replay, every yielded effect is validated against its journal entry. Only two fields are compared: `description.type` and `description.name`. If they match, replay proceeds. If they don't, a `DivergenceError` is raised immediately. + +```typescript +// Journal has: { type: "call", name: "fetchOrder" } +// Code yields: { type: "call", name: "chargeCard" } ← mismatch at position 0 +// → DivergenceError +``` + +Two additional terminal conditions are checked: + +- **Generator finishes early**: the code returns before consuming all journal entries — effects were removed. +- **Generator continues past close**: the journal shows the coroutine closed, but the code keeps yielding — effects were added. + +Both indicate the code has changed in a way that makes the stored history invalid. The solution for intentional code changes is `versionCheck`: + +```typescript +function* orderWorkflow(orderId: string): Workflow { + const version = yield* versionCheck("add-fraud-check", { minVersion: 0, maxVersion: 1 }); + + if (version >= 1) { + // New in v1 — in-flight v0 workflows skip this, new v1 workflows run it + yield* durableCall("fraudCheck", () => fraudCheck(orderId)); + } + + yield* durableCall("fetchOrder", () => fetchOrder(orderId)); + yield* durableCall("chargeCard", () => chargeCard(orderId)); +} +``` + +### Divergence policy + +The default divergence policy is strict — any mismatch is fatal. You can override this per-scope using `scope.around(Divergence, ...)`: + +```typescript +scope.around(Divergence, { + decide([info], next) { + // "run-live" disables replay from this point forward for this coroutine + if (info.kind === "description-mismatch" && canRecoverFrom(info)) { + return { type: "run-live" }; + } + return next(info); + } +}); +``` + +The `run-live` decision tells the runtime to disable replay for that coroutine and execute all subsequent effects live, effectively treating the crash point as the beginning of a fresh run. + +--- + +## Replay guards + +Divergence detection catches *structural* mismatches — the effect sequence changed. Replay guards catch *staleness* mismatches — the effect sequence is the same, but the external world has changed since the journal entry was recorded. + +The canonical example is a file-backed effect. If the workflow previously read `./component.mdx` and that file has since been edited, replaying the stored result would silently use stale content. A replay guard detects this and can halt replay with an error. + +### The two-phase model + +Every replay guard has two phases, separated by a strict I/O boundary: + +**Phase 1 — `check`**: runs in generator context before replay begins. I/O is allowed. Use it to gather current state (compute file hashes, check timestamps) and cache results in the middleware closure. + +**Phase 2 — `decide`**: runs synchronously inside the replay loop, after identity matching succeeds. Must be pure — no I/O, no side effects. Reads from the cache populated during `check` and returns a `ReplayOutcome`. + +This separation is necessary because the replay loop is synchronous. All observation-gathering must happen upfront. + +### Writing a replay guard + +Use `scope.around(ReplayGuard, ...)` to install a guard. The guard receives each `Yield` event from the journal: + +```typescript +import { ReplayGuard, type ReplayOutcome } from "@effectionx/durable-streams"; +import { call, useScope } from "effection"; +import type { Operation } from "effection"; + +function* useMyGuard(): Operation { + const scope = yield* useScope(); + + // The cache lives in this closure — populated during check, read during decide + const cache = new Map(); + + scope.around(ReplayGuard, { + // Phase 1: gather observations (I/O allowed, runs before replay starts) + *check([event], next): Operation { + const resourceId = event.description.resourceId; + if (typeof resourceId === "string" && !cache.has(resourceId)) { + const currentVersion = yield* call(() => fetchCurrentVersion(resourceId)); + cache.set(resourceId, currentVersion); + } + return yield* next(event); // always call next — other guards may need this event + }, + + // Phase 2: make a decision (synchronous, pure, no I/O) + decide([event], next): ReplayOutcome { + const resourceId = event.description.resourceId; + if (typeof resourceId !== "string") { + return next(event); // not our event — delegate + } + + const storedVersion = (event.result as any)?.value?.version; + const currentVersion = cache.get(resourceId); + + if (currentVersion && currentVersion !== storedVersion) { + return { + outcome: "error", + error: new Error( + `Resource changed: ${resourceId} (stored: ${storedVersion}, current: ${currentVersion})` + ), + }; + } + + return next(event); // no opinion — delegate + }, + }); +} +``` + +Install the guard before calling `durableRun`: + +```typescript +function* supervisedWorkflow(): Operation { + yield* useMyGuard(); // children inherit this through Effection's scope inheritance + + yield* durableRun(() => myWorkflow(), { stream }); +} +``` + +### Effect descriptions carry input data; results carry output data + +For a guard to work, the effect being guarded must store the information needed for validation: + +- **Input fields** (the path, resource ID, URL) go in extra fields on `EffectDescription`. These fields are stored verbatim in the journal but never compared during divergence detection. +- **Output fields** (content hash, ETag, version) go in `result.value` alongside the actual content. + +```typescript +import { readFile } from "node:fs/promises"; + +function* durableReadFile(path: string): Workflow { + const { content } = yield* durableCall("readFile", async () => { + const content = await readFile(path, "utf8"); + const hashBuffer = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(content)); + const contentHash = Array.from(new Uint8Array(hashBuffer)) + .map(b => b.toString(16).padStart(2, "0")).join(""); + + return { content, contentHash }; + // ↑ content hash returned alongside the actual content + }); + return content; +} + +// The description stored in the journal: +// { type: "call", name: "readFile", path: "./input.txt" } +// ↑ path stored as extra field + +// The result stored in the journal: +// { status: "ok", value: { content: "...", contentHash: "sha256:abc123" } } +// ↑ hash stored in result value +``` + +The guard's `check` phase reads `event.description.path` and computes the current hash. The `decide` phase reads `event.result.value.contentHash` and compares. No separate metadata or side-channel is needed. + +### Pre-built guards in `@effectionx/durable-effects` + +The `@effectionx/durable-effects` package provides ready-to-use guards for common staleness scenarios: + +- `useFileContentGuard()` — detects when a file's content has changed via hash comparison +- `useGlobContentGuard()` — detects when a directory scan's results have changed +- `useCodeFreshnessGuard(lookup)` — detects when eval source or bindings have changed + +```typescript +import { useFileContentGuard } from "@effectionx/durable-effects"; + +function* myPipeline(): Operation { + yield* useFileContentGuard(); + yield* durableRun(() => buildDocuments(), { stream }); +} +``` + +### Guard composition + +Multiple guards compose naturally. Each guard either returns an outcome or calls `next(event)` to pass control to the next guard in the chain: + +```typescript +import { useFileContentGuard, useGlobContentGuard } from "@effectionx/durable-effects"; + +function* supervisedPipeline(): Operation { + yield* useFileContentGuard(); // checks file-backed effects + yield* useGlobContentGuard(); // checks directory scan results + yield* useMyCustomGuard(); // your own guard + + yield* durableRun(() => pipeline(), { stream }); +} +``` + +If any guard returns `{ outcome: "error" }`, replay halts. Guards that return `next(event)` delegate, and the default at the bottom of the chain always returns `{ outcome: "replay" }` — preserving "logs are authoritative" for events that no guard has an opinion on. + +--- + +## Stream backends + +`DurableStream` is an abstract interface: + +```typescript +interface DurableStream { + readAll(): Operation; + append(event: DurableEvent): Operation; +} +``` + +### In-memory (testing) + +```typescript +import { InMemoryStream } from "@effectionx/durable-streams"; + +const stream = new InMemoryStream(); +// Or pre-populate with events: +const prepopulatedStream = new InMemoryStream(existingEvents); +// Inspect append count, inject failures: +stream.appendCount; +stream.injectFailure = new Error("disk full"); +``` + +### HTTP (Durable Streams protocol) + +Backed by the [Durable Streams](https://durable.run) protocol — an append-only HTTP streaming protocol with idempotent producers and epoch-based fencing. + +```typescript +import { useHttpDurableStream } from "@effectionx/durable-streams"; + +const stream = yield* useHttpDurableStream({ + baseUrl: "http://localhost:4437", + streamId: "workflow-abc-123", + producerId: "scheduler-worker-1", + epoch: 1, // increment this on scheduler restart to fence zombie writers +}); +``` + +Appends are serialized via an internal queue and worker — concurrent `append()` calls from `durableAll` children are safely sequenced without application-level coordination. Every append is synchronous (no `lingerMs` batching) to preserve `persist-before-resume`. + +### Custom backends + +Implement `DurableStream` directly. The only requirements are append-only semantics, prefix-closure (no gaps), and that `append()` only resolves after the event is durably persisted: + +```typescript +class PostgresStream implements DurableStream { + *readAll(): Operation { + return yield* call(() => + db.query("SELECT event FROM events WHERE stream_id = $1 ORDER BY position", [this.streamId]) + .then(r => r.rows.map(r => r.event)) + ); + } + + *append(event: DurableEvent): Operation { + yield* call(() => + db.query("INSERT INTO events (stream_id, event) VALUES ($1, $2)", [this.streamId, event]) + ); + } +} +``` + +--- + +## Long-running workflows + +For workflows that process unbounded streams, journals grow without limit. The `durableEach` + Continue-As-New pattern bounds this growth: after N iterations, the workflow signals for a fresh start with the current cursor position as its seed. This is a planned feature — in the meantime, `durableEach` is appropriate for bounded batches where journal size is not a constraint. + +--- diff --git a/durable-streams/combinators.ts b/durable-streams/combinators.ts new file mode 100644 index 00000000..d8ad045c --- /dev/null +++ b/durable-streams/combinators.ts @@ -0,0 +1,280 @@ +/** + * Structured concurrency combinators for durable workflows. + * + * durableSpawn, durableAll, durableRace — each wraps child workflows + * with DurableContext (coroutine IDs, Close events) so that structured + * concurrency is fully journaled and replayable. + * + * Each combinator returns Workflow (not Operation) so it can be + * used directly inside a Workflow via yield*. Internally, the infrastructure + * effects (useScope, spawn, all, race) are wrapped with ephemeral() — + * these are durable-safe operations that set up scope/context and don't + * need journaling. See DEC-034. + * + * Child workflows must be Workflow — bare Operations are rejected at + * compile time. Use ephemeral() to explicitly opt in to non-durable + * children. + * + * See protocol spec §7 (structured concurrency), §10 (race semantics). + */ + +import { + all as effectionAll, + race as effectionRace, + spawn, + suspend, + useScope, +} from "effection"; +import type { Operation, Task } from "effection"; +import { type DurableContext, DurableCtx } from "./context.ts"; +import { ephemeral } from "./ephemeral.ts"; +import { deserializeError, serializeError } from "./serialize.ts"; +import type { Close, Json, Workflow, WorkflowValue } from "./types.ts"; + +// --------------------------------------------------------------------------- +// Internal: wrap a child workflow with DurableContext + Close emission +// --------------------------------------------------------------------------- + +/** + * Run a child workflow within a spawned scope, setting up its own + * DurableContext and emitting a Close event when it terminates. + * + * This is the core building block for all structured concurrency combinators. + * + * It: + * 1. Checks if the child already completed (has Close event) — short-circuits + * 2. Sets DurableCtx on the child's scope with the child's coroutineId + * 3. Runs the child workflow (its DurableEffects use the child's coroutineId) + * 4. Appends Close(ok|err) when the child terminates + * + * IMPORTANT: This must be called inside a spawn() so it gets its own scope. + * The caller is responsible for spawn(). + */ +function* runDurableChild( + childWorkflow: () => Workflow, + childId: string, + parentCtx: DurableContext, +): Operation { + const { replayIndex, stream } = parentCtx; + + // Short-circuit: child already completed in a previous run. + // NOTE: Replay guard validation is not bypassed here — the check phase + // (runCheckPhase in durableRun) already iterated ALL Yield events + // (including this child's) before any workflow code runs. Guards have + // already had a chance to veto stale data. This fast-path only skips + // re-running the child's generator, not the guard validation. + if (replayIndex.hasClose(childId)) { + const closeEvent = replayIndex.getClose(childId)!; + if (closeEvent.result.status === "ok") { + return closeEvent.result.value as T; + } else if (closeEvent.result.status === "err") { + throw deserializeError(closeEvent.result.error); + } else { + // cancelled — this child was cancelled in a previous run (e.g., + // a race loser). Instead of throwing, we suspend forever. The + // parent combinator (race/all) will cancel this child as part of + // normal structured concurrency teardown, just like the original + // run. The Close(cancelled) event already exists in the journal, + // so we skip re-emitting it (the finally block checks for this). + // + // INVARIANT: This branch is only reachable when a parent combinator + // (durableRace or durableAll with a failed sibling) will cancel this + // child. Close(cancelled) in the journal means the child was + // previously cancelled by structured concurrency, so on replay the + // same combinator will cancel it again. This cannot deadlock. + yield* suspend(); + // unreachable — suspend blocks until cancelled + return undefined as T; + } + } + + // Set child's DurableContext on this scope + const scope = yield* useScope(); + scope.set(DurableCtx, { + replayIndex, + stream, + coroutineId: childId, + childCounter: 0, + }); + + // Track whether we completed normally or via error, so that + // the finally block can detect cancellation (the remaining case). + let closeEvent: Close | undefined; + + try { + // Run the child workflow. DurableEffects inside the child read + // DurableCtx from the scope, so they'll use childId. + const result: T = yield* childWorkflow(); + + // Record Close(ok) — will be appended in finally + closeEvent = { + type: "close", + coroutineId: childId, + result: { status: "ok", value: result as Json }, + }; + + return result; + } catch (error) { + // Record Close(err) — will be appended in finally + closeEvent = { + type: "close", + coroutineId: childId, + result: { + status: "err", + error: serializeError( + error instanceof Error ? error : new Error(String(error)), + ), + }, + }; + + throw error; + } finally { + // If closeEvent is still undefined, the generator was cancelled + // (Effection called iterator.return(), skipping both the normal + // return path and the catch block). + if (!closeEvent) { + closeEvent = { + type: "close", + coroutineId: childId, + result: { status: "cancelled" }, + }; + } + + // Don't re-emit a Close event if one already exists in the journal + // (e.g., a cancelled child being replayed via suspend()). + if (!replayIndex.hasClose(childId)) { + // Append the Close event. + yield* stream.append(closeEvent!); + } + } +} + +// --------------------------------------------------------------------------- +// durableSpawn — spawn a single durable child, returns Task +// --------------------------------------------------------------------------- + +/** + * Spawn a durable child workflow. + * + * Assigns a deterministic coroutine ID (parentId.N), sets up DurableContext + * on the child scope, and ensures Close events are emitted. + * + * Returns a Task that can be yield*-ed to get the child's result. + * + * Returns Workflow> via ephemeral() — the infrastructure effects + * (useScope, spawn) are durable-safe scope setup that doesn't need + * journaling and re-runs correctly on replay. + */ +export function durableSpawn( + childWorkflow: () => Workflow, +): Workflow> { + return ephemeral( + (function* (): Operation> { + const scope = yield* useScope(); + const ctx = scope.expect(DurableCtx); + + // Assign deterministic child ID + const childIndex = ctx.childCounter++; + const childId = `${ctx.coroutineId}.${childIndex}`; + + // Spawn the child with durable wrapping + return yield* spawn(() => runDurableChild(childWorkflow, childId, ctx)); + })(), + ); +} + +// --------------------------------------------------------------------------- +// durableAll — fork/join, wait for all children +// --------------------------------------------------------------------------- + +/** + * Run multiple durable workflows concurrently and wait for all to complete. + * + * Each child gets a deterministic coroutine ID (parentId.0, parentId.1, ...). + * Each child's effects are journaled under its own coroutineId. + * Each child emits a Close event on termination. + * + * If any child fails, remaining children are cancelled (fail-fast, + * Effection's default structured concurrency behavior via all()). + * + * Returns an array of results in the same order as the input workflows. + * + * See spec §7, §11.5. + */ +export function durableAll( + workflows: (() => Workflow)[], +): Workflow { + return ephemeral( + (function* (): Operation { + const scope = yield* useScope(); + const ctx = scope.expect(DurableCtx); + + // Build child Operations, one per workflow. Each gets its own + // deterministic coroutineId and Close event handling. + const childOps: Operation[] = workflows.map((workflow) => { + const childIndex = ctx.childCounter++; + const childId = `${ctx.coroutineId}.${childIndex}`; + + return { + *[Symbol.iterator]() { + return yield* runDurableChild(workflow, childId, ctx); + }, + }; + }); + + // Delegate to Effection's native all() which uses trap() internally + // for proper error isolation. This means: + // - Child errors are catchable by the caller via try/catch + // - When any child fails, remaining siblings are cancelled + // - The error propagates with the original message intact + return yield* effectionAll(childOps); + })(), + ); +} + +// --------------------------------------------------------------------------- +// durableRace — first child to complete wins, others cancelled +// --------------------------------------------------------------------------- + +/** + * Race multiple durable workflows. The first to complete wins; + * remaining children are cancelled. + * + * Each child gets a deterministic coroutine ID. When the winner + * completes, Effection cancels the remaining children via + * iterator.return(). The runDurableChild wrapper detects this + * (closeEvent is undefined in the finally block) and emits + * Close(cancelled) for each loser. + * + * On replay, children with Close(cancelled) in the journal suspend + * indefinitely (yield* suspend()), letting the parent race cancel + * them naturally — matching the original live behavior. + * + * See spec §10. + */ +export function durableRace( + workflows: (() => Workflow)[], +): Workflow { + return ephemeral( + (function* (): Operation { + const scope = yield* useScope(); + const ctx = scope.expect(DurableCtx); + + // Build Operations for each child — each gets its own coroutineId + // and Close event handling via runDurableChild. + const childOps: Operation[] = workflows.map((workflow) => { + const childIndex = ctx.childCounter++; + const childId = `${ctx.coroutineId}.${childIndex}`; + + return { + *[Symbol.iterator]() { + return yield* runDurableChild(workflow, childId, ctx); + }, + }; + }); + + // Use Effection's native race() which handles cancellation properly + return yield* effectionRace(childOps); + })(), + ); +} diff --git a/durable-streams/context.ts b/durable-streams/context.ts new file mode 100644 index 00000000..9b2d96a4 --- /dev/null +++ b/durable-streams/context.ts @@ -0,0 +1,30 @@ +/** + * DurableContext — the scope-local state for durable execution. + * + * Stored on each Effection scope via createContext(). Child scopes + * inherit the shared replayIndex and stream, but get their own + * coroutineId and childCounter. + */ + +import { type Context, createContext } from "effection"; +import type { ReplayIndex } from "./replay-index.ts"; +import type { DurableStream } from "./stream.ts"; +import type { CoroutineId } from "./types.ts"; + +export interface DurableContext { + /** Shared replay index (built from stream on startup). */ + replayIndex: ReplayIndex; + /** Shared durable stream for appending events. */ + stream: DurableStream; + /** This coroutine's hierarchical ID. */ + coroutineId: CoroutineId; + /** Counter for assigning child IDs. */ + childCounter: number; +} + +/** + * Effection Context for durable execution state. + * Set on the root scope by durableRun(); inherited by child scopes. + */ +export const DurableCtx: Context = + createContext("@effection/durable"); diff --git a/durable-streams/demo/README.md b/durable-streams/demo/README.md new file mode 100644 index 00000000..659b3404 --- /dev/null +++ b/durable-streams/demo/README.md @@ -0,0 +1,80 @@ +# Durable Dinner Demo + +A live demo of durable execution built on Effection. It runs a cooking workflow, +lets you hard-kill the process, then restart and watch it replay from the +journal without duplicating completed work. + +## Prerequisites + +- Node.js 22+ +- `tmux` 3.x+ (`brew install tmux`) + +## Quick Start (tmux launcher) + +From `durable-streams/`: + +```sh +./demo/start.sh +``` + +Or with a custom stream ID: + +```sh +./demo/start.sh my-stream +``` + +This opens a `tmux` session named `durable-dinner` with 3 panes: + +```text +┌─────────────────────┬──────────────────┐ +│ │ Cook (focused) │ +│ Observer ├──────────────────┤ +│ (server + journal) │ Control │ +│ │ (kill cmd) │ +└─────────────────────┴──────────────────┘ +``` + +- **Observer** (`demo/observe.ts`) — starts the server and tails the journal + via SSE, printing color-coded events as they arrive +- **Cook** (`demo/cook.ts`) — the durable cooking workflow (focused, press Enter) +- **Control** — pre-typed kill command to simulate a crash + +## Demo Script + +1. Start with `./demo/start.sh` +2. In the cook pane, press Enter to run the workflow +3. Watch color-coded journal events stream in the observer pane +4. In the control pane, press Enter to hard-kill the cook process +5. Back in the cook pane, rerun: + + ```sh + node --experimental-strip-types demo/cook.ts + ``` + +You should see: + +- `Found N events in journal — replaying...` +- No new observer events during replay +- New events only after replay catches up and live execution resumes + +## Run Without tmux + +Open 2 terminals from `durable-streams/`: + +Terminal 1 (observer — server + tailer): + +```sh +DURABLE_STREAM_ID=my-stream node --experimental-strip-types demo/observe.ts +``` + +Terminal 2 (workflow): + +```sh +DURABLE_STREAM_ID=my-stream node --experimental-strip-types demo/cook.ts +``` + +Then kill and restart Terminal 2 to observe replay behavior. + +## Environment Variables + +- `DURABLE_STREAM_ID` (default: `dinner-demo`) diff --git a/durable-streams/demo/cook.ts b/durable-streams/demo/cook.ts new file mode 100644 index 00000000..1a357232 --- /dev/null +++ b/durable-streams/demo/cook.ts @@ -0,0 +1,188 @@ +/** + * Pane B: "Durable Dinner" cooking workflow. + * + * Usage: node --experimental-strip-types demo/cook.ts + * + * Connects to the Durable Streams server, runs a multi-dish cooking + * workflow using durableAll/durableRace with durableCall and durableSleep. + * + * Kill this process mid-cook using the control pane in demo/start.sh, then restart. + * The journal already has all completed checkpoints — replay produces zero + * new events, then live execution resumes seamlessly from the next step. + */ + +import { randomUUID } from "node:crypto"; +import { run } from "effection"; +import { + type Json, + type Workflow, + durableAll, + durableCall, + durableRace, + durableRun, + durableSleep, + useHttpDurableStream, +} from "../mod.ts"; + +// --------------------------------------------------------------------------- +// Config +// --------------------------------------------------------------------------- + +const SERVER_URL = process.env.DURABLE_SERVER_URL ?? "http://localhost:4437"; +const STREAM_ID = process.env.DURABLE_STREAM_ID ?? "dinner-demo"; + +// Fresh producerId each run — avoids seq/epoch bookkeeping in the demo +const PRODUCER_ID = `cook-${randomUUID().slice(0, 8)}`; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +/** Fake async work — simulate the named cooking step. */ +function fakeWork( + _name: string, + value: T, + ms = 50, +): () => Promise { + return async () => { + // Tiny delay to simulate real async I/O + await new Promise((r) => setTimeout(r, ms)); + return value; + }; +} + +function log(emoji: string, msg: string) { + const ts = new Date().toISOString().slice(11, 23); + console.log(` ${ts} ${emoji} ${msg}`); +} + +// --------------------------------------------------------------------------- +// Dish 1: Tomato Sauce (sequential steps + timers) +// --------------------------------------------------------------------------- + +function* makeSauce(): Workflow { + log("🧅", "Chopping onion..."); + yield* durableCall("chop-onion", fakeWork("chop-onion", "onion chopped")); + + log("🧅", "Sweating onions..."); + yield* durableSleep(1500); + + log("🍅", "Adding tomatoes..."); + yield* durableCall("add-tomato", fakeWork("add-tomato", "tomatoes in")); + + log("🍅", "Simmering sauce..."); + yield* durableSleep(2000); + + log("👅", "Tasting sauce..."); + const taste = yield* durableCall( + "taste-sauce", + fakeWork("taste-sauce", "perfetto"), + ); + log("👅", `Sauce verdict: ${taste}`); + + return "sauce-done"; +} + +// --------------------------------------------------------------------------- +// Dish 2: Focaccia (race: oven timer vs periodic check) +// --------------------------------------------------------------------------- + +function* bakeFocaccia(): Workflow { + log("🫒", "Mixing dough..."); + yield* durableCall("mix-dough", fakeWork("mix-dough", "dough mixed")); + + log("🫒", "Dimpling & topping..."); + yield* durableCall("dimple-top", fakeWork("dimple-top", "dimpled")); + + log("🔥", "Into the oven! Racing timer vs periodic check..."); + + // Race: oven timer (8s) vs periodic peek (every 2s, done after 3 peeks) + const winner = yield* durableRace([ + function* ovenTimer(): Workflow { + yield* durableSleep(8000); + log("⏱️", "Oven timer went off!"); + return "timer-done"; + }, + function* periodicCheck(): Workflow { + let peeks = 0; + while (true) { + yield* durableSleep(2000); + peeks++; + const look = yield* durableCall( + `peek-${peeks}`, + fakeWork(`peek-${peeks}`, peeks >= 3 ? "golden" : "not-yet"), + ); + log("👀", `Peek #${peeks}: ${look}`); + if (look === "golden") { + return "looks-done"; + } + } + }, + ]); + + log("🍞", `Focaccia done! Winner: ${winner}`); + return "focaccia-done"; +} + +// --------------------------------------------------------------------------- +// Dish 3: Roasted Vegetables (short parallel branch) +// --------------------------------------------------------------------------- + +function* roastVeg(): Workflow { + log("🥕", "Prepping vegetables..."); + yield* durableCall("chop-veg", fakeWork("chop-veg", "veggies chopped")); + + log("🥕", "Tossing with oil & seasoning..."); + yield* durableCall("season-veg", fakeWork("season-veg", "seasoned")); + + log("🥕", "Roasting..."); + yield* durableSleep(3000); + + log("🥕", "Vegetables roasted!"); + return "veg-done"; +} + +// --------------------------------------------------------------------------- +// Main: cookDinner — all three dishes in parallel +// --------------------------------------------------------------------------- + +function* cookDinner(): Workflow { + log("👨‍🍳", "Starting dinner prep — 3 dishes in parallel!"); + + const results = yield* durableAll([makeSauce, bakeFocaccia, roastVeg]); + + log("🍽️", `All done! Results: ${results.join(", ")}`); + return "Dinner is served!"; +} + +// --------------------------------------------------------------------------- +// Entry point +// --------------------------------------------------------------------------- + +console.log(`\n Durable Dinner Demo`); +console.log(` ═══════════════════`); +console.log(` Server: ${SERVER_URL}`); +console.log(` Stream: ${STREAM_ID}`); +console.log(` Producer: ${PRODUCER_ID}`); +console.log(); + +try { + const result = await run(function* () { + const stream = yield* useHttpDurableStream({ + baseUrl: SERVER_URL, + streamId: STREAM_ID, + producerId: PRODUCER_ID, + epoch: 1, + }); + + return yield* durableRun(cookDinner, { stream }); + }); + + console.log(); + console.log(` ✅ ${result}`); + console.log(); +} catch (err) { + const error = err instanceof Error ? err : new Error(String(err)); + console.error("\n ❌ Workflow failed:", error.message); + process.exit(1); +} diff --git a/durable-streams/demo/observe.ts b/durable-streams/demo/observe.ts new file mode 100644 index 00000000..3db7790d --- /dev/null +++ b/durable-streams/demo/observe.ts @@ -0,0 +1,183 @@ +/** + * Durable Streams observer: server + live journal tailer in one process. + * + * Usage: node --experimental-strip-types demo/observe.ts + * + * Starts a DurableStreamTestServer and tails the journal via SSE, printing + * color-coded events as they arrive. Ctrl+C triggers clean shutdown. + * + * #1 yield root.0 call(chop-onion) ok "onion chopped" + * #5 yield root.0 sleep(sleep) ok + * #18 close root.1.1 cancelled + * #22 close root ok "Dinner is served!" + */ + +import { FetchError, stream as fetchStream } from "@durable-streams/client"; +import { DurableStreamTestServer } from "@durable-streams/server"; +import { call, createChannel, each, main, resource, spawn } from "effection"; +import type { Operation, Stream } from "effection"; +import type { DurableEvent } from "../mod.ts"; + +// --------------------------------------------------------------------------- +// ANSI helpers +// --------------------------------------------------------------------------- + +const c = { + reset: "\x1b[0m", + bold: "\x1b[1m", + dim: "\x1b[2m", + + // foreground + red: "\x1b[31m", + green: "\x1b[32m", + yellow: "\x1b[33m", + blue: "\x1b[34m", + magenta: "\x1b[35m", + cyan: "\x1b[36m", + white: "\x1b[37m", + gray: "\x1b[90m", +}; + +// --------------------------------------------------------------------------- +// Config +// --------------------------------------------------------------------------- + +const STREAM_ID = process.env.DURABLE_STREAM_ID ?? "dinner-demo"; + +// --------------------------------------------------------------------------- +// Resources +// --------------------------------------------------------------------------- + +function useDurableStreamTestServer(): Operation { + return resource(function* (provide) { + const server = new DurableStreamTestServer(); + yield* call(() => server.start()); + try { + yield* provide(server); + } finally { + yield* call(() => server.stop()); + } + }); +} + +interface TailOptions { + url: string; + offset?: string; +} + +function useDurableStreamTail(opts: TailOptions): Stream { + return resource(function* (provide) { + const channel = createChannel(); + + const res = yield* call(() => + fetchStream({ + url: opts.url, + offset: opts.offset ?? "-1", + live: true, + onError: async (error) => { + if (error instanceof FetchError && error.status === 404) { + await new Promise((r) => setTimeout(r, 500)); + return {}; + } + return undefined; + }, + }), + ); + + yield* spawn(function* () { + const reader = res.jsonStream().getReader(); + try { + while (true) { + const { done, value } = yield* call(() => reader.read()); + if (done) break; + yield* channel.send(value); + } + } finally { + reader.releaseLock(); + res.cancel(); + } + }); + + yield* provide(yield* channel); + }); +} + +// --------------------------------------------------------------------------- +// Formatting +// --------------------------------------------------------------------------- + +function statusColor(status: string): string { + switch (status) { + case "ok": + return c.green; + case "err": + return c.red; + case "cancelled": + return c.yellow; + default: + return c.white; + } +} + +function typeColor(type: string): string { + return type === "yield" ? c.cyan : c.magenta; +} + +function formatEvent(n: number, event: DurableEvent): string { + const idx = `${c.dim}#${String(n).padEnd(3)}${c.reset}`; + const type = `${typeColor(event.type)}${event.type.padEnd(5)}${c.reset}`; + const cid = `${c.blue}${event.coroutineId.padEnd(14)}${c.reset}`; + + const status = event.result.status; + const styledStatus = `${statusColor(status)}${status}${c.reset}`; + + if (event.type === "yield") { + const desc = `${event.description.type}(${c.bold}${event.description.name}${c.reset})`; + const descPad = desc + " ".repeat(Math.max(0, 22 - plainLength(desc))); + const val = + event.result.status === "ok" && event.result.value !== undefined + ? ` ${c.dim}${JSON.stringify(event.result.value)}${c.reset}` + : ""; + return ` ${idx} ${type} ${cid} ${descPad} ${styledStatus}${val}`; + } + + // close event + const pad = " ".repeat(22); + const val = + event.result.status === "ok" && event.result.value !== undefined + ? ` ${c.dim}${JSON.stringify(event.result.value)}${c.reset}` + : ""; + return ` ${idx} ${type} ${cid} ${pad} ${styledStatus}${val}`; +} + +/** Length of a string without ANSI escape codes. */ +function plainLength(s: string): number { + // biome-ignore lint/suspicious/noControlCharactersInRegex: stripping ANSI escapes + return s.replace(/\x1b\[[0-9;]*m/g, "").length; +} + +// --------------------------------------------------------------------------- +// Entry point +// --------------------------------------------------------------------------- + +await main(function* () { + const server = yield* useDurableStreamTestServer(); + const streamUrl = `${server.url}/${STREAM_ID}`; + + console.log(); + console.log(` ${c.bold}${c.cyan}Durable Streams Observer${c.reset}`); + console.log(` ${c.dim}${"═".repeat(24)}${c.reset}`); + console.log(` ${c.dim}Server${c.reset} ${server.url}`); + console.log(` ${c.dim}Stream${c.reset} ${STREAM_ID}`); + console.log(` ${c.dim}Mode${c.reset} SSE/live`); + console.log(); + console.log(` ${c.dim}Watching for events...${c.reset}`); + console.log(); + + let eventCount = 0; + for (const event of yield* each(useDurableStreamTail({ url: streamUrl }))) { + eventCount++; + console.log(formatEvent(eventCount, event)); + yield* each.next(); + } +}); diff --git a/durable-streams/demo/start.sh b/durable-streams/demo/start.sh new file mode 100755 index 00000000..0900adf6 --- /dev/null +++ b/durable-streams/demo/start.sh @@ -0,0 +1,98 @@ +#!/usr/bin/env bash +# +# Durable Dinner — tmux demo launcher +# +# Starts a 3-pane tmux session: +# +# ┌─────────────────────┬──────────────────┐ +# │ │ Cook (focused) │ 80% +# │ Observer ├──────────────────┤ +# │ (server + journal) │ Control │ 20% +# │ │ (kill cmd) │ +# └─────────────────────┴──────────────────┘ +# +# Usage: +# ./demo/start.sh [stream-id] # launch with optional stream ID +# +# The cook pane (top-right) is focused with the command pre-typed. +# The control pane (bottom-right) has the kill command pre-typed. +# Press Enter in each when ready. + +set -euo pipefail + +SESSION="durable-dinner" +DIR="$(cd "$(dirname "$0")/.." && pwd)" +STREAM_ID="${1:-dinner-demo}" +NODE="node --experimental-strip-types" + +# Export so all panes inherit it +export DURABLE_STREAM_ID="$STREAM_ID" + +# ------------------------------------------------------------------ +# Preflight +# ------------------------------------------------------------------ + +if ! command -v tmux &>/dev/null; then + echo "Error: tmux is not installed. Install it with: brew install tmux" >&2 + exit 1 +fi + +if ! command -v node &>/dev/null; then + echo "Error: node is not installed. See https://nodejs.org" >&2 + exit 1 +fi + +# Kill any leftover session (idempotent) +tmux kill-session -t "$SESSION" 2>/dev/null || true + +# ------------------------------------------------------------------ +# Build the layout +# ------------------------------------------------------------------ + +# Pane 0 (left): Observer — full height +tmux new-session -d -s "$SESSION" -c "$DIR" -x 200 -y 50 + +# Set stream ID as a tmux environment variable +tmux set-environment -t "$SESSION" DURABLE_STREAM_ID "$STREAM_ID" + +# Split vertically — right side gets 50% width (cook + control) +tmux split-window -h -t "$SESSION" -c "$DIR" -p 50 + +# Split the right pane horizontally — bottom 20% becomes control +tmux split-window -v -t "${SESSION}:0.1" -c "$DIR" -p 20 + +# After all splits, pane indices are: +# 0 = left (Observer) +# 1 = top-right (Cook) +# 2 = bottom-right (Control) + +# ------------------------------------------------------------------ +# Start processes +# ------------------------------------------------------------------ + +# Pane 0: Start the observer (server + tailer) +tmux send-keys -t "${SESSION}:0.0" "$NODE demo/observe.ts" Enter + +# Give the server a moment to bind its port +sleep 2 + +# Pane 1: Pre-type the cook command (presenter hits Enter when ready) +tmux send-keys -t "${SESSION}:0.1" "$NODE demo/cook.ts" + +# Pane 2: Pre-type a kill command targeting only the cook pane's process tree. +# Uses tmux's pane_pid to scope the kill to this session instead of a global pkill. +tmux send-keys -t "${SESSION}:0.2" "kill -9 \$(tmux display-message -t '${SESSION}:0.1' -p '#{pane_pid}' | xargs -I{} pgrep -P {})" + +# ------------------------------------------------------------------ +# Focus & attach +# ------------------------------------------------------------------ + +# Focus the cook pane (top-right) +tmux select-pane -t "${SESSION}:0.1" + +# Attach (or switch if already inside tmux) +if [ -n "${TMUX:-}" ]; then + tmux switch-client -t "$SESSION" +else + tmux attach-session -t "$SESSION" +fi diff --git a/durable-streams/deterministic-id.test.ts b/durable-streams/deterministic-id.test.ts new file mode 100644 index 00000000..10cfbd2c --- /dev/null +++ b/durable-streams/deterministic-id.test.ts @@ -0,0 +1,307 @@ +/** + * Tier 4 tests — deterministic identity. + * + * Tests 24-27 from the protocol specification. These validate that + * coroutine IDs are stable and deterministic across live vs. replay runs, + * and that the structured concurrency combinators produce consistent IDs. + */ + +import { describe, it } from "@effectionx/bdd"; +import { expect } from "expect"; +import { + type DurableEvent, + InMemoryStream, + type Json, + durableAll, + durableCall, + durableRace, + durableRun, +} from "./mod.ts"; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +/** Extract all unique coroutine IDs from events, sorted. */ +function coroutineIds(events: DurableEvent[]): string[] { + return [...new Set(events.map((e) => e.coroutineId))].sort(); +} + +/** Extract the event types and coroutine IDs as a compact trace. */ +function eventTrace(events: DurableEvent[]): string[] { + return events.map((e) => { + if (e.type === "yield") { + return `yield:${e.coroutineId}:${e.description.type}(${e.description.name})`; + } + return `close:${e.coroutineId}:${e.result.status}`; + }); +} + +function createCallTracker() { + const calls: string[] = []; + return { + calls, + fn(name: string, value: T): () => Promise { + return () => { + calls.push(name); + return Promise.resolve(value); + }; + }, + }; +} + +describe("deterministic IDs", () => { + // --------------------------------------------------------------------------- + // Test 24: Stable IDs across runs — same inputs produce same IDs + // --------------------------------------------------------------------------- + + it("same workflow produces same coroutine IDs across two live runs", function* () { + function makeWorkflow(tracker: ReturnType) { + return function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("fetchA", tracker.fn("fetchA", "alpha")); + }, + function* () { + return yield* durableCall("fetchB", tracker.fn("fetchB", "beta")); + }, + ]); + return results.join("-"); + }; + } + + // Run 1 + const stream1 = new InMemoryStream(); + const tracker1 = createCallTracker(); + yield* durableRun(makeWorkflow(tracker1), { stream: stream1 }); + const events1 = stream1.snapshot(); + + // Run 2 + const stream2 = new InMemoryStream(); + const tracker2 = createCallTracker(); + yield* durableRun(makeWorkflow(tracker2), { stream: stream2 }); + const events2 = stream2.snapshot(); + + // Coroutine IDs must be identical + expect(coroutineIds(events1)).toEqual(coroutineIds(events2)); + + // Event traces must be identical (same types, same coroutineIds, same descriptions) + expect(eventTrace(events1)).toEqual(eventTrace(events2)); + }); + + // --------------------------------------------------------------------------- + // Test 25: Stable IDs: live vs. replay + // --------------------------------------------------------------------------- + + it("live run and replay produce identical coroutine IDs", function* () { + // Live run + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + yield* durableRun( + function* () { + const a = yield* durableCall("step1", tracker.fn("step1", "one")); + const results = yield* durableAll([ + function* () { + return yield* durableCall("childA", tracker.fn("childA", "alpha")); + }, + function* () { + return yield* durableCall("childB", tracker.fn("childB", "beta")); + }, + ]); + return `${a}-${results.join(",")}`; + }, + { stream }, + ); + + const liveEvents = stream.snapshot(); + const liveIds = coroutineIds(liveEvents); + + // Replay run — same stream, no effects should execute + const replayStream = new InMemoryStream(liveEvents); + const tracker2 = createCallTracker(); + + yield* durableRun( + function* () { + const a = yield* durableCall("step1", tracker2.fn("step1", "WRONG")); + const results = yield* durableAll([ + function* () { + return yield* durableCall("childA", tracker2.fn("childA", "WRONG")); + }, + function* () { + return yield* durableCall("childB", tracker2.fn("childB", "WRONG")); + }, + ]); + return `${a}-${results.join(",")}`; + }, + { stream: replayStream }, + ); + + // No effects re-executed during replay + expect(tracker2.calls).toEqual([]); + + // Since it's a full replay (root has Close), the replay returns the + // stored result directly without generating new events. The coroutine + // IDs from the original run are what matter. + expect(liveIds.length > 0).toBe(true); + + // Verify expected IDs: root, root.0, root.1 + expect(liveIds).toEqual(["root", "root.0", "root.1"]); + }); + + // --------------------------------------------------------------------------- + // Test 26: Nested scope IDs are stable + // --------------------------------------------------------------------------- + + it("nested all produces hierarchical IDs", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + // root.0 has its own nested all + const inner = yield* durableAll([ + function* () { + return yield* durableCall("deep1", tracker.fn("deep1", "d1")); + }, + function* () { + return yield* durableCall("deep2", tracker.fn("deep2", "d2")); + }, + ]); + return inner.join("+") as string; + }, + function* () { + return yield* durableCall("shallow", tracker.fn("shallow", "s")); + }, + ]); + return results.join("-"); + }, + { stream }, + ); + + const events = stream.snapshot(); + const ids = coroutineIds(events); + + // root.0 is the first child of the outer all + // root.0.0 and root.0.1 are children of the inner all (inside root.0) + // root.1 is the second child of the outer all + expect(ids).toEqual(["root", "root.0", "root.0.0", "root.0.1", "root.1"]); + }); + + // --------------------------------------------------------------------------- + // Test 27: Dynamic spawn count divergence + // --------------------------------------------------------------------------- + + it("changing child count produces divergence on replay", function* () { + // Golden run with 2 children + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("childA", tracker.fn("childA", "a")); + }, + function* () { + return yield* durableCall("childB", tracker.fn("childB", "b")); + }, + ]); + // After the all(), do another call to verify the sequential effect + const after = yield* durableCall("after", tracker.fn("after", "z")); + return `${results.join(",")}-${after}`; + }, + { stream }, + ); + + // Full replay returns stored result (root has Close), so divergence + // from changing child count isn't detected on full replay. + // For partial replay: strip the root Close and the "after" yield. + const allEvents = stream.snapshot(); + + // Keep only: child yields + child closes (no root close, no "after" yield) + const partialEvents = allEvents.filter((e) => { + if (e.coroutineId === "root") return false; + return true; + }); + + const partialStream = new InMemoryStream(partialEvents); + + // Now replay with 3 children instead of 2 — the third child (root.2) + // will execute live since it has no journal entries. But the "after" + // step will try to replay with journal entry for root, which should + // now be different since root.2 has new events. + // Actually, root.2 will just execute live (no journal entries for it). + // The divergence happens only if the post-join effect's description + // mismatches. + const tracker2 = createCallTracker(); + + // With 3 children, root's childCounter goes to 3, but journal has + // root.0 and root.1 Close events. root.2 is new (no journal). + // This should work without divergence — the third child just executes live. + const result = yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("childA", tracker2.fn("childA", "WRONG")); + }, + function* () { + return yield* durableCall("childB", tracker2.fn("childB", "WRONG")); + }, + function* () { + return yield* durableCall("childC", tracker2.fn("childC", "c")); + }, + ]); + const after = yield* durableCall("after", tracker2.fn("after", "z")); + return `${results.join(",")}-${after}`; + }, + { stream: partialStream }, + ); + + // Children 0 and 1 replayed, child 2 executed live, "after" executed live + expect(result).toBe("a,b,c-z"); + expect(tracker2.calls).toEqual(["childC", "after"]); + }); + + // --------------------------------------------------------------------------- + // Test: Race coroutine IDs are stable + // --------------------------------------------------------------------------- + + it("race children get sequential IDs", function* () { + const stream = new InMemoryStream(); + + yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + return yield* durableCall("fast", () => Promise.resolve("winner")); + }, + function* () { + return yield* durableCall( + "slow", + () => + new Promise(() => { + /* never */ + }), + ); + }, + ]); + }, + { stream }, + ); + + const events = stream.snapshot(); + + // Winner should be root.0 + const winnerYield = events.find( + (e) => e.type === "yield" && e.coroutineId === "root.0", + ); + expect(winnerYield !== undefined).toBe(true); + + // Verify IDs include root.0 (winner) + const ids = coroutineIds(events); + expect(ids.includes("root.0")).toBe(true); + }); +}); diff --git a/durable-streams/divergence-api.test.ts b/durable-streams/divergence-api.test.ts new file mode 100644 index 00000000..3534b049 --- /dev/null +++ b/durable-streams/divergence-api.test.ts @@ -0,0 +1,250 @@ +/** + * Divergence API tests — pluggable policy via createApi() middleware. + * + * Tests that the Divergence API correctly delegates divergence decisions + * and that middleware can override default strict behavior. See DEC-031. + * + * Since durableRun is now an Operation (DEC-032), middleware is + * installed by the caller's scope before yield*-ing into durableRun. + * Tests use a wrapper Operation that calls useScope(), installs + * middleware via scope.around(), then yield*s into durableRun. + */ + +import { describe, it } from "@effectionx/bdd"; +import { call, run, useScope } from "effection"; +import type { Operation } from "effection"; +import { expect } from "expect"; +import { + ContinuePastCloseDivergenceError, + Divergence, + type DivergenceDecision, + DivergenceError, + type DurableEvent, + InMemoryStream, + type Workflow, + durableCall, + durableRun, +} from "./mod.ts"; + +describe("Divergence API", () => { + // --------------------------------------------------------------------------- + // Test 1: Default strict — description mismatch → DivergenceError + // --------------------------------------------------------------------------- + + it("default strict — description mismatch throws DivergenceError", function* () { + // Journal has call("stepA"), code yields call("stepX") + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + return yield* durableCall("stepX", () => + Promise.resolve("x"), + ); + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("DivergenceError"); + if (e instanceof DivergenceError) { + expect(e.expected).toEqual({ type: "call", name: "stepA" }); + expect(e.actual).toEqual({ type: "call", name: "stepX" }); + } + } + }); + + // --------------------------------------------------------------------------- + // Test 2: Default strict — continue past close → ContinuePastCloseDivergenceError + // --------------------------------------------------------------------------- + + it("default strict — continue past close throws ContinuePastCloseDivergenceError", function* () { + const scope = yield* useScope(); + const decision = Divergence.invoke(scope, "decide", [ + { kind: "continue-past-close", coroutineId: "root.0", yieldCount: 2 }, + ]); + + expect(decision.type).toBe("throw"); + if (decision.type === "throw") { + expect(decision.error).toBeInstanceOf(ContinuePastCloseDivergenceError); + } + }); + + // --------------------------------------------------------------------------- + // Test 3: Middleware override — mismatch → run-live → continues with new effect + // --------------------------------------------------------------------------- + + it("middleware override — mismatch triggers run-live and executes new effect", function* () { + // Journal has call("stepA") then call("stepB"). + // Code changes stepB to stepX. + // Middleware overrides divergence to run-live for description mismatches. + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + ]; + const stream = new InMemoryStream(events); + + const liveCalls: string[] = []; + + // Install divergence middleware on the caller's scope + const scope = yield* useScope(); + scope.around(Divergence, { + decide([info], next) { + if (info.kind === "description-mismatch") { + return { type: "run-live" } as DivergenceDecision; + } + return next(info); + }, + }); + + // Now yield* into durableRun — it inherits the scope with middleware + const result = yield* durableRun( + function* (): Workflow { + // stepA matches journal — replayed + const a = yield* durableCall("stepA", () => { + liveCalls.push("stepA"); + return Promise.resolve("alpha-live"); + }); + + // stepB was renamed to stepX — divergence detected, middleware returns run-live + const x = yield* durableCall("stepX", () => { + liveCalls.push("stepX"); + return Promise.resolve("x-live"); + }); + + return `${a}-${x}`; + }, + { stream }, + ); + + // stepA was replayed (got stored value "alpha"), stepX ran live + expect(result).toBe("alpha-x-live"); + // stepA should NOT have been called live; stepX should have been + expect(liveCalls).toEqual(["stepX"]); + }); + + // --------------------------------------------------------------------------- + // Test 4: Middleware is per-scope — two runs, only one with middleware + // --------------------------------------------------------------------------- + + it("middleware is per-scope — only the configured run tolerates divergence", function* () { + // Same journal for both runs + const makeEvents = (): DurableEvent[] => [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + ]; + + // Run 1: WITH middleware — should succeed with run-live + const stream1 = new InMemoryStream(makeEvents()); + const scope1 = yield* useScope(); + scope1.around(Divergence, { + decide([info], next) { + if (info.kind === "description-mismatch") { + return { type: "run-live" } as DivergenceDecision; + } + return next(info); + }, + }); + + const result1 = yield* durableRun( + function* (): Workflow { + return yield* durableCall("stepX", () => + Promise.resolve("x-live"), + ); + }, + { stream: stream1 }, + ); + expect(result1).toBe("x-live"); + + // Run 2: WITHOUT middleware on a fresh scope — should throw DivergenceError. + const stream2 = new InMemoryStream(makeEvents()); + try { + yield* call(() => + run(() => + durableRun( + function* (): Workflow { + return yield* durableCall("stepX", () => + Promise.resolve("x-live"), + ); + }, + { stream: stream2 }, + ), + ), + ); + throw new Error("expected strict divergence error without middleware"); + } catch (e) { + expect(e).toBeInstanceOf(DivergenceError); + } + }); + + // --------------------------------------------------------------------------- + // Test 5: No regression — replay feeds stored results when matching + // --------------------------------------------------------------------------- + + it("no regression — replay still feeds stored results when descriptions match", function* () { + // Full journal with Close — should replay without any live execution + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + { + type: "close", + coroutineId: "root", + result: { status: "ok", value: "alpha-beta" }, + }, + ]; + const stream = new InMemoryStream(events); + const liveCalls: string[] = []; + + const result = yield* durableRun( + function* (): Workflow { + const a = yield* durableCall("stepA", () => { + liveCalls.push("stepA"); + return Promise.resolve("should-not-be-called"); + }); + const b = yield* durableCall("stepB", () => { + liveCalls.push("stepB"); + return Promise.resolve("should-not-be-called"); + }); + return `${a}-${b}`; + }, + { stream }, + ); + + // Full replay returns stored Close result, no live calls + expect(result).toBe("alpha-beta"); + expect(liveCalls).toEqual([]); + }); +}); diff --git a/durable-streams/divergence.test.ts b/durable-streams/divergence.test.ts new file mode 100644 index 00000000..fdc4db43 --- /dev/null +++ b/durable-streams/divergence.test.ts @@ -0,0 +1,396 @@ +/** + * Tier 2 tests — divergence detection. + * + * Tests 8-14 from the protocol specification. These validate that + * durableRun correctly detects when the workflow code has changed + * in ways incompatible with the stored journal. + */ + +import { describe, it } from "@effectionx/bdd"; +import { expect } from "expect"; +import { + ContinuePastCloseDivergenceError, + DivergenceError, + type DurableEvent, + EarlyReturnDivergenceError, + InMemoryStream, + type Workflow, + durableCall, + durableRun, + durableSleep, +} from "./mod.ts"; + +describe("divergence detection", () => { + // --------------------------------------------------------------------------- + // Test 8: Added step divergence + // --------------------------------------------------------------------------- + + it("added step — generator yields more effects than journal (completed workflow stays completed)", function* () { + // Journal recorded a workflow with 2 steps, but now code has 3 + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + { + type: "close", + coroutineId: "root", + result: { status: "ok", value: "done" }, + }, + ]; + const stream = new InMemoryStream(events); + + // This workflow has a Close event, so durableRun returns stored result + // directly. The added step isn't detected because the workflow is never + // re-run. This is correct: a completed workflow stays completed. + const result = yield* durableRun( + function* (): Workflow { + yield* durableCall("stepA", () => Promise.resolve("alpha")); + yield* durableCall("stepNew", () => Promise.resolve("new")); + yield* durableCall("stepB", () => Promise.resolve("beta")); + return "done"; + }, + { stream }, + ); + + expect(result).toBe("done"); + }); + + it("added step — detected during partial replay", function* () { + // Journal has 2 steps but NO Close. The new code inserts a step between them. + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + yield* durableCall("stepA", () => Promise.resolve("alpha")); + // This step wasn't in the journal — journal[1] is stepB, not stepNew + yield* durableCall("stepNew", () => Promise.resolve("new")); + yield* durableCall("stepB", () => Promise.resolve("beta")); + return "done"; + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("DivergenceError"); + } + }); + + // --------------------------------------------------------------------------- + // Test 9: Removed step divergence + // --------------------------------------------------------------------------- + + it("removed step — generator finishes before journal exhausted", function* () { + // Journal has 3 steps, but new code only has 2 + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepC" }, + result: { status: "ok", value: "gamma" }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + yield* durableCall("stepA", () => Promise.resolve("alpha")); + yield* durableCall("stepB", () => Promise.resolve("beta")); + // stepC was removed — generator returns early + return "done"; + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("EarlyReturnDivergenceError"); + if (e instanceof EarlyReturnDivergenceError) { + expect(e.consumedCount).toBe(2); + expect(e.totalCount).toBe(3); + } + } + }); + + // --------------------------------------------------------------------------- + // Test 10: Reordered steps divergence + // --------------------------------------------------------------------------- + + it("reordered steps — description mismatch at position", function* () { + // Journal: stepA then stepB. Code: stepB then stepA. + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + // Reordered: stepB first, but journal has stepA at position 0 + yield* durableCall("stepB", () => Promise.resolve("beta")); + yield* durableCall("stepA", () => Promise.resolve("alpha")); + return "done"; + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("DivergenceError"); + if (e instanceof DivergenceError) { + expect(e.position).toBe(0); + expect(e.expected).toEqual({ type: "call", name: "stepA" }); + expect(e.actual).toEqual({ type: "call", name: "stepB" }); + } + } + }); + + // --------------------------------------------------------------------------- + // Test 11: Type mismatch divergence + // --------------------------------------------------------------------------- + + it("type mismatch — call vs sleep", function* () { + // Journal recorded a "call" effect, but code now yields a "sleep" + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + // Journal has call("stepA"), but we yield sleep("sleep") + yield* durableSleep(1000); + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("DivergenceError"); + if (e instanceof DivergenceError) { + expect(e.expected.type).toBe("call"); + expect(e.actual.type).toBe("sleep"); + } + } + }); + + // --------------------------------------------------------------------------- + // Test 12: Name mismatch divergence + // --------------------------------------------------------------------------- + + it("name mismatch — same type, different name", function* () { + // Journal has call("fetchOrder"), code has call("fetchUser") + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "fetchOrder" }, + result: { status: "ok", value: "order-data" }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + return yield* durableCall("fetchUser", () => + Promise.resolve("user-data"), + ); + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("DivergenceError"); + if (e instanceof DivergenceError) { + expect(e.expected).toEqual({ type: "call", name: "fetchOrder" }); + expect(e.actual).toEqual({ type: "call", name: "fetchUser" }); + } + } + }); + + // --------------------------------------------------------------------------- + // Test 13: Generator finishes early divergence + // --------------------------------------------------------------------------- + + it("generator finishes early — returns with unconsumed yields", function* () { + // Journal has 3 yields, generator returns after consuming only 1 + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepC" }, + result: { status: "ok", value: "gamma" }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + const a = yield* durableCall("stepA", () => + Promise.resolve("alpha"), + ); + // Steps B and C were removed + return a; + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("EarlyReturnDivergenceError"); + if (e instanceof EarlyReturnDivergenceError) { + expect(e.consumedCount).toBe(1); + expect(e.totalCount).toBe(3); + } + } + }); + + // --------------------------------------------------------------------------- + // Test 14: Generator continues past close divergence + // --------------------------------------------------------------------------- + + it("continues past close — journal has Close but generator keeps yielding (completed workflow stays completed)", function* () { + // Journal: 1 yield + Close. Code adds a second step. + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "close", + coroutineId: "root", + result: { status: "ok", value: "alpha" }, + }, + ]; + const stream = new InMemoryStream(events); + + // The workflow already has a Close event, so durableRun returns stored + // result directly. The new step isn't detected. + // This is the correct behavior: a completed workflow stays completed. + const result = yield* durableRun( + function* (): Workflow { + yield* durableCall("stepA", () => Promise.resolve("alpha")); + yield* durableCall("stepB", () => Promise.resolve("beta")); + return "done"; + }, + { stream }, + ); + + expect(result).toBe("alpha"); + }); + + it("ContinuePastCloseDivergenceError can be constructed", function* () { + // Verify the error class exists and can be constructed. + const err = new ContinuePastCloseDivergenceError("root.0", 2); + expect(err.name).toBe("ContinuePastCloseDivergenceError"); + expect(err.coroutineId).toBe("root.0"); + expect(err.yieldCount).toBe(2); + }); + + // --------------------------------------------------------------------------- + // Additional divergence: action vs call type mismatch + // --------------------------------------------------------------------------- + + it("action type mismatch — action vs call", function* () { + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "action", name: "doSomething" }, + result: { status: "ok", value: 42 }, + }, + ]; + const stream = new InMemoryStream(events); + + try { + yield* durableRun( + function* (): Workflow { + // Journal has action("doSomething"), code has call("doSomething") + return yield* durableCall("doSomething", () => + Promise.resolve(42), + ); + }, + { stream }, + ); + throw new Error("expected divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).name).toBe("DivergenceError"); + if (e instanceof DivergenceError) { + expect(e.expected.type).toBe("action"); + expect(e.actual.type).toBe("call"); + } + } + }); +}); diff --git a/durable-streams/divergence.ts b/durable-streams/divergence.ts new file mode 100644 index 00000000..7016af3c --- /dev/null +++ b/durable-streams/divergence.ts @@ -0,0 +1,136 @@ +/** + * Divergence API — pluggable policy for handling replay mismatches. + * + * When a durable effect's description doesn't match the replay index + * during replay, or when a generator continues to yield effects past + * a recorded Close event, a divergence is detected. + * + * By default, divergence is fatal (throws DivergenceError). Users can + * override this behavior per-scope via Effection's around() middleware + * to implement custom policies (e.g., switching to live execution). + * + * Uses createApi() from @effection/effection/experimental to get + * proper middleware dispatch with caching and invalidation. The + * circular initialization bug that prevented this in alpha.5 was + * fixed in alpha.6 (see DEC-031). + * + * The core decide() function is synchronous (not a generator) because + * it is called from inside Effect.enter(), which is a synchronous + * callback. createApi().invoke() dispatches synchronously, so this + * is safe. See DEC-031. + */ + +import type { Api } from "effection"; +import { createApi } from "effection/experimental"; +import { ContinuePastCloseDivergenceError, DivergenceError } from "./errors.ts"; +import type { CoroutineId, EffectDescription } from "./types.ts"; + +// --------------------------------------------------------------------------- +// Types +// --------------------------------------------------------------------------- + +/** The two kinds of divergence detected during replay. */ +export type DivergenceKind = "description-mismatch" | "continue-past-close"; + +/** + * Information about a detected divergence. + * + * Discriminated union on `kind` so that TypeScript enforces correct + * field access per variant. + */ +export type DivergenceInfo = + | { + kind: "description-mismatch"; + coroutineId: CoroutineId; + /** Cursor position (yield index) where divergence was detected. */ + cursor: number; + /** The description from the journal (what was expected). */ + expected: EffectDescription; + /** The description from the generator (what was actually yielded). */ + actual: EffectDescription; + } + | { + kind: "continue-past-close"; + coroutineId: CoroutineId; + /** Number of yield entries recorded for this coroutine. */ + yieldCount: number; + }; + +/** + * The policy decision returned by the Divergence API. + * + * - "throw": Fail the workflow with the provided error (default behavior). + * - "run-live": Disable replay for this coroutine and execute live from + * this point forward. Previous replay entries are ignored. + */ +export type DivergenceDecision = + | { type: "throw"; error: Error } + | { type: "run-live" }; + +// --------------------------------------------------------------------------- +// API shape (synchronous — not generator-based) +// --------------------------------------------------------------------------- + +/** + * The core shape of the Divergence API. + * + * decide() is synchronous because it is called from Effect.enter(), + * which cannot yield. Middleware installed via scope.around() also + * runs synchronously in the chain. + * + * Usage from Effect.enter() (synchronous): + * Divergence.invoke(scope, "decide", [info]) + * + * Middleware installation (from a generator): + * scope.around(Divergence, { decide: ([info], next) => { ... } }) + */ +interface DivergenceApi { + decide(info: DivergenceInfo): DivergenceDecision; +} + +// --------------------------------------------------------------------------- +// Default policy (strict — all divergences are fatal) +// --------------------------------------------------------------------------- + +/** The default (strict) decide function. */ +function defaultDecide(info: DivergenceInfo): DivergenceDecision { + if (info.kind === "description-mismatch") { + return { + type: "throw", + error: new DivergenceError( + info.coroutineId, + info.cursor, + info.expected, + info.actual, + ), + }; + } else { + return { + type: "throw", + error: new ContinuePastCloseDivergenceError( + info.coroutineId, + info.yieldCount, + ), + }; + } +} + +// --------------------------------------------------------------------------- +// The Divergence API instance +// --------------------------------------------------------------------------- + +/** + * The Divergence API. + * + * Created via Effection's createApi() which provides proper middleware + * dispatch with WeakMap-based handle caching, automatic cache + * invalidation on scope.around(), and a fast-path that skips + * middleware dispatch entirely when no middleware is installed. + * + * Default behavior is strict: all divergences produce a throw decision + * with the appropriate error type. + */ +export const Divergence: Api = createApi( + "DurableEffection.Divergence", + { decide: defaultDecide }, +); diff --git a/durable-streams/durable-each.test.ts b/durable-streams/durable-each.test.ts new file mode 100644 index 00000000..12610d61 --- /dev/null +++ b/durable-streams/durable-each.test.ts @@ -0,0 +1,479 @@ +/** + * Tier 4 tests — durable iteration (durableEach). + * + * Validates that durableEach correctly journals each fetch, + * replays from the journal, detects advance guard violations, + * handles break/cancellation, and integrates with durableCall. + */ + +import { describe, it } from "@effectionx/bdd"; +import type { Operation } from "effection"; +import { expect } from "expect"; +import { + type DurableEvent, + type DurableSource, + InMemoryStream, + type Json, + durableCall, + durableEach, + durableRun, +} from "./mod.ts"; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +/** Create a DurableSource from an array of items. */ +function arraySource( + items: T[], +): DurableSource & { closed: boolean } { + let index = 0; + const src = { + closed: false, + *next(): Operation<{ value: T } | { done: true }> { + if (index < items.length) { + return { value: items[index++]! }; + } + return { done: true as const }; + }, + close() { + src.closed = true; + }, + }; + return src; +} + +/** Track which functions were actually called during live execution. */ +function createCallTracker() { + const calls: string[] = []; + return { + calls, + fn(name: string, value: T): () => Promise { + return () => { + calls.push(name); + return Promise.resolve(value); + }; + }, + }; +} + +describe("durableEach", () => { + // --------------------------------------------------------------------------- + // Test 1: Golden run — 3 items + // --------------------------------------------------------------------------- + + it("golden run — 3 items processed, correct journal", function* () { + const stream = new InMemoryStream(); + const source = arraySource(["a", "b", "c"]); + const processed: string[] = []; + + const result = yield* durableRun( + function* () { + for (const msg of yield* durableEach("queue", source)) { + processed.push(msg); + yield* durableEach.next(); + } + return "done"; + }, + { stream }, + ); + + expect(result).toBe("done"); + expect(processed).toEqual(["a", "b", "c"]); + + // Verify journal: 4 each events (a, b, c, done) + 1 root Close + const events = stream.snapshot(); + const yieldEvents = events.filter((e) => e.type === "yield"); + expect(yieldEvents.length).toBe(4); // 3 items + 1 done sentinel + + // Check each event description + for (const y of yieldEvents) { + if (y.type === "yield") { + expect(y.description).toEqual({ type: "each", name: "queue" }); + } + } + + // Check result values + if (yieldEvents[0]!.type === "yield") { + expect(yieldEvents[0]!.result).toEqual({ + status: "ok", + value: { value: "a" }, + }); + } + if (yieldEvents[1]!.type === "yield") { + expect(yieldEvents[1]!.result).toEqual({ + status: "ok", + value: { value: "b" }, + }); + } + if (yieldEvents[2]!.type === "yield") { + expect(yieldEvents[2]!.result).toEqual({ + status: "ok", + value: { value: "c" }, + }); + } + if (yieldEvents[3]!.type === "yield") { + expect(yieldEvents[3]!.result).toEqual({ + status: "ok", + value: { done: true }, + }); + } + + // Root Close event + const closeEvents = events.filter((e) => e.type === "close"); + expect(closeEvents.length).toBe(1); + expect(closeEvents[0]!.coroutineId).toBe("root"); + }); + + // --------------------------------------------------------------------------- + // Test 2: Empty source — loop body never executes + // --------------------------------------------------------------------------- + + it("empty source — loop body never executes", function* () { + const stream = new InMemoryStream(); + const source = arraySource([]); + const processed: string[] = []; + + const result = yield* durableRun( + function* () { + for (const msg of yield* durableEach("empty", source)) { + processed.push(msg); + yield* durableEach.next(); + } + return "done"; + }, + { stream }, + ); + + expect(result).toBe("done"); + expect(processed).toEqual([]); + + // Journal: 1 each event (done) + 1 root Close + const events = stream.snapshot(); + const yieldEvents = events.filter((e) => e.type === "yield"); + expect(yieldEvents.length).toBe(1); + if (yieldEvents[0]!.type === "yield") { + expect(yieldEvents[0]!.result).toEqual({ + status: "ok", + value: { done: true }, + }); + } + }); + + // --------------------------------------------------------------------------- + // Test 3: Full replay — no source calls, items replayed from journal + // --------------------------------------------------------------------------- + + it("full replay — items replayed from journal without calling source", function* () { + // Pre-populate stream with all yield events but NO root Close. + // durableRun will re-run the generator, but all DurableEffects resolve + // from the replay index — source.next() is never called. + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "each", name: "queue" }, + result: { status: "ok", value: { value: "x" } }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "each", name: "queue" }, + result: { status: "ok", value: { value: "y" } }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "each", name: "queue" }, + result: { status: "ok", value: { done: true } }, + }, + ]; + const stream = new InMemoryStream(events); + + // Source should never be called during replay + let sourceCalled = false; + const source: DurableSource = { + *next(): Operation<{ value: string } | { done: true }> { + sourceCalled = true; + return { done: true as const }; + }, + }; + const processed: string[] = []; + + const result = yield* durableRun( + function* () { + for (const msg of yield* durableEach("queue", source)) { + processed.push(msg); + yield* durableEach.next(); + } + return "done"; + }, + { stream }, + ); + + // Generator ran but all effects were replayed from journal + expect(result).toBe("done"); + expect(processed).toEqual(["x", "y"]); + expect(sourceCalled).toBe(false); + }); + + // --------------------------------------------------------------------------- + // Test 4: Crash recovery (partial replay) + // --------------------------------------------------------------------------- + + it("crash recovery — partial replay then live", function* () { + // Journal has 2 items replayed, 3rd will be live + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "each", name: "queue" }, + result: { status: "ok", value: { value: "a" } }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "each", name: "queue" }, + result: { status: "ok", value: { value: "b" } }, + }, + ]; + const stream = new InMemoryStream(events); + + // Source should only be called for the 3rd item onward + const sourceItems = ["a", "b", "c"]; // source has all items but replay covers a, b + let sourceCallCount = 0; + let sourceIndex = 2; // start from where replay left off + const source: DurableSource = { + *next(): Operation<{ value: string } | { done: true }> { + sourceCallCount++; + if (sourceIndex < sourceItems.length) { + return { value: sourceItems[sourceIndex++]! }; + } + return { done: true as const }; + }, + }; + const processed: string[] = []; + + const result = yield* durableRun( + function* () { + for (const msg of yield* durableEach("queue", source)) { + processed.push(msg); + yield* durableEach.next(); + } + return "done"; + }, + { stream }, + ); + + expect(result).toBe("done"); + expect(processed).toEqual(["a", "b", "c"]); + // Source called twice: once for "c", once for the done sentinel + expect(sourceCallCount).toBe(2); + }); + + // --------------------------------------------------------------------------- + // Test 5: With durableCall in loop body — interleaved events + // --------------------------------------------------------------------------- + + it("with durableCall in loop — interleaved journal events", function* () { + const stream = new InMemoryStream(); + const source = arraySource(["msg1", "msg2"]); + const tracker = createCallTracker(); + + yield* durableRun( + function* () { + for (const msg of yield* durableEach("queue", source)) { + yield* durableCall( + `process-${msg}`, + tracker.fn(`process-${msg}`, null), + ); + yield* durableEach.next(); + } + }, + { stream }, + ); + + expect(tracker.calls).toEqual(["process-msg1", "process-msg2"]); + + // Verify interleaved journal structure + const events = stream.snapshot(); + const nonClose = events.filter((e) => e.type === "yield"); + + // each(msg1), call(process-msg1), each(msg2), call(process-msg2), each(done) + expect(nonClose.length).toBe(5); + if (nonClose[0]!.type === "yield") { + expect(nonClose[0]!.description).toEqual({ type: "each", name: "queue" }); + } + if (nonClose[1]!.type === "yield") { + expect(nonClose[1]!.description).toEqual({ + type: "call", + name: "process-msg1", + }); + } + if (nonClose[2]!.type === "yield") { + expect(nonClose[2]!.description).toEqual({ type: "each", name: "queue" }); + } + if (nonClose[3]!.type === "yield") { + expect(nonClose[3]!.description).toEqual({ + type: "call", + name: "process-msg2", + }); + } + if (nonClose[4]!.type === "yield") { + expect(nonClose[4]!.description).toEqual({ type: "each", name: "queue" }); + expect(nonClose[4]!.result).toEqual({ + status: "ok", + value: { done: true }, + }); + } + }); + + // --------------------------------------------------------------------------- + // Test 6: Divergence detection — source name mismatch + // --------------------------------------------------------------------------- + + it("divergence — mismatched source name", function* () { + // Journal was recorded with name "queue" but workflow uses "other" + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "each", name: "queue" }, + result: { status: "ok", value: { value: "a" } }, + }, + ]; + const stream = new InMemoryStream(events); + const source = arraySource(["a"]); + + try { + yield* durableRun( + function* () { + for (const _msg of yield* durableEach("other", source)) { + yield* durableEach.next(); + } + }, + { stream }, + ); + throw new Error("expected Divergence error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).message).toContain("Divergence"); + } + }); + + // --------------------------------------------------------------------------- + // Test 7: Source error — propagated through Effection + // --------------------------------------------------------------------------- + + it("source error — propagated to workflow", function* () { + const stream = new InMemoryStream(); + const source: DurableSource = { + *next(): Operation<{ value: string } | { done: true }> { + throw new Error("connection lost"); + }, + }; + + try { + yield* durableRun( + function* () { + for (const _msg of yield* durableEach("queue", source)) { + yield* durableEach.next(); + } + }, + { stream }, + ); + throw new Error("expected connection lost error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).message).toContain("connection lost"); + } + }); + + // --------------------------------------------------------------------------- + // Test 8: Advance guard — missing durableEach.next() + // --------------------------------------------------------------------------- + + it("advance guard — throws when durableEach.next() is missing", function* () { + const stream = new InMemoryStream(); + const source = arraySource(["a", "b"]); + + try { + yield* durableRun( + function* () { + for (const _msg of yield* durableEach("queue", source)) { + // Missing: yield* durableEach.next(); + // The second iteration should trigger the advance guard + } + }, + { stream }, + ); + throw new Error("expected advance guard error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).message).toContain( + "yield* durableEach.next() must be called", + ); + } + }); + + // --------------------------------------------------------------------------- + // Test 9: Break exits cleanly, source closed + // --------------------------------------------------------------------------- + + it("break exits cleanly and closes source", function* () { + const stream = new InMemoryStream(); + const source = arraySource(["a", "b", "c"]); + const processed: string[] = []; + + const result = yield* durableRun( + function* () { + for (const msg of yield* durableEach("queue", source)) { + processed.push(msg); + if (msg === "b") break; + yield* durableEach.next(); + } + return "stopped"; + }, + { stream }, + ); + + expect(result).toBe("stopped"); + expect(processed).toEqual(["a", "b"]); + // Source should be closed via ensure() cleanup + expect(source.closed).toBe(true); + }); + + // --------------------------------------------------------------------------- + // Test 10: Null values in source — not confused with done signal + // --------------------------------------------------------------------------- + + it("null values are valid items, not done signals", function* () { + const stream = new InMemoryStream(); + const source = arraySource([null, "after-null", null]); + const processed: Json[] = []; + + const result = yield* durableRun( + function* () { + for (const msg of yield* durableEach("queue", source)) { + processed.push(msg); + yield* durableEach.next(); + } + return "done"; + }, + { stream }, + ); + + expect(result).toBe("done"); + expect(processed).toEqual([null, "after-null", null]); + + // Verify journal stores { value: null } not { done: true } + const events = stream.snapshot(); + const yieldEvents = events.filter((e) => e.type === "yield"); + if (yieldEvents[0]!.type === "yield") { + expect(yieldEvents[0]!.result).toEqual({ + status: "ok", + value: { value: null }, + }); + } + }); +}); diff --git a/durable-streams/durable-run.test.ts b/durable-streams/durable-run.test.ts new file mode 100644 index 00000000..1372af87 --- /dev/null +++ b/durable-streams/durable-run.test.ts @@ -0,0 +1,363 @@ +/** + * Tier 1 tests — core replay correctness. + * + * Tests 1-7 from the protocol specification. These validate that + * durableRun correctly executes workflows live, replays them from + * stored events, and handles crash recovery scenarios. + */ + +import { describe, it } from "@effectionx/bdd"; +import { expect } from "expect"; +import { + type DurableEvent, + InMemoryStream, + type Json, + type Workflow, + durableCall, + durableRun, +} from "./mod.ts"; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +/** Track which functions were actually called during live execution. */ +function createCallTracker() { + const calls: string[] = []; + return { + calls, + fn(name: string, value: T): () => Promise { + return () => { + calls.push(name); + return Promise.resolve(value); + }; + }, + }; +} + +describe("durableRun", () => { + // --------------------------------------------------------------------------- + // Test 1: Golden run — execute workflow end-to-end + // --------------------------------------------------------------------------- + + it("golden run: executes all effects live and records events", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + function* workflow(): Workflow { + const a = yield* durableCall("stepA", tracker.fn("stepA", "alpha")); + const b = yield* durableCall("stepB", tracker.fn("stepB", "beta")); + return `${a}-${b}`; + } + + const result = yield* durableRun(workflow, { stream }); + + // Verify result + expect(result).toBe("alpha-beta"); + + // Verify all effects were called + expect(tracker.calls).toEqual(["stepA", "stepB"]); + + // Verify stream has 2 Yield events + 1 Close event + const events = stream.snapshot(); + expect(events.length).toBe(3); + + expect(events[0]!.type).toBe("yield"); + expect(events[0]!.coroutineId).toBe("root"); + if (events[0]!.type === "yield") { + expect(events[0]!.description).toEqual({ type: "call", name: "stepA" }); + expect(events[0]!.result).toEqual({ status: "ok", value: "alpha" }); + } + + expect(events[1]!.type).toBe("yield"); + if (events[1]!.type === "yield") { + expect(events[1]!.description).toEqual({ type: "call", name: "stepB" }); + expect(events[1]!.result).toEqual({ status: "ok", value: "beta" }); + } + + expect(events[2]!.type).toBe("close"); + expect(events[2]!.coroutineId).toBe("root"); + if (events[2]!.type === "close") { + expect(events[2]!.result).toEqual({ status: "ok", value: "alpha-beta" }); + } + }); + + // --------------------------------------------------------------------------- + // Test 2: Full replay — replay entire stream + // --------------------------------------------------------------------------- + + it("full replay: returns stored result without re-executing effects", function* () { + // Pre-populate stream with a complete run + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + { + type: "close", + coroutineId: "root", + result: { status: "ok", value: "alpha-beta" }, + }, + ]; + const stream = new InMemoryStream(events); + const tracker = createCallTracker(); + + function* workflow(): Workflow { + const a = yield* durableCall("stepA", tracker.fn("stepA", "alpha")); + const b = yield* durableCall("stepB", tracker.fn("stepB", "beta")); + return `${a}-${b}`; + } + + const result = yield* durableRun(workflow, { stream }); + + // Result comes from the stored Close event + expect(result).toBe("alpha-beta"); + + // No effects were actually called — fully replayed from stored Close + expect(tracker.calls).toEqual([]); + + // Stream was not modified (no new events appended) + expect(stream.appendCount).toBe(0); + }); + + // --------------------------------------------------------------------------- + // Test 3: Crash before first effect — empty stream + // --------------------------------------------------------------------------- + + it("crash before first effect: empty stream, all live", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + function* workflow(): Workflow { + const a = yield* durableCall("stepA", tracker.fn("stepA", "alpha")); + return a; + } + + const result = yield* durableRun(workflow, { stream }); + + expect(result).toBe("alpha"); + expect(tracker.calls).toEqual(["stepA"]); + + const events = stream.snapshot(); + expect(events.length).toBe(2); // 1 Yield + 1 Close + }); + + // --------------------------------------------------------------------------- + // Test 4: Crash at position N — partial replay + // --------------------------------------------------------------------------- + + it("crash at position N: first N replayed, rest live", function* () { + // Stream has only the first Yield event (simulates crash after stepA) + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + ]; + const stream = new InMemoryStream(events); + const tracker = createCallTracker(); + + function* workflow(): Workflow { + const a = yield* durableCall("stepA", tracker.fn("stepA", "WRONG")); + const b = yield* durableCall("stepB", tracker.fn("stepB", "beta")); + return `${a}-${b}`; + } + + const result = yield* durableRun(workflow, { stream }); + + // stepA was replayed (returns stored "alpha", not "WRONG") + // stepB was executed live + expect(result).toBe("alpha-beta"); + + // Only stepB was actually called + expect(tracker.calls).toEqual(["stepB"]); + + // Stream now has: original Yield(stepA) + new Yield(stepB) + Close + const finalEvents = stream.snapshot(); + expect(finalEvents.length).toBe(3); + expect(finalEvents[0]!.type).toBe("yield"); + expect(finalEvents[1]!.type).toBe("yield"); + expect(finalEvents[2]!.type).toBe("close"); + + // Only 2 appends: Yield(stepB) + Close + expect(stream.appendCount).toBe(2); + }); + + // --------------------------------------------------------------------------- + // Test 5: Crash after last effect — all Yields but no Close + // --------------------------------------------------------------------------- + + it("crash after last effect: all Yields replayed, Close appended", function* () { + // Stream has both Yield events but no Close + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + ]; + const stream = new InMemoryStream(events); + const tracker = createCallTracker(); + + function* workflow(): Workflow { + const a = yield* durableCall("stepA", tracker.fn("stepA", "WRONG")); + const b = yield* durableCall("stepB", tracker.fn("stepB", "WRONG")); + return `${a}-${b}`; + } + + const result = yield* durableRun(workflow, { stream }); + + // Both effects replayed from journal + expect(result).toBe("alpha-beta"); + expect(tracker.calls).toEqual([]); + + // Only Close event was appended + expect(stream.appendCount).toBe(1); + const finalEvents = stream.snapshot(); + expect(finalEvents.length).toBe(3); + expect(finalEvents[2]!.type).toBe("close"); + }); + + // --------------------------------------------------------------------------- + // Test 6: Persist-before-resume — write completes before generator advances + // --------------------------------------------------------------------------- + + it("persist-before-resume: generator does not advance until write completes", function* () { + const stream = new InMemoryStream(); + const order: string[] = []; + + // Hook into append to track ordering + stream.onAppend = (event) => { + if (event.type === "yield") { + order.push(`persist:${event.type}`); + } + }; + + function* workflow(): Workflow { + yield* durableCall("step1", () => { + order.push("execute:step1"); + return Promise.resolve("one" as const); + }); + order.push("resumed:after-step1"); + + yield* durableCall("step2", () => { + order.push("execute:step2"); + return Promise.resolve("two" as const); + }); + order.push("resumed:after-step2"); + + return "done"; + } + + yield* durableRun(workflow, { stream }); + + // Verify ordering: execute → persist → resume for each step + expect(order).toEqual([ + "execute:step1", + "persist:yield", + "resumed:after-step1", + "execute:step2", + "persist:yield", + "resumed:after-step2", + ]); + }); + + // --------------------------------------------------------------------------- + // Test 7: Actor handoff — Process A writes N events, Process B resumes + // --------------------------------------------------------------------------- + + it("actor handoff: Process B resumes from Process A's events", function* () { + // Process A: execute first 2 steps then "crash" (we just take the events) + const streamA = new InMemoryStream(); + const trackerA = createCallTracker(); + + function* workflow(): Workflow { + const a = yield* durableCall("stepA", trackerA.fn("stepA", "alpha")); + const b = yield* durableCall("stepB", trackerA.fn("stepB", "beta")); + const c = yield* durableCall("stepC", trackerA.fn("stepC", "gamma")); + return `${a}-${b}-${c}`; + } + + yield* durableRun(workflow, { stream: streamA }); + + // Process A executed all steps + expect(trackerA.calls).toEqual(["stepA", "stepB", "stepC"]); + + // Simulate handoff: take only the first 2 Yield events (no Close, no stepC) + const allEvents = streamA.snapshot(); + const partialEvents = allEvents.slice(0, 2); + + // Process B: resume with partial events + const streamB = new InMemoryStream(partialEvents); + const trackerB = createCallTracker(); + + function* workflowB(): Workflow { + const a = yield* durableCall("stepA", trackerB.fn("stepA", "WRONG")); + const b = yield* durableCall("stepB", trackerB.fn("stepB", "WRONG")); + const c = yield* durableCall("stepC", trackerB.fn("stepC", "gamma")); + return `${a}-${b}-${c}`; + } + + const result = yield* durableRun(workflowB, { stream: streamB }); + + // stepA and stepB replayed, stepC executed live + expect(result).toBe("alpha-beta-gamma"); + expect(trackerB.calls).toEqual(["stepC"]); + + // Stream B: 2 original + 1 new Yield + 1 Close + const finalEvents = streamB.snapshot(); + expect(finalEvents.length).toBe(4); + }); + + // --------------------------------------------------------------------------- + // Additional: error propagation + // --------------------------------------------------------------------------- + + it("golden run with error: records Close(err) event", function* () { + const stream = new InMemoryStream(); + + function* workflow(): Workflow { + yield* durableCall("failingStep", () => + Promise.reject(new Error("boom")), + ); + return "unreachable"; + } + + try { + yield* durableRun(workflow, { stream }); + throw new Error("expected error from durableRun"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).message).toContain("boom"); + } + + // Stream has Yield(err) + Close(err) + const events = stream.snapshot(); + expect(events.length).toBe(2); + + if (events[0]!.type === "yield") { + expect(events[0]!.result.status).toBe("err"); + } + expect(events[1]!.type).toBe("close"); + if (events[1]!.type === "close") { + expect(events[1]!.result.status).toBe("err"); + } + }); +}); diff --git a/durable-streams/each.ts b/durable-streams/each.ts new file mode 100644 index 00000000..90d45804 --- /dev/null +++ b/durable-streams/each.ts @@ -0,0 +1,271 @@ +/** + * durableEach — durable iteration primitive for Effection workflows. + * + * Mirrors Effection's `each()` / `each.next()` pattern but journals + * every fetch as a DurableEffect, so iteration survives crashes and + * replays from the journal. + * + * Usage: + * for (let msg of yield* durableEach("queue", source)) { + * yield* durableCall("process", () => process(msg)); + * yield* durableEach.next(); + * } + * + * See effection-integration.md §12.6 for the full design. + */ + +import { ensure } from "effection"; +import type { Operation } from "effection"; +import { createDurableOperation } from "./effect.ts"; +import { ephemeral } from "./ephemeral.ts"; +import type { Json, Workflow } from "./types.ts"; + +// --------------------------------------------------------------------------- +// Types +// --------------------------------------------------------------------------- + +/** + * Source of items for durable iteration. + * + * Each call to `next()` blocks until the next item is available. + * Returns `{ value: T }` for an item, `{ done: true }` for exhaustion. + * + * The `{ done: true }` wrapper (rather than `T | null`) avoids ambiguity + * when `null` is a legitimate JSON value from the source. + */ +export interface DurableSource { + /** Read the next item, blocking until available. */ + next(): Operation<{ value: T } | { done: true }>; + /** + * Teardown — called on cancellation or completion. + * + * Must be idempotent: may be called more than once (once from effect + * teardown during an in-flight fetch, once from scope cleanup via + * ensure()). Subsequent calls after the first should be no-ops. + */ + close?(): void; +} + +// --------------------------------------------------------------------------- +// Internal state +// --------------------------------------------------------------------------- + +/** Sentinel for source exhaustion. Not exported — cannot collide with JSON. */ +const DONE: unique symbol = Symbol("durableEach.done"); +type ItemOrDone = T | typeof DONE; + +/** Type guard for the DONE sentinel. */ +function isDone(value: ItemOrDone): value is typeof DONE { + return value === DONE; +} + +/** State shared between durableEach and durableEach.next(). */ +interface DurableEachState { + name: string; + source: DurableSource; + current: ItemOrDone; + advanced: boolean; +} + +/** + * Module-level active state for sharing between durableEach() and durableEach.next(). + * + * Safe because durable execution is single-threaded — only one coroutine + * runs at a time, so there's no concurrent access. durableEach() sets this + * before returning the iterable; durableEach.next() reads it directly. + * + * This avoids using Effection context, which doesn't work when both + * functions are individually wrapped in ephemeral() (each gets its own + * child scope, making context invisible across them). + * + * LIMITATION: Only one durableEach iteration can be active at a time across + * the entire process. Concurrent durableEach calls (even in separate + * durableRun instances) will collide. The nesting guard below throws a + * clear error if this happens. For concurrent iterations, use child scopes + * via durableSpawn so each finishes its iteration before another starts. + * + * TODO: Consider migrating to a scope-local mechanism if Effection adds + * support for context that spans across ephemeral() boundaries. + */ +let activeState: DurableEachState | null = null; + +// --------------------------------------------------------------------------- +// durableEachFetch — shared helper for fetching one item +// --------------------------------------------------------------------------- + +/** + * Fetch a single item from the source (or replay it from the journal). + * + * Both the initial fetch (inside durableEach) and subsequent fetches + * (inside durableEach.next) go through this helper. Same effect + * description, same journal format, same replay path. + * + * Uses createDurableOperation to run the source's Operation-native next() + * with full structured concurrency — cancellation of the scope cancels + * the in-flight source.next() call. + * + * Journal shape: Yield event with description { type: "each", name } + * and result value { value: T } | { done: true }. + */ +function durableEachFetch( + name: string, + source: DurableSource, +): Workflow> { + return (function* () { + const result = (yield createDurableOperation<{ value: T } | { done: true }>( + { type: "each", name }, + () => source.next(), + )) as { value: T } | { done: true }; + + if ("done" in result) return DONE; + return result.value; + })(); +} + +// --------------------------------------------------------------------------- +// durableEach — initial fetch + returns synchronous iterable +// --------------------------------------------------------------------------- + +/** + * Durable iteration over a DurableSource (internal implementation). + * + * Returns Operation> because it uses ensure() (an + * infrastructure Operation). The public API wraps this in ephemeral() + * to return Workflow>. + * + * durableEach and durableEach.next share state through a module-level + * variable (activeState). This is safe because durable execution is + * single-threaded — only one coroutine runs at a time. This avoids + * Effection context, which doesn't work when both functions are + * individually wrapped in ephemeral() (each would get its own child + * scope, making context invisible across them). + */ +function* _durableEachOp( + name: string, + source: DurableSource, +): Operation> { + // Guard against nested durableEach — the single module-level slot + // would clobber the outer iteration's state. + if (activeState !== null) { + throw new Error( + `durableEach("${name}"): cannot nest durableEach calls in the same scope. Use a child scope (e.g., via spawn) for inner iterations.`, + ); + } + + // Register source teardown on scope exit — ensures cleanup even if + // the for...of loop breaks or the scope is cancelled without an + // active effect. Safe to call alongside effect-level teardown + // because DurableSource.close() must be idempotent. + yield* ensure(() => { + source.close?.(); + }); + + // Durable fetch of first item — journaled as a Yield event. + // ensure() is already registered, so cancellation here is safe. + const first: ItemOrDone = yield* durableEachFetch(name, source); + + // Store state in module-level slot for durableEach.next() to access. + // Cleared when the iteration completes (done or break). + const state: DurableEachState = { + name, + source, + current: first, + advanced: true, // first item was just fetched + }; + activeState = state as DurableEachState; + + // Return a synchronous iterable. The iterator generator checks + // the shared state on each re-entry. The try/finally ensures + // source.close() is called when the loop exits — whether by + // exhaustion (DONE), break, or throw. This provides immediate + // cleanup without waiting for scope teardown (ensure() is still + // registered as a safety net for cancellation during fetch). + return { + *[Symbol.iterator]() { + try { + while (!isDone(state.current)) { + // Advance guard: detect missing yield* durableEach.next() + if (!state.advanced) { + throw new Error( + `durableEach("${name}"): yield* durableEach.next() must be called before the next iteration. Each loop body must end with yield* durableEach.next() to checkpoint progress and fetch the next item.`, + ); + } + state.advanced = false; + yield state.current as T; + } + } finally { + // Clear module-level state so a subsequent durableEach can run. + activeState = null; + source.close?.(); + } + }, + }; +} + +/** + * Durable iteration over a DurableSource. + * + * Returns Workflow> via ephemeral() — the infrastructure + * effect (ensure) is durable-safe and re-runs correctly on replay. + */ +function _durableEach( + name: string, + source: DurableSource, +): Workflow> { + return ephemeral(_durableEachOp(name, source)); +} + +// --------------------------------------------------------------------------- +// durableEach.next — static method to advance iteration +// --------------------------------------------------------------------------- + +/** + * Advance the current durable iteration. + * + * Reads state from the module-level activeState slot (set by durableEach). + * This is a pure Workflow — no infrastructure effects, no ephemeral() + * needed. The only yielded effect is durableEachFetch, which is already + * a DurableEffect (journaled). + */ +function* _durableEachNext(): Workflow { + if (activeState === null) { + throw new Error( + "durableEach.next(): no active durableEach iteration. " + + "durableEach.next() must be called inside a durableEach loop.", + ); + } + const state = activeState as DurableEachState; + // Fetch next item first, then mark advanced. If the fetch throws + // (source error), advanced stays false and re-entry triggers the + // advance guard — preventing stale current from being re-yielded. + state.current = yield* durableEachFetch(state.name, state.source); + state.advanced = true; +} + +// --------------------------------------------------------------------------- +// Public API — durableEach with static .next() method +// --------------------------------------------------------------------------- + +/** + * Durable iteration over a DurableSource. + * + * Returns a Workflow that fetches the first item and yields a + * synchronous iterable. Use with `for...of` inside a Workflow: + * + * ```typescript + * for (let msg of yield* durableEach("queue", source)) { + * yield* durableCall("process", () => process(msg)); + * yield* durableEach.next(); + * } + * ``` + * + * @param name Stable name for the iteration + * @param source Operation-native source of items + */ +export const durableEach: { + ( + name: string, + source: DurableSource, + ): Workflow>; + next(): Workflow; +} = Object.assign(_durableEach, { next: _durableEachNext }); diff --git a/durable-streams/effect.ts b/durable-streams/effect.ts new file mode 100644 index 00000000..7932dfb5 --- /dev/null +++ b/durable-streams/effect.ts @@ -0,0 +1,360 @@ +/** + * createDurableEffect / createDurableOperation — core factories for durable effects. + * + * Each DurableEffect handles its own replay/live dispatch inside enter(). + * It reads DurableContext from the scope, checks the replay index, and + * either feeds the stored result (replay) or executes live with + * persist-before-resume semantics. + * + * Two factories are provided: + * - createDurableEffect: callback-based executor (resolve/reject/teardown) + * for timer-like and callback-based APIs (durableSleep, durableAction). + * - createDurableOperation: Operation-based executor for structured + * concurrency. The live path runs entirely as a generator — execute, + * capture result, persist, resolve. No callbacks, no .then(). + * + * Divergence policy is delegated to the Divergence API (DEC-031). + * By default, mismatches are fatal. Users can install middleware via + * scope.around(Divergence, ...) to override behavior per-scope. + * + * See integration doc §5.1, protocol spec §4.2, §5, §6. + */ + +import type { Operation } from "effection"; +import { type DurableContext, DurableCtx } from "./context.ts"; +import { Divergence } from "./divergence.ts"; +import { StaleInputError } from "./errors.ts"; +import { ReplayGuard } from "./replay-guard.ts"; +import { protocolToEffection, serializeError } from "./serialize.ts"; +import type { + CoroutineView, + DurableEffect, + EffectDescription, + EffectionResult, + Json, + Resolve, + Result, + Yield, +} from "./types.ts"; + +/** Effection void-ok result, used for no-op teardowns. */ +const VOID_OK: EffectionResult = { + ok: true, + value: undefined as undefined, +}; + +/** + * Executor function signature for live execution (callback-based). + * + * The executor receives: + * - resolve: call with a protocol Result when the effect completes + * - reject: call with an Error for unexpected failures + * + * Returns a teardown function called during scope destruction/cancellation. + */ +export type Executor = ( + resolve: (result: Result) => void, + reject: (error: Error) => void, +) => () => void; + +// --------------------------------------------------------------------------- +// Shared replay path +// --------------------------------------------------------------------------- + +/** + * Result of the replay check: either the effect was replayed (and enter() + * should return immediately) or the live path should execute. + */ +type ReplayResult = + | { + path: "replayed"; + teardown: (resolve: Resolve>) => void; + } + | { path: "live" }; + +/** + * Shared replay logic for both createDurableEffect and createDurableOperation. + * + * Checks the replay index for a matching entry, runs divergence detection, + * and runs replay guards. If replay succeeds, resolves the generator + * synchronously and returns "replayed". Otherwise returns "live" to + * indicate the caller should execute the effect. + */ +function checkReplay( + desc: EffectDescription, + resolve: Resolve>, + routine: CoroutineView, + ctx: DurableContext, +): ReplayResult { + const entry = ctx.replayIndex.peekYield(ctx.coroutineId); + + // ── REPLAY PATH ── + // Use a labeled block so that divergence decisions of type "run-live" + // can break out to fall through to the live execution path. + // biome-ignore lint/suspicious/noConfusingLabels: deliberate labeled block for break-out-of-replay pattern + replay: { + if (entry) { + // §6.2: Validate description identity match. + // Only type and name are identity fields — extra metadata fields + // (path, URL, marker, etc.) are stored for replay guards to inspect + // but do not participate in divergence detection. + // The current desc is passed to the Divergence API as `actual` so + // guards can compare both sides if needed. + if ( + entry.description.type !== desc.type || + entry.description.name !== desc.name + ) { + // Delegate divergence policy to the Divergence API. + const cursor = ctx.replayIndex.getCursor(ctx.coroutineId); + const decision = Divergence.invoke(routine.scope, "decide", [ + { + kind: "description-mismatch", + coroutineId: ctx.coroutineId, + cursor, + expected: entry.description, + actual: desc, + }, + ]); + + if (decision.type === "throw") { + resolve({ ok: false, error: decision.error }); + return { path: "replayed", teardown: (exit) => exit(VOID_OK) }; + } + + // decision.type === "run-live" + ctx.replayIndex.disableReplay(ctx.coroutineId); + break replay; + } + + // Description matches — now check replay guards before replaying. + // ── REPLAY GUARD: Decide phase ── + const yieldEvent: Yield = { + type: "yield", + coroutineId: ctx.coroutineId, + description: entry.description, + result: entry.result, + }; + const outcome = ReplayGuard.invoke(routine.scope, "decide", [yieldEvent]); + + if (outcome.outcome === "error") { + ctx.replayIndex.consumeYield(ctx.coroutineId); + const error = + outcome.error ?? + new StaleInputError( + `Stale input detected for ${desc.type}("${desc.name}")`, + { coroutineId: ctx.coroutineId, description: desc }, + ); + resolve({ ok: false, error }); + return { path: "replayed", teardown: (exit) => exit(VOID_OK) }; + } + + // All guards approved — consume the entry and advance cursor + ctx.replayIndex.consumeYield(ctx.coroutineId); + + // Feed stored result synchronously + resolve(protocolToEffection(entry.result)); + return { path: "replayed", teardown: (exit) => exit(VOID_OK) }; + } + + // No replay entry. Check for continue-past-close divergence (§6.3). + if (ctx.replayIndex.hasClose(ctx.coroutineId)) { + const yieldCount = ctx.replayIndex.yieldCount(ctx.coroutineId); + const decision = Divergence.invoke(routine.scope, "decide", [ + { + kind: "continue-past-close", + coroutineId: ctx.coroutineId, + yieldCount, + }, + ]); + + if (decision.type === "throw") { + resolve({ ok: false, error: decision.error }); + return { path: "replayed", teardown: (exit) => exit(VOID_OK) }; + } + + // decision.type === "run-live" + ctx.replayIndex.disableReplay(ctx.coroutineId); + break replay; + } + } // end replay block + + return { path: "live" }; +} + +// --------------------------------------------------------------------------- +// createDurableEffect — callback-based (Executor pattern) +// --------------------------------------------------------------------------- + +/** + * Creates a DurableEffect using a callback-based executor. + * + * Use this for timer-like and callback-based APIs (durableSleep, durableAction) + * where the resolve/reject/teardown pattern is natural. + * + * For Operation-based effects, prefer createDurableOperation. + * + * @param desc Structured description for the journal and divergence detection + * @param execute Called only during live execution (skipped during replay) + */ +export function createDurableEffect( + desc: EffectDescription, + execute: Executor, +): DurableEffect { + return { + description: `${desc.type}(${desc.name})`, + effectDescription: desc, + + enter( + resolve: Resolve>, + routine, + ): (resolve: Resolve>) => void { + const ctx = routine.scope.expect(DurableCtx); + const replay = checkReplay(desc, resolve, routine, ctx); + if (replay.path === "replayed") return replay.teardown; + + // ── LIVE PATH ── + let settled = false; + let tornDown = false; + let teardown: () => void = () => {}; + + /** Persist a Yield event then resume the generator. */ + function persistAndResolve(result: Result): void { + if (settled) return; + settled = true; + + const event: Yield = { + type: "yield", + coroutineId: ctx.coroutineId, + description: desc, + result, + }; + // Uses scope.run() to call the Operation-returning stream.append() + // from inside the callback-based enter(). The append runs as a + // structured operation in the routine's scope — if the scope tears + // down, the append is cancelled. + routine.scope.run(function* () { + try { + yield* ctx.stream.append(event); + resolve(protocolToEffection(result)); + } catch (err) { + resolve({ + ok: false, + error: err instanceof Error ? err : new Error(String(err)), + }); + } + }); + } + + // Guard against synchronous throws from the executor. + try { + teardown = execute( + (result: Result) => persistAndResolve(result), + (error: Error) => { + persistAndResolve({ + status: "err", + error: serializeError(error), + }); + }, + ); + } catch (e) { + const error = e instanceof Error ? e : new Error(String(e)); + persistAndResolve({ + status: "err", + error: serializeError(error), + }); + return (exit) => exit(VOID_OK); + } + + // Return teardown that Effection calls during scope destruction + return (exit: Resolve>) => { + if (tornDown) { + exit(VOID_OK); + return; + } + + tornDown = true; + settled = true; + + try { + teardown(); + exit(VOID_OK); + } catch (e) { + const error = e instanceof Error ? e : new Error(String(e)); + exit({ ok: false, error }); + } + }; + }, + }; +} + +// --------------------------------------------------------------------------- +// createDurableOperation — Operation-based (structured concurrency) +// --------------------------------------------------------------------------- + +/** + * Creates a DurableEffect from an Operation-returning function. + * + * The live path runs entirely as a generator inside scope.run(): + * execute the Operation, capture the result, persist the Yield event, + * then resolve the generator. No callbacks, no .then(), full structured + * concurrency — if the scope tears down, the operation is cancelled. + * + * Use this for durableCall and any effect where the work is expressed + * as an Operation (or can be wrapped as one via Effection's call()). + * + * @param desc Structured description for the journal and divergence detection + * @param execute Returns an Operation to run during live execution + */ +export function createDurableOperation( + desc: EffectDescription, + execute: () => Operation, +): DurableEffect { + return { + description: `${desc.type}(${desc.name})`, + effectDescription: desc, + + enter( + resolve: Resolve>, + routine, + ): (resolve: Resolve>) => void { + const ctx = routine.scope.expect(DurableCtx); + const replay = checkReplay(desc, resolve, routine, ctx); + if (replay.path === "replayed") return replay.teardown; + + // ── LIVE PATH ── + // Run the entire execute → capture → persist → resolve sequence + // as a structured operation in the routine's scope. + routine.scope.run(function* () { + let result: Result; + try { + const value = yield* execute(); + result = { status: "ok", value: value as Json }; + } catch (e) { + const error = e instanceof Error ? e : new Error(String(e)); + result = { status: "err", error: serializeError(error) }; + } + + const event: Yield = { + type: "yield", + coroutineId: ctx.coroutineId, + description: desc, + result, + }; + + try { + yield* ctx.stream.append(event); + resolve(protocolToEffection(result)); + } catch (err) { + resolve({ + ok: false, + error: err instanceof Error ? err : new Error(String(err)), + }); + } + }); + + // No teardown needed — scope.run() ties the operation's lifecycle + // to the routine's scope. Cancellation is handled by Effection. + return (exit) => exit(VOID_OK); + }, + }; +} diff --git a/durable-streams/ephemeral.test.ts b/durable-streams/ephemeral.test.ts new file mode 100644 index 00000000..6a42fefe --- /dev/null +++ b/durable-streams/ephemeral.test.ts @@ -0,0 +1,228 @@ +/** + * Tests for ephemeral() — the explicit escape hatch for non-durable + * Operations inside Workflows. + * + * Validates that: + * - ephemeral operations execute and return values correctly + * - ephemeral is transparent to the journal (no Yield events written) + * - ephemeral operations re-run on replay (not cached) + * - ephemeral supports cancellation via structured concurrency + * - the type boundary is enforced (bare Operations rejected by combinators) + */ + +import { describe, it } from "@effectionx/bdd"; +import { useScope } from "effection"; +import type { Operation } from "effection"; +import { expect } from "expect"; +import { + type DurableEvent, + InMemoryStream, + durableAll, + durableCall, + durableRun, + ephemeral, +} from "./mod.ts"; + +describe("ephemeral", () => { + // --------------------------------------------------------------------------- + // Test 1: ephemeral executes and returns value + // --------------------------------------------------------------------------- + + it("executes operation and returns value", function* () { + const stream = new InMemoryStream(); + + const result = yield* durableRun( + function* () { + const value = yield* ephemeral( + (function* (): Operation { + return "hello from ephemeral"; + })(), + ); + return value; + }, + { stream }, + ); + + expect(result).toBe("hello from ephemeral"); + }); + + // --------------------------------------------------------------------------- + // Test 2: ephemeral is transparent to journal — no Yield events + // --------------------------------------------------------------------------- + + it("transparent to journal — no Yield events written", function* () { + const stream = new InMemoryStream(); + + yield* durableRun( + function* () { + // One durable call, one ephemeral, one more durable call + yield* durableCall("step1", () => Promise.resolve("a")); + yield* ephemeral( + (function* (): Operation { + return "ephemeral-value"; + })(), + ); + yield* durableCall("step2", () => Promise.resolve("b")); + return "done"; + }, + { stream }, + ); + + const events: DurableEvent[] = yield* stream.readAll(); + + // Should have: Yield(step1), Yield(step2), Close(root) — NO ephemeral Yield + const yieldEvents = events.filter((e) => e.type === "yield"); + expect(yieldEvents.length).toBe(2); + expect(yieldEvents[0]!.description.name).toBe("step1"); + expect(yieldEvents[1]!.description.name).toBe("step2"); + + // No event with type "ephemeral" should exist + const ephemeralEvents = events.filter( + (e) => e.type === "yield" && e.description.type === "ephemeral", + ); + expect(ephemeralEvents.length).toBe(0); + }); + + // --------------------------------------------------------------------------- + // Test 3: ephemeral re-runs on replay (not cached) + // --------------------------------------------------------------------------- + + it("re-runs on replay — not cached", function* () { + const stream = new InMemoryStream(); + let ephemeralCallCount = 0; + + // First run — ephemeral runs once + yield* durableRun( + function* () { + yield* durableCall("step1", () => Promise.resolve("a")); + yield* ephemeral( + (function* (): Operation { + ephemeralCallCount++; + })(), + ); + yield* durableCall("step2", () => Promise.resolve("b")); + return "done"; + }, + { stream }, + ); + expect(ephemeralCallCount).toBe(1); + + // Remove the Close event to simulate partial replay + // Actually, durableRun short-circuits on Close, so we need a fresh stream + // with the same events minus Close to trigger replay + re-run + const events = yield* stream.readAll(); + const withoutClose = events.filter((e) => e.type !== "close"); + const replayStream = new InMemoryStream(withoutClose); + + // Reset counter + ephemeralCallCount = 0; + + // Second run — durable calls replay, but ephemeral re-runs + yield* durableRun( + function* () { + yield* durableCall("step1", () => Promise.resolve("a")); + yield* ephemeral( + (function* (): Operation { + ephemeralCallCount++; + })(), + ); + yield* durableCall("step2", () => Promise.resolve("b")); + return "done"; + }, + { stream: replayStream }, + ); + + // ephemeral ran again during replay + expect(ephemeralCallCount).toBe(1); + }); + + // --------------------------------------------------------------------------- + // Test 4: ephemeral propagates errors + // --------------------------------------------------------------------------- + + it("propagates errors from the operation", function* () { + const stream = new InMemoryStream(); + + try { + yield* durableRun( + function* () { + yield* ephemeral( + (function* (): Operation { + throw new Error("ephemeral boom"); + })(), + ); + return "unreachable"; + }, + { stream }, + ); + throw new Error("expected ephemeral boom"); + } catch (e) { + expect(e instanceof Error).toBe(true); + expect((e as Error).message).toBe("ephemeral boom"); + } + }); + + // --------------------------------------------------------------------------- + // Test 5: ephemeral works inside durableAll children + // --------------------------------------------------------------------------- + + it("works inside durableAll children", function* () { + const stream = new InMemoryStream(); + + const result = yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + // Use ephemeral to run an Operation inside a Workflow child + const scope = yield* ephemeral(useScope()); + // Just verify we got a scope (infrastructure operation worked) + if (!scope) throw new Error("no scope"); + return yield* durableCall("child1", () => Promise.resolve("a")); + }, + function* () { + return yield* durableCall("child2", () => Promise.resolve("b")); + }, + ]); + return results.join("-"); + }, + { stream }, + ); + + expect(result).toBe("a-b"); + }); + + // --------------------------------------------------------------------------- + // Test 6: nested durableAll works directly (no ephemeral needed) + // --------------------------------------------------------------------------- + + it("nested durableAll works without ephemeral wrapping", function* () { + const stream = new InMemoryStream(); + + // durableAll now returns Workflow, so nested calls work directly + // inside a Workflow child — no ephemeral wrapping required + const result = yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + const inner = yield* durableAll([ + function* () { + return yield* durableCall("innerA", () => Promise.resolve("x")); + }, + function* () { + return yield* durableCall("innerB", () => Promise.resolve("y")); + }, + ]); + return inner.join("+") as string; + }, + function* () { + return yield* durableCall("outer", () => Promise.resolve("z")); + }, + ]); + return results.join("-"); + }, + { stream }, + ); + + expect(result).toBe("x+y-z"); + }); +}); diff --git a/durable-streams/ephemeral.ts b/durable-streams/ephemeral.ts new file mode 100644 index 00000000..24f94431 --- /dev/null +++ b/durable-streams/ephemeral.ts @@ -0,0 +1,95 @@ +/** + * ephemeral — explicit escape hatch for non-durable Operations inside Workflows. + * + * Wraps an Operation so it satisfies the Workflow type contract. + * The wrapped operation is **transparent to the journal** — no Yield event + * is written, no replay index entry is consumed. On replay, the operation + * simply re-runs. + * + * This is analogous to Rust's `unsafe {}` — it marks the boundary where + * the user is opting out of durable guarantees. Every non-durable + * Operation that needs to participate in a Workflow must go through + * ephemeral() to make the escape explicit and auditable. + * + * Usage: + * yield* durableAll([ + * () => myDurableWorkflow(), // Workflow — journaled + * function*() { // Workflow with ephemeral child + * return yield* ephemeral(someOperation()); + * }, + * ]); + * + * See DEC-034 for the design rationale. + */ + +import type { Operation } from "effection"; +import type { + DurableEffect, + EffectionResult, + Resolve, + Workflow, +} from "./types.ts"; + +/** Effection void-ok result, used for no-op teardowns. */ +const VOID_OK: EffectionResult = { + ok: true, + value: undefined as undefined, +}; + +/** + * Create a DurableEffect that runs an Operation transparently. + * + * - No journal write (no Yield event appended) + * - No replay index consumption (cursor not advanced) + * - The Operation runs via scope.run() with full structured concurrency + * - Cancellation of the scope cancels the inner Operation + * - On replay, the Operation re-runs (not cached) + * + * The effect is invisible to the durable execution protocol — it exists + * solely to satisfy the Workflow type constraint. + */ +function createEphemeralEffect(operation: Operation): DurableEffect { + return { + description: "ephemeral", + effectDescription: { type: "ephemeral", name: "ephemeral" }, + + enter( + resolve: Resolve>, + routine, + ): (resolve: Resolve>) => void { + // Run the operation in the routine's scope — full structured + // concurrency, proper cancellation. No journal interaction. + routine.scope.run(function* () { + try { + const value = yield* operation; + resolve({ ok: true, value }); + } catch (e) { + const error = e instanceof Error ? e : new Error(String(e)); + resolve({ ok: false, error }); + } + }); + + // No teardown needed — scope.run() ties the operation's lifecycle + // to the routine's scope. + return (exit) => exit(VOID_OK); + }, + }; +} + +/** + * Wrap a non-durable Operation so it can be used inside a Workflow. + * + * The operation is transparent to the durable execution protocol: + * - No Yield event is written to the journal + * - On replay, the operation re-runs (it is not cached) + * - Cancellation flows through normally via structured concurrency + * + * Use this when you need to run infrastructure Operations (or intentionally + * non-durable work) inside a Workflow where only DurableEffects are allowed. + * + * @param operation The Operation to wrap + * @returns A Workflow that yields the operation's result + */ +export function* ephemeral(operation: Operation): Workflow { + return (yield createEphemeralEffect(operation)) as T; +} diff --git a/durable-streams/errors.ts b/durable-streams/errors.ts new file mode 100644 index 00000000..f2dde242 --- /dev/null +++ b/durable-streams/errors.ts @@ -0,0 +1,126 @@ +/** + * Error types for the durable execution protocol. + */ + +import type { CoroutineId, EffectDescription } from "./types.ts"; + +/** + * Raised when the replay index entry at the current cursor position + * does not match the effect yielded by the generator. See spec §6.2. + * + * A DivergenceError is NOT recoverable. The workflow cannot continue + * because the generator's execution path has diverged from the recorded + * history. + */ +export class DivergenceError extends Error { + override name = "DivergenceError"; + + coroutineId: CoroutineId; + /** Cursor position within the coroutine where divergence was detected. */ + position: number; + /** The description from the journal (what was expected). */ + expected: EffectDescription; + /** The description from the generator (what was actually yielded). */ + actual: EffectDescription; + + constructor( + coroutineId: CoroutineId, + position: number, + expected: EffectDescription, + actual: EffectDescription, + message?: string, + ) { + super( + message ?? + `Divergence at ${coroutineId}[${position}]: ` + + `expected ${expected.type}("${expected.name}"), ` + + `got ${actual.type}("${actual.name}")`, + ); + this.coroutineId = coroutineId; + this.position = position; + this.expected = expected; + this.actual = actual; + } +} + +/** + * Raised when the generator finishes (returns) while the replay index + * still has unconsumed entries for this coroutine. See spec §6.3. + */ +export class EarlyReturnDivergenceError extends Error { + override name = "EarlyReturnDivergenceError"; + + coroutineId: CoroutineId; + consumedCount: number; + totalCount: number; + + constructor( + coroutineId: CoroutineId, + consumedCount: number, + totalCount: number, + ) { + super( + `Divergence: generator ${coroutineId} returned after ${consumedCount} yields, ` + + `but journal has ${totalCount} yield entries`, + ); + this.coroutineId = coroutineId; + this.consumedCount = consumedCount; + this.totalCount = totalCount; + } +} + +/** + * Raised when the journal has a Close event for a coroutine but the + * generator has not finished after consuming all recorded yields. + * See spec §6.3. + */ +export class ContinuePastCloseDivergenceError extends Error { + override name = "ContinuePastCloseDivergenceError"; + + coroutineId: CoroutineId; + yieldCount: number; + + constructor(coroutineId: CoroutineId, yieldCount: number) { + super( + `Divergence: journal shows ${coroutineId} closed after ${yieldCount} yields, but generator continues to yield effects`, + ); + this.coroutineId = coroutineId; + this.yieldCount = yieldCount; + } +} + +/** + * Raised by a replay guard when a journal entry's recorded result is + * stale (e.g., the source file has changed since the effect was + * originally executed). + * + * Guards detect staleness by comparing current state against data stored + * in the effect description (input fields like file path) and result + * value (output fields like content hash). + * + * StaleInputError is NOT a divergence — the effect identity matches, + * but the external world has changed. The correct response depends on + * application policy: re-run from scratch, accept stale results, or + * (in future versions) re-execute the effect and continue. + * + * See replay-guard-spec.md §4.4. + */ +export class StaleInputError extends Error { + override name = "StaleInputError"; + + /** The Yield event that was detected as stale. */ + event?: { coroutineId: string; description: { type: string; name: string } }; + + constructor( + /** Human-readable description of what changed. */ + message: string, + /** The Yield event that was detected as stale. */ + event?: { + coroutineId: string; + description: { type: string; name: string }; + }, + ) { + super(message); + this.event = event; + } +} diff --git a/durable-streams/http-stream.ts b/durable-streams/http-stream.ts new file mode 100644 index 00000000..a765c856 --- /dev/null +++ b/durable-streams/http-stream.ts @@ -0,0 +1,287 @@ +/** + * HTTP-backed DurableStream implementation using the Durable Streams protocol. + * + * Uses raw fetch() for appends (not IdempotentProducer) because durable + * execution requires synchronous acknowledgment on every write + * (persist-before-resume). See DEC-026. + * + * Concurrent appends are serialized via a Queue + spawned worker so that + * the server always receives them in sequence order. The worker lives + * inside a resource scope and is cancelled when the stream is no longer + * in use. See DEC-033 (supersedes DEC-027's Promise chain approach). + * + * The stream is created as an Effection resource via useHttpDurableStream(). + */ + +import { + stream, + PRODUCER_EPOCH_HEADER, + PRODUCER_EXPECTED_SEQ_HEADER, + PRODUCER_ID_HEADER, + PRODUCER_RECEIVED_SEQ_HEADER, + PRODUCER_SEQ_HEADER, + STREAM_OFFSET_HEADER, + SequenceGapError, + StaleEpochError, +} from "@durable-streams/client"; +import { call, createQueue, resource, spawn, withResolvers } from "effection"; +import type { Operation, Queue } from "effection"; +import type { DurableStream } from "./stream.ts"; +import type { DurableEvent } from "./types.ts"; + +/** + * Configuration for useHttpDurableStream. + */ +export interface HttpDurableStreamOptions { + /** Base URL of the Durable Streams server (e.g. "http://localhost:4437"). */ + baseUrl: string; + /** Stream identifier. Will be used as the URL path segment. */ + streamId: string; + /** Unique producer identifier for idempotent append tracking. */ + producerId: string; + /** Producer epoch — monotonically increasing. Stale epochs are fenced. */ + epoch: number; + /** Optional custom fetch implementation (for testing). */ + fetch?: typeof globalThis.fetch; +} + +/** + * Extended DurableStream with HTTP-specific observable state. + */ +export interface HttpDurableStreamHandle extends DurableStream { + /** + * Last Stream-Next-Offset received from the server. + * Tracked from both reads and writes (DEC-029). + * This is the resumption point for future tail() calls. + */ + lastOffset: string | undefined; +} + +/** Request sent to the serial append worker. */ +interface AppendRequest { + event: DurableEvent; + seq: number; + resolve: (value: undefined) => void; + reject: (error: Error) => void; +} + +/** + * Create an HTTP-backed DurableStream as an Effection resource. + * + * The resource: + * 1. Creates the stream on the server (idempotent PUT) + * 2. Spawns a serial worker that processes appends in FIFO order + * 3. Returns a DurableStream handle with Operation-native readAll/append + * + * The worker is cancelled when the resource scope is torn down. + * + * Usage: + * yield* useHttpDurableStream({ baseUrl, streamId, producerId, epoch }) + */ +export function useHttpDurableStream( + opts: HttpDurableStreamOptions, +): Operation { + return resource(function* (provide) { + const baseUrl = new URL(opts.baseUrl); + if (!baseUrl.pathname.endsWith("/")) { + baseUrl.pathname = `${baseUrl.pathname}/`; + } + const streamUrl = new URL( + encodeURIComponent(opts.streamId), + baseUrl, + ).toString(); + const producerId = opts.producerId; + const epoch = opts.epoch; + const fetchFn = opts.fetch ?? globalThis.fetch.bind(globalThis); + + // ── One-time setup: create the stream on the server ── + yield* call(async () => { + const res = await fetchFn(streamUrl, { + method: "PUT", + headers: { "Content-Type": "application/json" }, + }); + await res.text(); // consume body to free connection + if (res.status !== 201 && res.status !== 200) { + throw new Error(`Failed to create stream: HTTP ${res.status}`); + } + }); + + // ── Mutable state owned by the resource scope ── + let nextSeq = 0; + let fatalError: Error | undefined; + let lastOffset: string | undefined; + + // ── Append worker queue ── + const queue: Queue = createQueue< + AppendRequest, + void + >(); + + // ── Spawn the serial append worker ── + // Processes one append at a time in FIFO order. Each HTTP POST + // completes before the next one starts, guaranteeing server-side + // sequence ordering. Fatal errors (stale epoch, network failure) + // are propagated to the specific caller and stored for fail-fast. + yield* spawn(function* () { + let item = yield* queue.next(); + while (!item.done) { + const { event, seq, resolve, reject } = item.value; + try { + yield* call(() => doAppend(event, seq)); + resolve(undefined); + } catch (err) { + reject(err instanceof Error ? err : new Error(String(err))); + } + item = yield* queue.next(); + } + }); + + /** + * Execute a single HTTP append with the given event and sequence number. + * + * Any uncertain write outcome (network error, unexpected HTTP status, + * sequence gap) is treated as fatal — `fatalError` is set so all future + * appends fail-fast. + */ + async function doAppend(event: DurableEvent, seq: number): Promise { + // Double-check fatal error (may have been set by a preceding append) + if (fatalError) { + throw fatalError; + } + + let res: Response; + try { + res = await fetchFn(streamUrl, { + method: "POST", + headers: { + "Content-Type": "application/json", + [PRODUCER_ID_HEADER]: producerId, + [PRODUCER_EPOCH_HEADER]: String(epoch), + [PRODUCER_SEQ_HEADER]: String(seq), + }, + body: JSON.stringify(event), + }); + } catch (err) { + // Network failure — fatal, sequence state is now uncertain + const error = err instanceof Error ? err : new Error(String(err)); + fatalError = error; + throw error; + } + + // Always consume the body to free the connection + await res.text(); + + switch (res.status) { + case 200: { + // Success — capture offset + const offset = res.headers.get(STREAM_OFFSET_HEADER); + if (offset) { + lastOffset = offset; + } + return; + } + case 204: { + // Duplicate (idempotent success) — capture offset if present + const offset = res.headers.get(STREAM_OFFSET_HEADER); + if (offset) { + lastOffset = offset; + } + return; + } + case 403: { + // Stale epoch — fatal error + const currentEpoch = Number( + res.headers.get(PRODUCER_EPOCH_HEADER) ?? 0, + ); + const error = new StaleEpochError(currentEpoch); + fatalError = error; + throw error; + } + case 409: { + // Sequence gap — fatal (should never happen due to serialization, + // but if it does, sequence state is irrecoverably desynchronized) + const expected = Number( + res.headers.get(PRODUCER_EXPECTED_SEQ_HEADER) ?? 0, + ); + const received = Number( + res.headers.get(PRODUCER_RECEIVED_SEQ_HEADER) ?? 0, + ); + const error = new SequenceGapError(expected, received); + fatalError = error; + throw error; + } + default: { + // Unexpected status — fatal, write outcome is uncertain. + // TODO: Transient errors (500, 503) could be retried with the + // same seq in a future version. See DEC-026 rationale. + const error = new Error( + `Unexpected append response: HTTP ${res.status}`, + ); + fatalError = error; + throw error; + } + } + } + + // ── Provide the DurableStream handle ── + yield* provide({ + get lastOffset() { + return lastOffset; + }, + + /** + * Read all events in the stream, in append order. + * + * Uses the stream() function from @durable-streams/client with + * offset="-1" (start of stream) and live=false (no tailing). + */ + *readAll(): Operation { + return yield* call(async () => { + const res = await stream({ + url: streamUrl, + offset: "-1", + live: false, + fetch: fetchFn, + }); + const events = (await res.json()) as DurableEvent[]; + // Track offset from read (DEC-029) + if (res.offset) { + lastOffset = res.offset; + } + return events; + }); + }, + + /** + * Append an event to the stream. + * + * Sequence numbers are assigned synchronously when the generator is + * started. The actual HTTP call is dispatched to the serial worker + * via the queue. The caller suspends until the worker completes the + * POST and signals via withResolvers. + * + * Fatal errors (stale epoch, network failure) are stored and cause + * all future appends to fail-fast without enqueuing. + */ + *append(event: DurableEvent): Operation { + // Fail-fast if a fatal error has been set + if (fatalError) { + throw fatalError; + } + + // Assign seq synchronously — ordering is locked in before any + // async work, even when multiple coroutines call append concurrently + const seq = nextSeq++; + + // Create a resolver so we can wait for this specific append to complete + const { operation, resolve, reject } = withResolvers(); + + // Enqueue the request — the worker will process it in FIFO order + queue.add({ event, seq, resolve, reject }); + + // Suspend until the worker finishes the HTTP POST for this item + yield* operation; + }, + }); + }); +} diff --git a/durable-streams/mod.ts b/durable-streams/mod.ts new file mode 100644 index 00000000..057c5ac4 --- /dev/null +++ b/durable-streams/mod.ts @@ -0,0 +1,106 @@ +/** + * @module + * Durable execution for Effection. + * + * Implements the two-event durable execution protocol for generator-based + * structured concurrency, with Durable Streams as the persistence backend. + */ + +// Protocol types +export type { + Close, + CoroutineId, + CoroutineView, + DurableEffect, + DurableEvent, + EffectDescription, + EffectionResult, + Json, + Resolve, + Result, + SerializedError, + Workflow, + Yield, +} from "./types.ts"; + +// ReplayIndex +export { ReplayIndex } from "./replay-index.ts"; +export type { YieldEntry } from "./replay-index.ts"; + +// Stream interface +export type { DurableStream } from "./stream.ts"; +export { InMemoryStream } from "./stream.ts"; + +// HTTP-backed stream adapter +export { useHttpDurableStream } from "./http-stream.ts"; +export type { + HttpDurableStreamHandle, + HttpDurableStreamOptions, +} from "./http-stream.ts"; + +// Errors +export { + ContinuePastCloseDivergenceError, + DivergenceError, + EarlyReturnDivergenceError, + StaleInputError, +} from "./errors.ts"; + +// Divergence API — pluggable policy for replay mismatches (DEC-031) +export { Divergence } from "./divergence.ts"; +export type { + DivergenceDecision, + DivergenceInfo, + DivergenceKind, +} from "./divergence.ts"; + +// ReplayGuard API — pluggable validation for replay staleness detection +export { ReplayGuard } from "./replay-guard.ts"; +export type { ReplayOutcome } from "./replay-guard.ts"; + +// Runtime abstraction — platform-agnostic I/O for durable effects +export { DurableRuntimeCtx } from "./runtime.ts"; +export type { + DurableRuntime, + ResponseHeaders, + RuntimeFetchResponse, + StatResult, +} from "./runtime.ts"; + +// Context +export { DurableCtx } from "./context.ts"; +export type { DurableContext } from "./context.ts"; + +// Serialization utilities +export { + deserializeError, + effectionToProtocol, + protocolToEffection, + serializeError, +} from "./serialize.ts"; + +// Core effect factories +export { createDurableEffect, createDurableOperation } from "./effect.ts"; +export type { Executor } from "./effect.ts"; + +// Workflow-enabled effects +export { + durableAction, + durableCall, + durableSleep, + versionCheck, +} from "./operations.ts"; + +// Structured concurrency combinators +export { durableAll, durableRace, durableSpawn } from "./combinators.ts"; + +// Durable iteration +export { durableEach } from "./each.ts"; +export type { DurableSource } from "./each.ts"; + +// Ephemeral — explicit escape hatch for non-durable Operations in Workflows +export { ephemeral } from "./ephemeral.ts"; + +// Entry point +export { durableRun } from "./run.ts"; +export type { DurableRunOptions } from "./run.ts"; diff --git a/durable-streams/operations.ts b/durable-streams/operations.ts new file mode 100644 index 00000000..3315ada3 --- /dev/null +++ b/durable-streams/operations.ts @@ -0,0 +1,130 @@ +/** + * Workflow-enabled effects — durable equivalents of Effection's built-in + * operations. + * + * Each returns a Workflow (a generator that yields a single DurableEffect). + * These are the building blocks for durable workflows. + * + * See integration doc §6. + */ + +import { call } from "effection"; +import type { Operation } from "effection"; +import { createDurableEffect, createDurableOperation } from "./effect.ts"; +import type { Json, Workflow } from "./types.ts"; + +/** + * Durable sleep — pauses the workflow for `ms` milliseconds. + * + * During replay, resolves synchronously with the stored result. + * During live execution, uses setTimeout and persists the Yield event. + * + * Description: { type: "sleep", name: "sleep" } + */ +export function* durableSleep(ms: number): Workflow { + yield createDurableEffect( + { type: "sleep", name: "sleep" }, + (resolve) => { + const id = setTimeout(() => resolve({ status: "ok" }), ms); + return () => clearTimeout(id); + }, + ); +} + +/** + * Durable call — wraps a function for durable execution. + * + * Accepts functions returning either a Promise or an Operation. + * Effection's call() handles the dispatch at runtime: Promises are + * bridged, Operations run with full structured concurrency. + * + * The function is called during live execution; its resolved value is + * serialized and persisted. During replay, the stored value is returned + * without calling the function. + * + * Description: { type: "call", name } + * + * IMPORTANT: The function's return value must be JSON-serializable. + * + * @param name Stable identifier for the effect (used for divergence detection) + * @param fn Function returning a Promise or Operation (only called during live execution) + */ +export function* durableCall( + name: string, + fn: () => Promise | Operation, +): Workflow { + // call() dispatches at runtime: if fn() returns a Promise, it bridges + // via action(); if it returns an Operation, it evaluates directly. + // The cast is safe because call() always resolves to T regardless of + // which branch fn() takes. + return (yield createDurableOperation( + { type: "call", name }, + // biome-ignore lint/suspicious/noExplicitAny: fn returns Promise | Operation, call() accepts both + () => call(fn as any) as Operation, + )) as T; +} + +/** + * Durable action — generic effect with a custom executor. + * + * Like Effection's action(), but durable. The executor receives resolve/reject + * callbacks and returns a teardown function. + * + * Description: { type: "action", name } + */ +export function* durableAction( + name: string, + executor: ( + resolve: (value: T) => void, + reject: (error: Error) => void, + ) => () => void, +): Workflow { + return (yield createDurableEffect( + { type: "action", name }, + (protocolResolve, reject) => { + return executor( + (value: T) => protocolResolve({ status: "ok", value: value as Json }), + reject, + ); + }, + )) as T; +} + +/** + * Version gate — enables safe code evolution for durable workflows. + * + * During live execution, resolves with `maxVersion`. During replay, the + * stored version determines which code path the workflow takes. + * + * Description: { type: "version_gate", name } + * + * See spec §9. + */ +export function* versionCheck( + name: string, + opts: { minVersion: number; maxVersion: number }, +): Workflow { + if (opts.minVersion > opts.maxVersion) { + throw new Error( + `versionCheck("${name}"): minVersion (${opts.minVersion}) ` + + `cannot exceed maxVersion (${opts.maxVersion})`, + ); + } + + const version = (yield createDurableEffect( + { type: "version_gate", name }, + (resolve) => { + resolve({ status: "ok", value: opts.maxVersion }); + return () => {}; + }, + )) as number; + + if (version < opts.minVersion || version > opts.maxVersion) { + throw new Error( + `versionCheck("${name}"): replayed version ${version} is outside ` + + `supported range [${opts.minVersion}, ${opts.maxVersion}]`, + ); + } + + return version; +} diff --git a/durable-streams/package.json b/durable-streams/package.json new file mode 100644 index 00000000..021c1b6c --- /dev/null +++ b/durable-streams/package.json @@ -0,0 +1,44 @@ +{ + "name": "@effectionx/durable-streams", + "description": "Durable execution for Effection backed by Durable Streams", + "version": "0.1.0", + "type": "module", + "main": "./dist/mod.js", + "types": "./dist/mod.d.ts", + "exports": { + ".": { + "types": "./dist/mod.d.ts", + "development": "./mod.ts", + "import": "./dist/mod.js", + "default": "./dist/mod.js" + } + }, + "peerDependencies": { + "effection": "^3 || ^4" + }, + "dependencies": { + "@durable-streams/client": "^0.2.1" + }, + "license": "MIT", + "author": "engineering@frontside.com", + "repository": { + "type": "git", + "url": "git+https://github.com/thefrontside/effectionx.git" + }, + "bugs": { + "url": "https://github.com/thefrontside/effectionx/issues" + }, + "engines": { + "node": ">= 22" + }, + "sideEffects": false, + "keywords": ["streams", "concurrency"], + "files": ["dist", "*.ts", "README.md"], + "devDependencies": { + "@effectionx/bdd": "workspace:*", + "@effectionx/fs": "workspace:*", + "effection": "^4", + "@durable-streams/server": "^0.2.1", + "expect": "^29" + } +} diff --git a/durable-streams/replay-guard.test.ts b/durable-streams/replay-guard.test.ts new file mode 100644 index 00000000..9d797a2b --- /dev/null +++ b/durable-streams/replay-guard.test.ts @@ -0,0 +1,715 @@ +/** + * Replay Guard tests — pluggable validation for replay staleness detection. + * + * Tests the ReplayGuard API middleware system for detecting stale inputs + * during replay. See replay-guard-spec.md §9. + * + * Guards access `event.description.*` for input fields (e.g., file path) + * and `event.result.value.*` for output fields (e.g., content hash). + * There is no separate `meta` field — inputs belong in the effect + * description, outputs belong in the result. + */ + +import { describe, it } from "@effectionx/bdd"; +import { run, useScope } from "effection"; +import { expect } from "expect"; +import { + type DurableEvent, + InMemoryStream, + ReplayGuard, + type ReplayOutcome, + StaleInputError, + type Workflow, + type Yield, + durableCall, + durableRun, +} from "./mod.ts"; + +describe("replay guard", () => { + // --------------------------------------------------------------------------- + // Test 1: No guards installed → normal replay + // --------------------------------------------------------------------------- + + it("no guards installed — normal replay proceeds", function* () { + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + }, + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepB" }, + result: { status: "ok", value: "beta" }, + }, + { + type: "close", + coroutineId: "root", + result: { status: "ok", value: "alpha-beta" }, + }, + ]; + const stream = new InMemoryStream(events); + const liveCalls: string[] = []; + + const result = yield* durableRun( + function* (): Workflow { + const a = yield* durableCall("stepA", () => { + liveCalls.push("stepA"); + return Promise.resolve("should-not-be-called"); + }); + const b = yield* durableCall("stepB", () => { + liveCalls.push("stepB"); + return Promise.resolve("should-not-be-called"); + }); + return `${a}-${b}`; + }, + { stream }, + ); + + // Full replay returns stored Close result, no live calls + expect(result).toBe("alpha-beta"); + expect(liveCalls).toEqual([]); + }); + + // --------------------------------------------------------------------------- + // Test 2: Guard installed, event has no applicable fields → replay proceeds + // --------------------------------------------------------------------------- + + it("event without validation fields — replay proceeds", function* () { + // Event has no path in description — guard should pass it through + // Note: NO Close event, so workflow actually runs and replays + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA" }, + result: { status: "ok", value: "alpha" }, + // no extra fields in description, no contentHash in result + }, + // No close event - workflow runs and replays the yield, then executes live close + ]; + const stream = new InMemoryStream(events); + const checkEvents: Yield[] = []; + const decideEvents: Yield[] = []; + + const scope = yield* useScope(); + + // Install a guard that tracks which events it sees + scope.around(ReplayGuard, { + *check([event], next) { + checkEvents.push(event); + return yield* next(event); + }, + decide([event], next) { + decideEvents.push(event); + // No opinion — pass through + return next(event); + }, + }); + + const result = yield* durableRun( + function* (): Workflow { + return yield* durableCall("stepA", () => + Promise.resolve("should-not-be-called"), + ); + }, + { stream }, + ); + + // Replay should proceed normally (returns stored value, not live value) + expect(result).toBe("alpha"); + + // Guard should have seen the event in both phases + expect(checkEvents.length).toBe(1); + expect(decideEvents.length).toBe(1); + // No extra fields in description + expect(checkEvents[0]!.description.path).toBeUndefined(); + }); + + // --------------------------------------------------------------------------- + // Test 3: Description path and result hash match → replay proceeds + // --------------------------------------------------------------------------- + + it("description/result fields match — replay proceeds", function* () { + // Simulate a file hash that hasn't changed + // path is in description, contentHash is in result.value + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "readFile", path: "./test.txt" }, + result: { + status: "ok", + value: { content: "file contents", contentHash: "abc123" }, + }, + }, + { + type: "close", + coroutineId: "root", + result: { + status: "ok", + value: { content: "file contents", contentHash: "abc123" }, + }, + }, + ]; + const stream = new InMemoryStream(events); + + // Cache simulates current file having the same hash + const cache = new Map([["./test.txt", "abc123"]]); + + const scope = yield* useScope(); + + scope.around(ReplayGuard, { + *check([event], next) { + // In real usage, would compute hash here. For test, cache is pre-populated. + return yield* next(event); + }, + decide([event], next) { + const filePath = event.description.path; + const resultValue = + event.result.status === "ok" ? event.result.value : undefined; + const recordedHash = ( + resultValue as Record | undefined + )?.contentHash; + if (typeof filePath === "string" && typeof recordedHash === "string") { + const currentSHA = cache.get(filePath); + if (currentSHA && currentSHA !== recordedHash) { + return { + outcome: "error", + error: new StaleInputError(`File changed: ${filePath}`), + }; + } + } + return next(event); + }, + }); + + const result = yield* durableRun( + function* (): Workflow> { + return yield* durableCall>("readFile", () => + Promise.resolve({ + content: "should-not-be-called", + contentHash: "abc123", + }), + ); + }, + { stream }, + ); + + // Replay should proceed since hashes match + expect(result.content).toBe("file contents"); + }); + + // --------------------------------------------------------------------------- + // Test 4: Description path present but result hash differs → replay errors + // --------------------------------------------------------------------------- + + it("result hash mismatch — replay errors with StaleInputError", function* () { + // File hash in journal result differs from current hash + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "readFile", path: "./test.txt" }, + result: { + status: "ok", + value: { content: "old contents", contentHash: "abc123" }, + }, + }, + ]; + const stream = new InMemoryStream(events); + + // Cache simulates current file having a DIFFERENT hash + const cache = new Map([["./test.txt", "def456"]]); + + const scope = yield* useScope(); + + scope.around(ReplayGuard, { + *check([event], next) { + return yield* next(event); + }, + decide([event], next) { + const filePath = event.description.path; + const resultValue = + event.result.status === "ok" ? event.result.value : undefined; + const recordedHash = ( + resultValue as Record | undefined + )?.contentHash; + if (typeof filePath === "string" && typeof recordedHash === "string") { + const currentSHA = cache.get(filePath); + if (currentSHA && currentSHA !== recordedHash) { + return { + outcome: "error", + error: new StaleInputError( + `File changed: ${filePath} (recorded: ${recordedHash}, current: ${currentSHA})`, + ), + }; + } + } + return next(event); + }, + }); + + try { + yield* durableRun( + function* (): Workflow> { + return yield* durableCall>("readFile", () => + Promise.resolve({ + content: "should-not-be-called", + contentHash: "abc123", + }), + ); + }, + { stream }, + ); + throw new Error("expected StaleInputError"); + } catch (e) { + expect(e).toBeInstanceOf(StaleInputError); + expect((e as Error).message).toContain("File changed"); + expect((e as Error).message).toContain("./test.txt"); + } + }); + + // --------------------------------------------------------------------------- + // Test 5: Multiple guards, one errors → replay halts + // --------------------------------------------------------------------------- + + it("multiple guards — error from any guard halts replay", function* () { + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { + type: "call", + name: "step", + checkA: "pass", + checkB: "fail", + }, + result: { status: "ok", value: "result" }, + }, + ]; + const stream = new InMemoryStream(events); + + const scope = yield* useScope(); + + // Guard A: passes + scope.around(ReplayGuard, { + *check([event], next) { + return yield* next(event); + }, + decide([event], next) { + // No opinion — let it through + return next(event); + }, + }); + + // Guard B: errors + scope.around(ReplayGuard, { + *check([event], next) { + return yield* next(event); + }, + decide([event], next) { + if (event.description.checkB === "fail") { + return { + outcome: "error", + error: new StaleInputError("Guard B failed"), + }; + } + return next(event); + }, + }); + + try { + yield* durableRun( + function* (): Workflow { + return yield* durableCall("step", () => + Promise.resolve("should-not-be-called"), + ); + }, + { stream }, + ); + throw new Error("expected StaleInputError"); + } catch (e) { + expect(e).toBeInstanceOf(StaleInputError); + expect((e as Error).message).toBe("Guard B failed"); + } + }); + + // --------------------------------------------------------------------------- + // Test 6: Check runs before replay, not during + // --------------------------------------------------------------------------- + + it("check phase runs before workflow starts", function* () { + // Note: NO Close event, so workflow actually runs and replays + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "step", someKey: "someValue" }, + result: { status: "ok", value: "result" }, + }, + // No close event - workflow runs and replays the yield + ]; + const stream = new InMemoryStream(events); + + const timeline: string[] = []; + + const scope = yield* useScope(); + + scope.around(ReplayGuard, { + *check([_event], next) { + timeline.push("check"); + return yield* next(_event); + }, + decide([event], next) { + timeline.push("decide"); + return next(event); + }, + }); + + timeline.push("before-durableRun"); + + const result = yield* durableRun( + function* (): Workflow { + timeline.push("workflow-start"); + const r = yield* durableCall("step", () => { + timeline.push("live-call"); + return Promise.resolve("should-not-be-called"); + }); + timeline.push("workflow-end"); + return r; + }, + { stream }, + ); + + timeline.push("after-durableRun"); + + // Check should run before workflow, decide during workflow + // Replay means no live-call (effect is replayed from journal) + expect(timeline).toEqual([ + "before-durableRun", + "check", // check phase runs over all Yield events first + "workflow-start", + "decide", // decide runs during replay + "workflow-end", + "after-durableRun", + ]); + + expect(result).toBe("result"); + }); + + // --------------------------------------------------------------------------- + // Test 7: Decide is pure — same inputs, same output + // --------------------------------------------------------------------------- + + it("decide is pure — consistent results for same input", function* () { + // Note: NO Close event, so workflow actually runs and replays + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "step", key: "value" }, + result: { status: "ok", value: "result" }, + }, + // No close event - workflow runs and replays the yield + ]; + + const decideResults: ReplayOutcome[] = []; + + // Run twice with fresh scopes (via run()) so middleware doesn't stack. + // Each run() creates an independent scope — matching the original + // Deno test which also used explicit run() calls. + for (let i = 0; i < 2; i++) { + const stream = new InMemoryStream([...events]); + yield* run(function* () { + const scope = yield* useScope(); + + scope.around(ReplayGuard, { + *check([event], next) { + return yield* next(event); + }, + decide([event], next) { + const outcome = next(event); + decideResults.push(outcome); + return outcome; + }, + }); + + yield* durableRun( + function* (): Workflow { + return yield* durableCall("step", () => + Promise.resolve("should-not-be-called"), + ); + }, + { stream }, + ); + }); + } + + // Both runs should have the same decide outcome + expect(decideResults.length).toBe(2); + expect(decideResults[0]).toEqual(decideResults[1]); + expect(decideResults[0]!.outcome).toBe("replay"); + }); + + // --------------------------------------------------------------------------- + // Test 8: Decide not called if identity check fails + // --------------------------------------------------------------------------- + + it("decide not called if identity check fails", function* () { + // Journal has call("stepA"), code yields call("stepX") + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "stepA", key: "value" }, + result: { status: "ok", value: "alpha" }, + }, + ]; + const stream = new InMemoryStream(events); + + const checkCalls: number[] = []; + const decideCalls: number[] = []; + + const scope = yield* useScope(); + + scope.around(ReplayGuard, { + *check([event], next) { + checkCalls.push(1); + return yield* next(event); + }, + decide([event], next) { + decideCalls.push(1); + return next(event); + }, + }); + + try { + yield* durableRun( + function* (): Workflow { + // Yields stepX but journal has stepA — identity mismatch + return yield* durableCall("stepX", () => + Promise.resolve("should-not-be-called"), + ); + }, + { stream }, + ); + throw new Error("expected DivergenceError"); + } catch (_e) { + // DivergenceError expected + } + + // Check runs before workflow (always) + expect(checkCalls.length).toBe(1); + + // Decide should NOT be called because identity check failed first + expect(decideCalls.length).toBe(0); + }); + + // --------------------------------------------------------------------------- + // Test 9: Check deduplicates file hashes via cache + // --------------------------------------------------------------------------- + + it("check deduplicates via cache", function* () { + // 5 events all referencing the same file via description.path + // Note: NO Close event, so workflow actually runs and replays + const events: DurableEvent[] = []; + for (let i = 0; i < 5; i++) { + events.push({ + type: "yield", + coroutineId: "root", + description: { type: "call", name: `step${i}`, path: "./shared.txt" }, + result: { + status: "ok", + value: { content: `result${i}`, contentHash: "abc123" }, + }, + }); + } + // No close event - workflow runs and replays all yields + + const stream = new InMemoryStream(events); + + let hashComputations = 0; + const cache = new Map(); + + const scope = yield* useScope(); + + scope.around(ReplayGuard, { + *check([event], next) { + const filePath = event.description.path; + if (typeof filePath === "string") { + if (!cache.has(filePath)) { + hashComputations++; + cache.set(filePath, "abc123"); // Simulated hash + } + } + return yield* next(event); + }, + decide([event], next) { + return next(event); + }, + }); + + yield* durableRun( + function* (): Workflow { + for (let i = 0; i < 5; i++) { + yield* durableCall>(`step${i}`, () => + Promise.resolve({ + content: "should-not-be-called", + contentHash: "abc123", + }), + ); + } + return "done"; + }, + { stream }, + ); + + // Hash should be computed only once despite 5 events + expect(hashComputations).toBe(1); + }); + + // --------------------------------------------------------------------------- + // Test 10: Guard inherited by child scopes (via durableAll) + // --------------------------------------------------------------------------- + + // Note: This test would require durableAll but we'll test the simpler case + // that the guard middleware installed on the parent scope is visible to + // effects inside durableRun. + + it("guard visible from durableRun scope", function* () { + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "step", marker: "stale" }, + result: { status: "ok", value: "result" }, + }, + ]; + const stream = new InMemoryStream(events); + + const scope = yield* useScope(); + + // Install guard on parent scope + scope.around(ReplayGuard, { + *check([event], next) { + return yield* next(event); + }, + decide([event], next) { + // Always error on events with marker: "stale" in description + if (event.description.marker === "stale") { + return { + outcome: "error", + error: new StaleInputError("Stale marker detected"), + }; + } + return next(event); + }, + }); + + try { + // The guard should be visible inside durableRun's scope + yield* durableRun( + function* (): Workflow { + return yield* durableCall("step", () => + Promise.resolve("should-not-be-called"), + ); + }, + { stream }, + ); + throw new Error("expected StaleInputError"); + } catch (e) { + expect(e).toBeInstanceOf(StaleInputError); + expect((e as Error).message).toBe("Stale marker detected"); + } + }); + + // --------------------------------------------------------------------------- + // Test 11: Default behavior is pass-through (logs are authoritative) + // --------------------------------------------------------------------------- + + it("default behavior is pass-through (logs are authoritative)", function* () { + // Event has extra description fields that WOULD be stale if a guard + // checked them, but no guard is installed — should replay normally. + const events: DurableEvent[] = [ + { + type: "yield", + coroutineId: "root", + description: { type: "call", name: "step", path: "./file.txt" }, + result: { + status: "ok", + value: { + content: "result", + contentHash: "old-hash-that-no-one-checks", + }, + }, + }, + { + type: "close", + coroutineId: "root", + result: { + status: "ok", + value: { + content: "result", + contentHash: "old-hash-that-no-one-checks", + }, + }, + }, + ]; + const stream = new InMemoryStream(events); + + // No guard installed — default behavior + const result = yield* durableRun( + function* (): Workflow> { + return yield* durableCall>("step", () => + Promise.resolve({ + content: "should-not-be-called", + contentHash: "abc123", + }), + ); + }, + { stream }, + ); + + // Replay proceeds normally — extra fields are ignored without guards + expect(result.content).toBe("result"); + }); + + // --------------------------------------------------------------------------- + // Test 12: Rich result with contentHash is written during live execution + // --------------------------------------------------------------------------- + + it("rich result with contentHash is written during live execution", function* () { + const stream = new InMemoryStream([]); + + yield* durableRun( + function* (): Workflow> { + return yield* durableCall>("readFile", () => + Promise.resolve({ + content: "file contents", + contentHash: "hash-of-file-contents", + }), + ); + }, + { stream }, + ); + + // Check that the Yield event has the rich result + const events = stream.snapshot(); + expect(events.length).toBe(2); // yield + close + + const yieldEvent = events[0]!; + expect(yieldEvent.type).toBe("yield"); + if (yieldEvent.type === "yield") { + expect(yieldEvent.result).toEqual({ + status: "ok", + value: { + content: "file contents", + contentHash: "hash-of-file-contents", + }, + }); + } + }); +}); diff --git a/durable-streams/replay-guard.ts b/durable-streams/replay-guard.ts new file mode 100644 index 00000000..3f0732d4 --- /dev/null +++ b/durable-streams/replay-guard.ts @@ -0,0 +1,138 @@ +/** + * ReplayGuard API — pluggable validation for replay staleness detection. + * + * The durable execution protocol's default behavior is "logs are authoritative" + * — the journal is unconditionally trusted during replay. ReplayGuard extends + * this with opt-in validation: guards can examine effect descriptions and + * result values to validate that recorded results are still valid against + * current state before allowing replay to proceed. + * + * Guards access `event.description.*` for effect input fields (e.g., file + * path, URL, encoding) and `event.result.value.*` for effect output fields + * (e.g., content hash, status code). There is no separate metadata field — + * inputs belong in the effect description, outputs belong in the result. + * + * The API has two phases: + * + * 1. **check** (before replay begins): Runs in generator context inside + * `durableRun`, after the journal is loaded but before the workflow starts. + * I/O is allowed — this is where file hashing, network checks, and other + * observation-gathering happens. Results are cached in middleware closures. + * + * 2. **decide** (during replay): Runs synchronously inside + * `DurableEffect.enter()`, after identity matching succeeds but before + * the stored result is fed to the generator. Must be pure and side-effect- + * free. Reads from the cache populated during the check phase. + * + * Multiple guards compose via Effection's `scope.around()`. A guard that has + * an opinion returns an outcome directly; one that doesn't calls `next(event)` + * to delegate. The first `error` outcome wins — the chain short-circuits. + * + * See replay-guard-spec.md for the full design. + */ + +import type { Api, Operation } from "effection"; +import { createApi } from "effection/experimental"; +import type { Yield } from "./types.ts"; + +// --------------------------------------------------------------------------- +// Types +// --------------------------------------------------------------------------- + +/** + * The outcome of a replay guard's decision. + * + * - "replay": Proceed with replay — use the stored journal result. + * - "error": Halt replay with an error — the journal entry is stale. + * + * Future versions may add: + * - "reexecute": Re-execute the effect and replace the journal entry. + * - "fork": Create a new execution branch from this point. + */ +export type ReplayOutcome = + | { outcome: "replay" } + | { outcome: "error"; error?: Error }; + +// --------------------------------------------------------------------------- +// API shape +// --------------------------------------------------------------------------- + +/** + * The core shape of the ReplayGuard API. + * + * - `check`: Called once per Yield event, before replay begins. Runs in + * generator context — I/O is allowed. Use to gather current state (hash + * files, check timestamps) and cache results for the decide phase. + * + * - `decide`: Called during replay, after identity matching succeeds. + * Must be pure and synchronous — no I/O, no side effects. Returns the + * replay outcome based on cached observations. + */ +interface ReplayGuardApi { + /** Phase 1: Check — gather observations before replay (I/O allowed). */ + check(event: Yield): Operation; + /** Phase 2: Decide — return replay outcome (synchronous, pure). */ + decide(event: Yield): ReplayOutcome; +} + +// --------------------------------------------------------------------------- +// Default implementation (pass-through) +// --------------------------------------------------------------------------- + +/** + * Default check — no-op. Events pass through without observation. + */ +function* defaultCheck(_event: Yield): Operation { + // No observation — pass through to next middleware or default. +} + +/** + * Default decide — always replay. This preserves "logs are authoritative" + * as the default behavior. Guards must be explicitly installed to add + * validation. + */ +function defaultDecide(_event: Yield): ReplayOutcome { + return { outcome: "replay" }; +} + +// --------------------------------------------------------------------------- +// The ReplayGuard API instance +// --------------------------------------------------------------------------- + +/** + * The ReplayGuard API. + * + * Default behavior is pass-through: `check` does nothing, `decide` returns + * `{ outcome: "replay" }`. This preserves "logs are authoritative" unless + * middleware says otherwise. + * + * Install guards via `scope.around(ReplayGuard, { ... })` before calling + * `durableRun`. Guards are inherited by child scopes through Effection's + * context inheritance. + * + * Example: + * ```ts + * function* myWorkflow(): Operation { + * const scope = yield* useScope(); + * scope.around(ReplayGuard, { + * *check([event], next) { + * // Gather observations (I/O allowed here) + * return yield* next(event); + * }, + * decide([event], next) { + * // Make decision (pure, synchronous) + * if (isStale(event)) { + * return { outcome: "error", error: new StaleInputError(...) }; + * } + * return next(event); + * }, + * }); + * + * yield* durableRun(workflow, { stream }); + * } + * ``` + */ +export const ReplayGuard: Api = createApi( + "DurableEffection.ReplayGuard", + { check: defaultCheck, decide: defaultDecide }, +); diff --git a/durable-streams/replay-index.test.ts b/durable-streams/replay-index.test.ts new file mode 100644 index 00000000..63c8321f --- /dev/null +++ b/durable-streams/replay-index.test.ts @@ -0,0 +1,350 @@ +/** + * ReplayIndex unit tests. + * + * Tests the spec-compliant replay index (§4.1) in isolation. + * No Effection dependency — pure data structure. + */ + +import { describe, it } from "@effectionx/bdd"; +import { expect } from "expect"; +import { ReplayIndex } from "./replay-index.ts"; +import type { DurableEvent, Json } from "./types.ts"; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +function yieldEvent( + coroutineId: string, + type: string, + name: string, + value?: T, +): DurableEvent { + const result = + value === undefined + ? { status: "ok" as const } + : { status: "ok" as const, value }; + + return { + type: "yield", + coroutineId, + description: { type, name }, + result, + }; +} + +function closeEvent( + coroutineId: string, + status: "ok" | "err" | "cancelled" = "ok", + value?: T, +): DurableEvent { + if (status === "ok") { + const result = + value === undefined + ? { status: "ok" as const } + : { status: "ok" as const, value }; + + return { + type: "close", + coroutineId, + result, + }; + } + if (status === "err") { + return { + type: "close", + coroutineId, + result: { status: "err", error: { message: String(value ?? "error") } }, + }; + } + return { type: "close", coroutineId, result: { status: "cancelled" } }; +} + +describe("ReplayIndex", () => { + // --------------------------------------------------------------------------- + // Empty index + // --------------------------------------------------------------------------- + + describe("empty index", () => { + it("peekYield returns undefined", function* () { + const idx = new ReplayIndex([]); + expect(idx.peekYield("root")).toBeUndefined(); + expect(idx.peekYield("root.0")).toBeUndefined(); + }); + + it("hasClose returns false", function* () { + const idx = new ReplayIndex([]); + expect(idx.hasClose("root")).toBe(false); + }); + + it("getClose returns undefined", function* () { + const idx = new ReplayIndex([]); + expect(idx.getClose("root")).toBeUndefined(); + }); + + it("isFullyReplayed returns false", function* () { + const idx = new ReplayIndex([]); + expect(idx.isFullyReplayed("root")).toBe(false); + }); + + it("getCursor returns 0", function* () { + const idx = new ReplayIndex([]); + expect(idx.getCursor("root")).toBe(0); + }); + + it("yieldCount returns 0", function* () { + const idx = new ReplayIndex([]); + expect(idx.yieldCount("root")).toBe(0); + }); + }); + + // --------------------------------------------------------------------------- + // Single coroutine, single yield + // --------------------------------------------------------------------------- + + describe("single yield", () => { + it("peekYield returns the entry", function* () { + const idx = new ReplayIndex([ + yieldEvent("root.0", "call", "fetchOrder", 42), + ]); + const entry = idx.peekYield("root.0"); + expect(entry?.description).toEqual({ type: "call", name: "fetchOrder" }); + expect(entry?.result).toEqual({ status: "ok", value: 42 }); + }); + + it("consumeYield advances cursor", function* () { + const idx = new ReplayIndex([ + yieldEvent("root.0", "call", "fetchOrder", 42), + ]); + expect(idx.getCursor("root.0")).toBe(0); + idx.consumeYield("root.0"); + expect(idx.getCursor("root.0")).toBe(1); + expect(idx.peekYield("root.0")).toBeUndefined(); + }); + + it("yieldCount is 1", function* () { + const idx = new ReplayIndex([yieldEvent("root.0", "call", "fetchOrder")]); + expect(idx.yieldCount("root.0")).toBe(1); + }); + }); + + // --------------------------------------------------------------------------- + // Multiple yields, single coroutine + // --------------------------------------------------------------------------- + + describe("multiple yields", () => { + it("cursor advances through sequence", function* () { + const idx = new ReplayIndex([ + yieldEvent("root.0", "sleep", "sleep"), + yieldEvent("root.0", "call", "transform", "ALPHA"), + ]); + + expect(idx.yieldCount("root.0")).toBe(2); + + // First peek + expect(idx.peekYield("root.0")?.description).toEqual({ + type: "sleep", + name: "sleep", + }); + idx.consumeYield("root.0"); + + // Second peek + expect(idx.peekYield("root.0")?.description).toEqual({ + type: "call", + name: "transform", + }); + expect(idx.peekYield("root.0")?.result).toEqual({ + status: "ok", + value: "ALPHA", + }); + idx.consumeYield("root.0"); + + // Exhausted + expect(idx.peekYield("root.0")).toBeUndefined(); + expect(idx.getCursor("root.0")).toBe(2); + }); + }); + + // --------------------------------------------------------------------------- + // Close events + // --------------------------------------------------------------------------- + + describe("close events", () => { + it("hasClose returns true", function* () { + const idx = new ReplayIndex([closeEvent("root.0", "ok", "done")]); + expect(idx.hasClose("root.0")).toBe(true); + }); + + it("getClose returns the event", function* () { + const close = closeEvent("root.0", "ok", "done"); + const idx = new ReplayIndex([close]); + expect(idx.getClose("root.0")).toEqual(close); + }); + + it("cancelled: getClose returns cancelled result", function* () { + const close = closeEvent("root.0", "cancelled"); + const idx = new ReplayIndex([close]); + expect(idx.getClose("root.0")?.result).toEqual({ status: "cancelled" }); + }); + + it("error: getClose returns error result", function* () { + const close = closeEvent("root.0", "err", "boom"); + const idx = new ReplayIndex([close]); + expect(idx.getClose("root.0")?.result).toEqual({ + status: "err", + error: { message: "boom" }, + }); + }); + + it("getClose returns undefined when replay is disabled", function* () { + const close = closeEvent("root.0", "ok", "done"); + const idx = new ReplayIndex([close]); + idx.disableReplay("root.0"); + expect(idx.getClose("root.0")).toBeUndefined(); + }); + }); + + // --------------------------------------------------------------------------- + // isFullyReplayed + // --------------------------------------------------------------------------- + + describe("isFullyReplayed", () => { + it("true when all yields consumed and close exists", function* () { + const idx = new ReplayIndex([ + yieldEvent("root.0", "call", "fetch", 1), + yieldEvent("root.0", "call", "transform", 2), + closeEvent("root.0", "ok", "done"), + ]); + + expect(idx.isFullyReplayed("root.0")).toBe(false); // yields not consumed + idx.consumeYield("root.0"); + expect(idx.isFullyReplayed("root.0")).toBe(false); // 1 yield remaining + idx.consumeYield("root.0"); + expect(idx.isFullyReplayed("root.0")).toBe(true); // all consumed + close exists + }); + + it("false when yields consumed but no close", function* () { + const idx = new ReplayIndex([yieldEvent("root.0", "call", "fetch", 1)]); + + idx.consumeYield("root.0"); + expect(idx.isFullyReplayed("root.0")).toBe(false); // no close + }); + + it("false when close exists but yields not consumed", function* () { + const idx = new ReplayIndex([ + yieldEvent("root.0", "call", "fetch", 1), + closeEvent("root.0"), + ]); + + expect(idx.isFullyReplayed("root.0")).toBe(false); + }); + }); + + // --------------------------------------------------------------------------- + // Multiple coroutines (interleaved events) + // --------------------------------------------------------------------------- + + describe("interleaved events", () => { + it("per-coroutine cursors are independent", function* () { + const idx = new ReplayIndex([ + yieldEvent("root.0.0", "call", "fetchUser", { name: "alice" }), + yieldEvent("root.0.1", "call", "fetchUser", { name: "bob" }), + closeEvent("root.0.0", "ok", { name: "alice" }), + closeEvent("root.0.1", "ok", { name: "bob" }), + yieldEvent("root.0", "call", "merge", "merged"), + closeEvent("root.0", "ok", "done"), + ]); + + // Each coroutine has its own cursor + expect(idx.peekYield("root.0.0")?.description.name).toBe("fetchUser"); + expect(idx.peekYield("root.0.1")?.description.name).toBe("fetchUser"); + expect(idx.peekYield("root.0")?.description.name).toBe("merge"); + + // Consuming one doesn't affect others + idx.consumeYield("root.0.0"); + expect(idx.peekYield("root.0.0")).toBeUndefined(); + expect(idx.peekYield("root.0.1")?.description.name).toBe("fetchUser"); + expect(idx.peekYield("root.0")?.description.name).toBe("merge"); + + // Full replay status + expect(idx.isFullyReplayed("root.0.0")).toBe(true); // consumed + close + expect(idx.isFullyReplayed("root.0.1")).toBe(false); // not consumed + expect(idx.isFullyReplayed("root.0")).toBe(false); // not consumed + }); + }); + + // --------------------------------------------------------------------------- + // Race scenario (from spec §10) + // --------------------------------------------------------------------------- + + describe("race scenario", () => { + it("partial execution with cancellation", function* () { + // From spec §10.1: race([op1, op2]) where op1 wins after op2 partially executed + const idx = new ReplayIndex([ + yieldEvent("root.0.1", "call", "step1", null), // op2's first effect + yieldEvent("root.0.0", "call", "fetch", "data"), // op1 completes + closeEvent("root.0.0", "ok", "data"), // op1 done + closeEvent("root.0.1", "cancelled"), // op2 cancelled + closeEvent("root.0", "ok", "data"), // race returns op1's result + ]); + + // op1 (root.0.0): one yield, then close(ok) + expect(idx.yieldCount("root.0.0")).toBe(1); + expect(idx.hasClose("root.0.0")).toBe(true); + expect(idx.getClose("root.0.0")?.result.status).toBe("ok"); + + // op2 (root.0.1): one yield, then close(cancelled) + expect(idx.yieldCount("root.0.1")).toBe(1); + expect(idx.hasClose("root.0.1")).toBe(true); + expect(idx.getClose("root.0.1")?.result.status).toBe("cancelled"); + + // race scope (root.0): no yields, just close + expect(idx.yieldCount("root.0")).toBe(0); + expect(idx.hasClose("root.0")).toBe(true); + + // After consuming op2's yield, it's fully replayed (close exists) + idx.consumeYield("root.0.1"); + expect(idx.isFullyReplayed("root.0.1")).toBe(true); + }); + }); + + // --------------------------------------------------------------------------- + // Consuming yields on unknown coroutine + // --------------------------------------------------------------------------- + + describe("unknown coroutine", () => { + it("consuming yield advances cursor", function* () { + const idx = new ReplayIndex([]); + idx.consumeYield("nonexistent"); + expect(idx.getCursor("nonexistent")).toBe(1); + expect(idx.peekYield("nonexistent")).toBeUndefined(); + }); + }); + + // --------------------------------------------------------------------------- + // Sequential workflow (from spec §11.4) + // --------------------------------------------------------------------------- + + describe("sequential workflow", () => { + it("matches spec §11.4 example", function* () { + const idx = new ReplayIndex([ + yieldEvent("root.0", "sleep", "sleep"), + yieldEvent("root.0", "call", "transform", "ALPHA"), + closeEvent("root.0", "ok", "ALPHA"), + closeEvent("root", "ok", "ALPHA"), + ]); + + // root.0 has 2 yields + expect(idx.yieldCount("root.0")).toBe(2); + + // Consume both + idx.consumeYield("root.0"); + idx.consumeYield("root.0"); + expect(idx.isFullyReplayed("root.0")).toBe(true); + + // root has 0 yields but has a close + expect(idx.yieldCount("root")).toBe(0); + expect(idx.isFullyReplayed("root")).toBe(true); + }); + }); +}); diff --git a/durable-streams/replay-index.ts b/durable-streams/replay-index.ts new file mode 100644 index 00000000..9a0e4a6f --- /dev/null +++ b/durable-streams/replay-index.ts @@ -0,0 +1,158 @@ +/** + * ReplayIndex — derived, in-memory structure built from the stream on startup. + * + * Provides per-coroutine cursored access to Yield events and keyed access + * to Close events. See spec §4.1. + */ + +import type { + Close, + CoroutineId, + DurableEvent, + EffectDescription, + Result, +} from "./types.ts"; + +export interface YieldEntry { + description: EffectDescription; + result: Result; +} + +export class ReplayIndex { + private yields = new Map(); + private cursors = new Map(); + private closes = new Map(); + /** Coroutines where replay has been disabled (run-live mode). */ + private disabled = new Set(); + + constructor(events: DurableEvent[]) { + for (const event of events) { + if (event.type === "yield") { + let list = this.yields.get(event.coroutineId); + if (!list) { + list = []; + this.yields.set(event.coroutineId, list); + } + list.push({ + description: event.description, + result: event.result, + }); + } + if (event.type === "close") { + this.closes.set(event.coroutineId, event); + } + } + } + + /** + * Disable replay for a coroutine (run-live mode). + * + * Once disabled, peekYield() returns undefined and hasClose() returns + * false for this coroutine, so all subsequent effects execute live + * and no further divergence checks are triggered. + */ + disableReplay(coroutineId: CoroutineId): void { + this.disabled.add(coroutineId); + } + + /** Returns true if replay has been disabled for this coroutine. */ + isReplayDisabled(coroutineId: CoroutineId): boolean { + return this.disabled.has(coroutineId); + } + + /** + * Returns the next unconsumed yield for this coroutine, + * or undefined if the cursor is past the end or replay is disabled. + */ + peekYield(coroutineId: CoroutineId): YieldEntry | undefined { + if (this.disabled.has(coroutineId)) return undefined; + const list = this.yields.get(coroutineId); + const cursor = this.cursors.get(coroutineId) ?? 0; + return list?.[cursor]; + } + + /** Advances the cursor for this coroutine by one position. */ + consumeYield(coroutineId: CoroutineId): void { + const cursor = this.cursors.get(coroutineId) ?? 0; + this.cursors.set(coroutineId, cursor + 1); + } + + /** Returns the current cursor position for this coroutine. */ + getCursor(coroutineId: CoroutineId): number { + return this.cursors.get(coroutineId) ?? 0; + } + + /** Returns true if a Close event exists for this coroutine (and replay is not disabled). */ + hasClose(coroutineId: CoroutineId): boolean { + if (this.disabled.has(coroutineId)) return false; + return this.closes.has(coroutineId); + } + + /** Returns the Close event for this coroutine, or undefined. */ + getClose(coroutineId: CoroutineId): Close | undefined { + if (this.disabled.has(coroutineId)) return undefined; + return this.closes.get(coroutineId); + } + + /** + * Returns true if any non-disabled coroutine still has unconsumed yields. + * + * This is used by durableRun to detect early-return divergence even when + * unconsumed entries belong to child coroutines rather than the root. + */ + hasAnyUnconsumedYields(): boolean { + for (const [coroutineId, entries] of this.yields.entries()) { + if (this.disabled.has(coroutineId)) continue; + const cursor = this.cursors.get(coroutineId) ?? 0; + if (cursor < entries.length) return true; + } + return false; + } + + /** + * Return the first non-disabled coroutine with unconsumed yields. + * + * NOTE: Closed coroutines are skipped because their yields were consumed + * by the child's own replay path (via runDurableChild). This means + * orphaned children (recorded in the journal but never spawned in the + * current run) are not detected here. Orphan detection requires tracking + * which coroutine IDs were visited during the current run, which is a + * future enhancement. + */ + firstUnconsumed(): + | { + coroutineId: CoroutineId; + cursor: number; + totalYields: number; + } + | undefined { + for (const [coroutineId, entries] of this.yields.entries()) { + if (this.disabled.has(coroutineId)) continue; + if (this.closes.has(coroutineId)) continue; + const cursor = this.cursors.get(coroutineId) ?? 0; + if (cursor < entries.length) { + return { coroutineId, cursor, totalYields: entries.length }; + } + } + return undefined; + } + + /** + * Returns true if the cursor for this coroutine has been fully consumed + * AND a Close event exists. This means the coroutine completed in a + * previous run and can be treated as fully replayed. + * + * Returns false if replay is disabled (run-live mode). + */ + isFullyReplayed(coroutineId: CoroutineId): boolean { + if (this.disabled.has(coroutineId)) return false; + return ( + this.peekYield(coroutineId) === undefined && this.hasClose(coroutineId) + ); + } + + /** Returns the total number of yield entries for this coroutine. */ + yieldCount(coroutineId: CoroutineId): number { + return this.yields.get(coroutineId)?.length ?? 0; + } +} diff --git a/durable-streams/run.ts b/durable-streams/run.ts new file mode 100644 index 00000000..ea2b1b21 --- /dev/null +++ b/durable-streams/run.ts @@ -0,0 +1,180 @@ +/** + * durableRun — entry point for durable workflow execution. + * + * An Operation that reads the event stream, builds the ReplayIndex, + * sets DurableContext on the current scope, runs the workflow, and emits + * a Close event when the workflow terminates. + * + * Because durableRun is an Operation, it inherits the caller's Effection + * scope — including any middleware installed via scope.around(). This is + * how divergence policy overrides work: the caller installs middleware + * before yield*-ing into durableRun. See DEC-032. + * + * See integration doc §10, protocol spec §4. + */ + +import { useScope } from "effection"; +import type { Operation, Scope } from "effection"; +import { DurableCtx } from "./context.ts"; +import { EarlyReturnDivergenceError } from "./errors.ts"; +import { ReplayGuard } from "./replay-guard.ts"; +import { ReplayIndex } from "./replay-index.ts"; +import { deserializeError, serializeError } from "./serialize.ts"; +import type { DurableStream } from "./stream.ts"; +import type { + Close, + DurableEvent, + Json, + Workflow, + WorkflowValue, +} from "./types.ts"; + +/** + * Run the ReplayGuard check phase over all Yield events. + * + * This is Phase 1 of replay guard validation — it runs before the workflow + * starts, in generator context where I/O is allowed. Middleware uses this + * phase to gather observations (hash files, check timestamps) and cache + * results for the decide phase. + * + * See replay-guard-spec.md §5.5. + */ +function* runCheckPhase(events: DurableEvent[], scope: Scope): Operation { + for (const event of events) { + if (event.type === "yield") { + yield* ReplayGuard.invoke(scope, "check", [event]); + } + } +} + +/** + * Options for durableRun. + */ +export interface DurableRunOptions { + /** The durable stream to read from and append to. */ + stream: DurableStream; + /** Coroutine ID for the root workflow. Defaults to "root". */ + coroutineId?: string; +} + +/** + * Execute a durable workflow. + * + * 1. Reads all events from the stream and builds a ReplayIndex. + * 2. Sets DurableContext on the current scope (inherited from caller). + * 3. Runs the workflow — replayed effects resolve synchronously from + * the index; live effects execute and persist before resuming. + * 4. On completion, appends a Close event to the stream. + * 5. On error, appends a Close(err) event. + * + * Returns the workflow's result value. + * + * Usage: + * // From async code (standalone): + * await run(() => durableRun(workflow, { stream })); + * + * // From inside an Effection generator (inherits scope): + * const result = yield* durableRun(workflow, { stream }); + */ +export function* durableRun( + workflow: () => Workflow | Operation, + options: DurableRunOptions, +): Operation { + const { stream, coroutineId = "root" } = options; + + // Read all events and build replay index + const events = yield* stream.readAll(); + const replayIndex = new ReplayIndex(events); + + // Inherit the caller's scope — middleware (e.g., Divergence, ReplayGuard) + // is already installed by the caller before yield*-ing into durableRun. + const scope = yield* useScope(); + + scope.set(DurableCtx, { + replayIndex, + stream, + coroutineId, + childCounter: 0, + }); + + // ── REPLAY GUARD: Check phase ── + // Run before the workflow starts. Middleware can yield* for I/O (hash + // files, make network requests) to gather observations for the decide + // phase. The check loop iterates all Yield events in journal order. + // See replay-guard-spec.md §5.5. + yield* runCheckPhase(events, scope); + + // If the root coroutine already has a Close event in the journal, + // the workflow completed in a previous run. Return the stored result + // directly without re-running the workflow. + if (replayIndex.hasClose(coroutineId)) { + const closeEvent = replayIndex.getClose(coroutineId)!; + if (closeEvent.result.status === "ok") { + return closeEvent.result.value as T; + } else if (closeEvent.result.status === "err") { + throw deserializeError(closeEvent.result.error); + } else { + throw new Error("Workflow was cancelled"); + } + } + + try { + // Workflow is structurally assignable to Operation, so + // yield* accepts it directly — no cast needed. + const result: T = yield* workflow(); + + // §6.3: Check for early return divergence. + // If the generator returned but the replay index has unconsumed yields, + // the workflow has diverged. Skip this check when replay has been + // disabled (run-live mode) — the workflow intentionally diverged and + // the Divergence API already approved it. + if (!replayIndex.isReplayDisabled(coroutineId)) { + const unconsumed = replayIndex.firstUnconsumed(); + if (unconsumed) { + throw new EarlyReturnDivergenceError( + unconsumed.coroutineId, + unconsumed.cursor, + unconsumed.totalYields, + ); + } + } + + const closeEvent: Close = { + type: "close", + coroutineId, + result: { status: "ok", value: result as Json }, + }; + + yield* stream.append(closeEvent); + + return result; + } catch (error) { + // Normalize the error once — use the same Error object for both the + // Close event and the rethrow so that live runs and replayed Close + // events carry identical error shapes. + const primary = error instanceof Error ? error : new Error(String(error)); + const closeEvent: Close = { + type: "close", + coroutineId, + result: { + status: "err", + error: serializeError(primary), + }, + }; + + try { + yield* stream.append(closeEvent); + } catch (appendError) { + const appendFailure = + appendError instanceof Error + ? appendError + : new Error(String(appendError)); + throw new AggregateError( + [primary, appendFailure], + "Workflow failed and Close append also failed", + ); + } + + throw primary; + } +} diff --git a/durable-streams/runtime.ts b/durable-streams/runtime.ts new file mode 100644 index 00000000..a791fcb6 --- /dev/null +++ b/durable-streams/runtime.ts @@ -0,0 +1,112 @@ +/** + * DurableRuntime — platform-agnostic runtime abstraction for durable effects. + * + * Effects must not depend on Node-specific or Deno-specific APIs directly. + * This interface provides all platform operations. Every I/O method returns + * `Operation`, not `Promise`. Cancellation flows through Effection's + * structured concurrency — when a scope tears down, the operation is + * cancelled. No `AbortSignal` in the interface. + * + * Install via `scope.set(DurableRuntimeCtx, nodeRuntime())` before calling + * `durableRun`. Effects access the runtime inside `createDurableOperation` + * callbacks via `scope.expect(DurableRuntimeCtx)`. + */ + +import { createContext } from "effection"; +import type { Context, Operation } from "effection"; + +/** + * Minimal response headers interface. + * + * Uses a minimal interface instead of the global `Headers` type to avoid + * requiring DOM lib types in tsconfig. + */ +export interface ResponseHeaders { + get(key: string): string | null; +} + +/** + * Response shape returned by `DurableRuntime.fetch()`. + * + * Both the response object and `text()` are Operation-native — no Promises + * cross the interface boundary. + */ +export interface RuntimeFetchResponse { + status: number; + headers: ResponseHeaders; + /** Read the response body as text. */ + text(): Operation; +} + +/** + * Result of a `stat` call. + * + * For missing paths `stat` returns `{ exists: false, isFile: false, isDirectory: false }` + * instead of throwing — "does this exist?" has "no" as a valid answer. + */ +export interface StatResult { + exists: boolean; + isFile: boolean; + isDirectory: boolean; +} + +/** + * Platform-agnostic runtime for durable effects. + * + * Implementations exist for Node.js (`nodeRuntime()` in `@effectionx/durable-effects`) + * and testing (`stubRuntime()` in `@effectionx/durable-effects`). + */ +export interface DurableRuntime { + /** Execute a subprocess. Cancellation kills the process. */ + exec(options: { + command: string[]; + cwd?: string; + env?: Record; + timeout?: number; + }): Operation<{ exitCode: number; stdout: string; stderr: string }>; + + /** Read a text file. */ + readTextFile(path: string): Operation; + + /** + * Check file/directory existence and type. Never throws for missing paths. + * + * Returns `{ exists: false, isFile: false, isDirectory: false }` when the + * path does not exist. Permission errors and other filesystem errors still + * throw — they indicate a real problem, not "file doesn't exist." + */ + stat(path: string): Operation; + + /** Expand glob patterns. Returns relative paths with isFile flag. */ + glob(options: { + patterns: string[]; + root: string; + exclude?: string[]; + }): Operation>; + + /** Make an HTTP request. Cancellation aborts the request. */ + fetch( + input: string, + init?: { + method?: string; + headers?: Record; + body?: string; + timeout?: number; + }, + ): Operation; + + /** Read an environment variable. Returns undefined if not set. */ + env(name: string): string | undefined; + + /** Return platform information. */ + platform(): { os: string; arch: string }; +} + +/** + * Effection Context for the DurableRuntime. + * + * Set on the scope before calling `durableRun()`. Effects access the + * runtime via `scope.expect(DurableRuntimeCtx)`. + */ +export const DurableRuntimeCtx: Context = + createContext("@effectionx/durable-streams/runtime"); diff --git a/durable-streams/serialize.ts b/durable-streams/serialize.ts new file mode 100644 index 00000000..90ed49a3 --- /dev/null +++ b/durable-streams/serialize.ts @@ -0,0 +1,75 @@ +/** + * Serialization utilities for the durable execution protocol. + * + * Converts between: + * - Protocol Result ({ status: "ok" | "err" | "cancelled" }) + * - Effection Result ({ ok: true, value } | { ok: false, error }) + * - Error ↔ SerializedError + */ + +import type { + EffectionResult, + Json, + Result, + SerializedError, +} from "./types.ts"; + +// --------------------------------------------------------------------------- +// Error serialization +// --------------------------------------------------------------------------- + +/** Serialize an Error to a JSON-safe SerializedError. */ +export function serializeError(error: Error): SerializedError { + return { + message: error.message, + name: error.name, + stack: error.stack, + }; +} + +/** Deserialize a SerializedError back to an Error. */ +export function deserializeError(se: SerializedError): Error { + const error = new Error(se.message); + if (se.name) error.name = se.name; + if (se.stack) error.stack = se.stack; + return error; +} + +// --------------------------------------------------------------------------- +// Result conversion: Protocol ↔ Effection +// --------------------------------------------------------------------------- + +/** + * Convert a protocol Result to an Effection Result. + * + * - ok → { ok: true, value } + * - err → { ok: false, error } (deserialized) + * - cancelled → { ok: false, error } with a CancelledError + * + * The value is returned as-is (Json). The caller is responsible for any + * narrowing to a specific type T. + */ +export function protocolToEffection(result: Result): EffectionResult { + switch (result.status) { + case "ok": + return { ok: true, value: result.value as T }; + case "err": + return { ok: false, error: deserializeError(result.error) }; + case "cancelled": + return { ok: false, error: new Error("cancelled") }; + } +} + +/** + * Convert an Effection Result to a protocol Result. + * + * The value must be JSON-serializable. This function does NOT validate + * serializability — that is the caller's responsibility. + */ +export function effectionToProtocol(result: EffectionResult): Result { + if (result.ok) { + return { status: "ok", value: result.value as Json }; + } else { + return { status: "err", error: serializeError(result.error) }; + } +} diff --git a/durable-streams/smoke.test.ts b/durable-streams/smoke.test.ts new file mode 100644 index 00000000..77f71788 --- /dev/null +++ b/durable-streams/smoke.test.ts @@ -0,0 +1,37 @@ +/** + * Smoke test to verify project scaffolding works. + */ + +import { describe, it } from "@effectionx/bdd"; +import { expect } from "expect"; +import { InMemoryStream, ReplayIndex } from "./mod.ts"; +import type { DurableEvent } from "./mod.ts"; + +describe("smoke tests", () => { + it("ReplayIndex can be constructed with empty events", function* () { + const index = new ReplayIndex([]); + expect(index.peekYield("root")).toBeUndefined(); + expect(index.hasClose("root")).toBe(false); + expect(index.isFullyReplayed("root")).toBe(false); + }); + + it("InMemoryStream starts empty", function* () { + const stream = new InMemoryStream(); + expect(stream.snapshot()).toEqual([]); + }); + + it("InMemoryStream stores and retrieves events", function* () { + const stream = new InMemoryStream(); + const event: DurableEvent = { + type: "yield", + coroutineId: "root.0", + description: { type: "call", name: "fetchOrder" }, + result: { status: "ok", value: 42 }, + }; + yield* stream.append(event); + const events = stream.snapshot(); + expect(events.length).toBe(1); + expect(events[0]).toEqual(event); + expect(stream.appendCount).toBe(1); + }); +}); diff --git a/durable-streams/specs/DECISIONS.md b/durable-streams/specs/DECISIONS.md new file mode 100644 index 00000000..2590eebd --- /dev/null +++ b/durable-streams/specs/DECISIONS.md @@ -0,0 +1,841 @@ +# Decision Log + +Every architectural, technical, and implementation decision made during the +build of the durable execution integration is recorded here. Decisions are +append-only — superseded decisions are marked `[SUPERSEDED by DEC-NNN]` but +never deleted. + +Updated before completion of every phase and committed at the end of each phase. + +--- + +## DEC-001: Use Deno as project runtime [SUPERSEDED] + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** Need a runtime for the project. Effection 4.x uses Deno as its + primary development tool and publishes to JSR. +- **Options considered:** + 1. Node.js with npm/TypeScript toolchain + 2. Deno with JSR imports +- **Decision:** Deno as the project runtime (deno.json, deno test, JSR imports) +- **Rationale:** User preference. Effection itself uses Deno for development. + JSR imports are first-class. `deno test` eliminates the need for a separate + test runner. +- **Consequences:** npm packages (durable-streams) are imported via `npm:` + specifiers. Native addons (lmdb in @durable-streams/server) may need + `nodeModulesDir: "auto"` if build scripts are required. +- **Update:** The project has since moved to the `effectionx` monorepo using + Node.js 22 with pnpm, TypeScript 5+, and the Node.js test runner. The + `@effectionx/durable-streams` package is published to npm, not JSR. + +## DEC-002: Target effection 4.1.0-alpha.5 with /experimental endpoint + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** Need the latest Effection with the Api middleware system for + intercepting scope lifecycle events. +- **Options considered:** + 1. Effection 4.0.2 (stable) — no `/experimental` endpoint + 2. Effection 4.1.0-alpha.5 — has `createApi`, `api.Scope`, `api.Main` +- **Decision:** Use `@effection/effection@4.1.0-alpha.5` +- **Rationale:** The `/experimental` endpoint exposes `api.Scope` with + `create`, `destroy`, `set`, `delete` operations and `around()` middleware. + This is the extension point needed for intercepting scope destruction to + emit Close events (future phases). The alpha is published to JSR and works + with Deno. +- **Consequences:** API surface may change before 4.1 stable. We depend on + the experimental endpoint which is explicitly unstable. We should pin the + exact version and be prepared to adapt. +- **Update:** Now pinned at `effection@4.1.0-alpha.7` via pnpm overrides in + the effectionx monorepo root `package.json`. Tracking issue #181 to remove + the override when a stable release is available. + +## DEC-003: Single package structure + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** Could structure as a monorepo with separate packages + (core protocol, durable-streams backend, test utilities) or a single package. +- **Options considered:** + 1. Monorepo with separate packages from day one + 2. Single package, split later when boundaries stabilize +- **Decision:** Single package +- **Rationale:** The boundaries between core protocol, effects, and backend + adapter are not yet proven. Premature separation adds overhead without + benefit. Split when the interfaces are stable and there's a concrete need + (e.g., supporting a second backend). +- **Consequences:** All code lives under `lib/`. The module entry point is + `lib/mod.ts`. Re-exports control the public API surface. +- **Update:** The project has since been split into two packages in the + effectionx monorepo: `@effectionx/durable-streams` (core protocol, replay, + effects, combinators) and `@effectionx/durable-effects` (higher-level + durable operations like durableExec, durableFetch, etc.). Source files live + at the package root (e.g., `effect.ts`, `run.ts`) rather than under `lib/`. + The entry point is `mod.ts`. + +## DEC-004: Use @std/assert for test assertions [SUPERSEDED] + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** Need an assertion library for `deno test`. +- **Options considered:** + 1. `https://deno.land/std` URL imports (legacy style) + 2. `jsr:@std/assert` (modern JSR-style) +- **Decision:** `jsr:@std/assert@1` via import map +- **Rationale:** JSR is the standard for Deno dependencies. Avoids uncached + URL resolution issues. +- **Consequences:** Added to deno.json imports. +- **Update:** Tests now use the Node.js test runner (`node --test`) with + `@effectionx/bdd` for describe/it/beforeEach and `expect` for assertions. + +## DEC-005: Sequential workflows only in initial scope + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** The protocol spec covers sequential execution, fork/join, + races, cancellation, and version gates. Implementing everything at once + is risky. +- **Options considered:** + 1. Full set (call, sleep, action, spawn, all, race, versionCheck) from start + 2. Minimal: core + call + sleep, add spawn/all/race in a second phase +- **Decision:** Minimal initial scope: ReplayIndex, DurableEffect, + durableCall, durableSleep, versionCheck, durableRun. No spawn/all/race. +- **Rationale:** Enough to validate core replay correctness (Tier 1) and + divergence detection (Tier 2) — the fundamental protocol. Structured + concurrency (Tier 3-4) adds significant complexity that benefits from + a solid foundation. +- **Consequences:** Spec tests 15-27 (structured concurrency, deterministic + identity) are deferred. Close event emission can be simplified to + try/finally in durableRun rather than scope middleware. + +## DEC-006: Tier 1-2 tests first + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** The spec defines 37 tests across 7 tiers. Need to decide + initial test coverage target. +- **Options considered:** + 1. Tier 1-2 first (tests 1-14) + 2. Tier 1-4 all at once +- **Decision:** Tier 1-2 first +- **Rationale:** Core replay correctness (Tier 1, tests 1-7) and divergence + detection (Tier 2, tests 8-14) validate the fundamental protocol. These + are achievable with sequential workflows. Tier 3-4 require spawn/all/race. +- **Consequences:** Tests 15-37 deferred to future phases. + +## DEC-007: Protocol types are Effection-independent + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** The protocol types (DurableEvent, Yield, Close, Result, etc.) + could depend on Effection types or be standalone. +- **Decision:** Protocol types in `types.ts` have zero Effection imports. + The DurableEffect and Workflow types that bridge to Effection are defined + separately (also in types.ts for now, but with a placeholder shape). +- **Rationale:** The protocol is designed to be runtime-agnostic (spec §1.2). + Keeping types independent enables potential reuse with other runtimes and + makes the types testable without Effection. +- **Consequences:** The DurableEffect interface in types.ts uses a generic + shape for `enter()` that will be aligned with Effection's exact Effect + interface in Phase 1. + +## DEC-008: Three distinct divergence error types + +- **Phase:** 0 (Scaffolding) +- **Date:** 2026-02-28 +- **Context:** Spec §6.2-6.3 defines three divergence conditions: description + mismatch, generator finishes early, generator continues past close. +- **Options considered:** + 1. Single DivergenceError class with a `kind` field + 2. Three separate error classes +- **Decision:** Three classes: `DivergenceError`, `EarlyReturnDivergenceError`, + `ContinuePastCloseDivergenceError`. All share `name = "DivergenceError"`. +- **Rationale:** Each carries different diagnostic fields (expected/actual + descriptions vs. consumed/total counts). Separate classes enable precise + `instanceof` checks in tests while sharing the same error name for catch-all + handling. +- **Consequences:** Error handling code can match on the common name + `"DivergenceError"` or use instanceof for specific cases. + +## DEC-009: Workflow = Generator, T, unknown> + +- **Phase:** 1 (Protocol Types) +- **Date:** 2026-02-28 +- **Context:** Need a type that constrains generator yields to durable effects + only, while remaining assignable to Effection's Operation. +- **Options considered:** + 1. `Iterable, T, unknown>` — TypeScript's Iterable + only has 1 type parameter in the standard lib, cannot constrain yields. + 2. `Generator, T, unknown>` — has 3 type parameters + (Yield, Return, Next). + 3. Custom interface extending both Generator and Operation. +- **Decision:** `Generator, T, unknown>` +- **Rationale:** Generator's 3 type parameters give TypeScript enough + information to enforce the yield constraint. When a user writes + `function*(): Workflow`, TS checks that every `yield` expression + produces a value assignable to `DurableEffect`. Verified: + `yield* sleep(1000)` inside a Workflow produces TS2741 error. +- **Consequences:** Workflow generators use `yield` (not `yield*`) for direct + DurableEffect interaction, and `yield*` for delegating to other Workflows. + The cast `as T` is needed when `yield`-ing a DurableEffect since TS types + the yield expression as `unknown`. + +## DEC-010: DurableEffect mirrors Effection's Effect interface shape exactly + +- **Phase:** 1 (Protocol Types) +- **Date:** 2026-02-28 +- **Context:** DurableEffect needs to be structurally compatible with + Effection's `Effect` interface so the reducer processes it identically. +- **Decision:** DurableEffect has the same `description: string` and + `enter(resolve, routine)` signature as Effect, plus the additional + `effectDescription: EffectDescription` field. +- **Rationale:** Effection's Effect uses: + - `enter(resolve: Resolve>, routine: Coroutine)` + - returns `(resolve: Resolve>) => void` (teardown) + - `Result = { ok: true, value: T } | { ok: false, error: Error }` + DurableEffect replicates this exactly. The extra field doesn't affect + structural compatibility — the reducer ignores unknown properties. +- **Consequences:** Two different "Result" types exist — Effection's internal + `{ ok, value/error }` and the protocol's `{ status, value/error }`. We + define `EffectionResult` in types.ts to bridge them without importing + from Effection. + +## DEC-011: CoroutineView — minimal interface instead of importing Coroutine + +- **Phase:** 1 (Protocol Types) +- **Date:** 2026-02-28 +- **Context:** The `enter()` callback receives an Effection `Coroutine` object. + We need `routine.scope` to read DurableContext. Coroutine is marked + `@ignore` in Effection's types (not part of public API). +- **Options considered:** + 1. Import Coroutine type from Effection internals + 2. Use `unknown` and cast at runtime + 3. Define a minimal CoroutineView interface with only what we need +- **Decision:** Define `CoroutineView` with `scope` property typed to match + Scope's `get()`, `expect()`, `set()` methods. +- **Rationale:** Avoids depending on Effection's private API surface. The + minimal interface documents exactly which Coroutine fields we rely on. + If Effection's internal shape changes, the break is localized to this + interface. +- **Consequences:** At runtime, `enter()` receives the full Coroutine object. + TypeScript sees only our CoroutineView. This works because we only access + `scope.expect(context)` which is a public Scope method. + +## DEC-012: Verified — routine.scope is accessible in enter() callback + +- **Phase:** 1 (Protocol Types) +- **Date:** 2026-02-28 +- **Context:** The key risk identified in the plan was whether `routine.scope` + is accessible from within `enter()`. Needed to confirm from Effection 4.1 + alpha source. +- **Decision:** Confirmed. Effection's `Coroutine` interface in + `lib/types.ts` (line ~465) has `scope: Scope`. The reducer passes the + full Coroutine object to `enter()`. We can access `routine.scope.expect(ctx)` + to read DurableContext from within a DurableEffect's enter method. +- **Rationale:** Verified by reading Effection 4.1.0-alpha.5 source: + `interface Coroutine { scope: Scope; data: { ... }; next(...); return(...); }` +- **Consequences:** No workaround needed. The direct approach from the + integration doc works. + +## DEC-013: ReplayIndex follows spec §4.1 exactly with no extensions + +- **Phase:** 2 (ReplayIndex) +- **Date:** 2026-02-28 +- **Context:** The spec provides a reference implementation of ReplayIndex in + §4.1. Could add extra features (e.g., event filtering, offset tracking). +- **Decision:** Follow the spec exactly. Only additions are `getCursor()` and + `yieldCount()` which are trivial derived accessors for diagnostics/testing. +- **Rationale:** The ReplayIndex is a critical correctness component. Staying + minimal and spec-aligned reduces the risk of subtle bugs. Extra features + can be added if needed. +- **Consequences:** All replay logic depends on this class. It is thoroughly + tested (21 tests covering empty index, single/multiple yields, close events, + interleaved coroutines, race scenarios, and spec examples). + +## DEC-014: createDurableEffect handles replay/live dispatch inside enter() + +- **Phase:** 3 (Durable Runner) +- **Date:** 2026-02-28 +- **Context:** The protocol requires each durable effect to check the replay + index, validate descriptions, and either feed stored results or execute + live with persist-before-resume. This logic could live in a central + runner/reducer or inside each effect. +- **Decision:** Each `DurableEffect.enter()` handles its own replay/live + dispatch internally, reading `DurableContext` from the scope via + `routine.scope.expect(DurableCtx)`. +- **Rationale:** Keeps the Effection reducer completely untouched. The reducer + calls `enter()` on every effect — whether `enter()` resolves synchronously + (replay) or asynchronously (live + persist) is invisible to it. This is + the architecture from the integration doc §5.1. +- **Consequences:** No changes to Effection internals. The `createDurableEffect` + factory encapsulates all replay/persistence logic. Each workflow-enabled + effect (durableSleep, durableCall, etc.) is a thin wrapper over this factory. + +## DEC-015: Workflow is directly assignable to Operation — no casts needed + +- **Phase:** 3 (Durable Runner) +- **Date:** 2026-02-28 +- **Context:** `durableRun` calls `scope.run(workflow)` where workflow returns + `Workflow` (which is `Generator, T, unknown>`). + Need to confirm this is assignable to Effection's `Operation`. +- **Options considered:** + 1. Cast `workflow as () => Operation` or use `as any` + 2. Rely on structural assignability +- **Decision:** No cast needed. `DurableEffect` extends `Effect` structurally, + and TypeScript's covariant yield type means `Generator` + is assignable to the iterator type that `Operation` expects. +- **Rationale:** Verified empirically — `scope.run(workflow)` compiles without + any type assertions. This confirms the type system design from DEC-009/010. +- **Consequences:** The type boundary between Workflow and Operation is seamless. + +## DEC-016: durableRun short-circuits on existing Close event + +- **Phase:** 3 (Durable Runner) +- **Date:** 2026-02-28 +- **Context:** When `durableRun` is called with a stream that already contains + a Close event for the root coroutine, should it re-run the workflow or + return the stored result directly? +- **Decision:** Short-circuit. If `replayIndex.hasClose(coroutineId)` is true, + return the stored result from the Close event without creating a scope or + running the workflow. +- **Rationale:** A Close event means the workflow completed in a previous run. + Re-running it would be wasteful and could produce unexpected behavior + (e.g., side effects from live effects). The stored result is the canonical + outcome. +- **Consequences:** Fully-completed workflows return instantly. The early-return + check uses `hasClose()` (not `isFullyReplayed()`, which requires cursor + advancement that hasn't happened yet). + +## DEC-017: Persist-before-resume via Strategy B (async append + deferred resolve) + +- **Phase:** 3 (Durable Runner) +- **Date:** 2026-02-28 +- **Context:** The spec §5 defines the persist-before-resume invariant with + three strategies. Need to choose one for the Effection integration. +- **Decision:** Strategy B — the effect's `enter()` calls `stream.append(event)` + and places `resolve()` inside the `.then()` callback. The generator does + not advance until the durable write completes. +- **Rationale:** This is the natural fit for Effection's async resolve model. + The reducer waits for `resolve()` to be called, so deferring it until after + the append guarantees persist-before-resume. Verified by the ordering test + (execute → persist → resume for each step). +- **Consequences:** Live execution has one async hop per effect (the stream + append). During replay, `resolve()` is called synchronously — zero async + overhead. + +## DEC-018: durableCall constrains T extends Json for serializability + +- **Phase:** 3 (Durable Runner) +- **Date:** 2026-02-28 +- **Context:** `durableCall(name, fn)` stores the function's return value + in the journal. The value must be JSON-serializable per the protocol. +- **Decision:** Constrain `T extends Json` at the type level. +- **Rationale:** Catches non-serializable return values at compile time rather + than silently producing corrupt journal entries. The `Json` type from + `types.ts` covers all JSON-serializable values. +- **Consequences:** Users must ensure their async functions return JSON-compatible + values. Complex objects (Dates, class instances) need explicit serialization. + The constraint is intentionally strict — relaxing it later is easy, but + tightening it would be a breaking change. + +## DEC-019: Delegate durableAll to Effection's native all() via child Operation wrapping + +- **Phase:** 4 (Structured Concurrency) +- **Date:** 2026-02-28 +- **Context:** `durableAll` needs to run multiple child workflows concurrently, + wait for all to complete, and propagate errors so that parent generators + can catch them via try/catch (error boundary pattern, spec §7.3). +- **Options considered:** + 1. `scoped()` + `spawn()` + sequential join loop + 2. `spawn()` + sequential join loop + manual `task.halt()` on error + 3. Wrap children as `Operation` objects, delegate to Effection's `all()` +- **Decision:** Option 3 — wrap each child workflow in an Operation that + runs `runDurableChild()`, then pass the array to Effection's `all()`. +- **Rationale:** Effection's `all()` uses the internal `trap()` mechanism + which provides proper error isolation — child errors are catchable by + the caller via try/catch, and remaining siblings are cancelled on failure. + Option 1 (`scoped()`) was tried first but `scoped()` transforms child + errors into "halted" when the child's finally/catch blocks perform async + work (via `yield* call()`), because scope teardown kills the async + operation mid-flight. Option 2 works for error propagation but errors + from spawned children fail the parent scope directly (not catchable by + the parent generator's try/catch). +- **Consequences:** `durableAll` and `durableRace` delegate to Effection's + native combinators. The durable layer wraps each child in an Operation + that (1) checks for replay short-circuit, (2) sets DurableCtx with a + child coroutineId, and (3) emits Close events in finally. This is a + thin wrapper that preserves Effection's error semantics perfectly. + +## DEC-020: Error catching from durableAll — three approaches and their failure modes + +- **Phase:** 4 (Structured Concurrency) +- **Date:** 2026-02-28 +- **Context:** When a child in `durableAll` throws, the error must propagate + to the parent in a way that: (a) preserves the original error identity + (message, stack), (b) allows try/catch in the parent generator to intercept + it, and (c) properly cancels sibling children. Three approaches were tested. +- **Finding — `scoped()` + `spawn()` + join loop:** + `scoped()` creates a hermetic scope. When a spawned child throws, the + scope is torn down. If the child's error/finally handler performs any + async operation (e.g., `yield* call(() => stream.append(closeEvent))`), + the scope teardown kills that async operation mid-flight, and the error + that reaches the caller is "halted" (from `task.ts:98`) rather than the + original "child-boom". Even without async in the error path, the join + loop's `yield* task` receives "halted" because the scope destruction + interrupts the task's iterator. **This approach masks error identity.** +- **Finding — bare `spawn()` + join loop + manual `task.halt()`:** + Without `scoped()`, spawned children that throw propagate the error + through Effection's scope hierarchy. The parent task fails with the + correct error message. However, errors from spawned children are + delivered via `iterator.return()` (scope cancellation), not via the + generator's normal execution path. This means try/catch in the parent + generator **cannot** intercept the error — it bypasses the catch block + entirely. **This approach breaks error boundaries.** +- **Finding — delegate to Effection's native `all()`:** + Effection's `all()` uses the internal `trap()` function which creates + a proper catch boundary. Child errors are caught, remaining siblings are + halted, and the error re-thrown in a way that is catchable by the caller's + try/catch. Error identity is preserved. **This is the only approach that + satisfies all three requirements.** +- **Decision:** Use Effection's native `all()` and `race()` as the + concurrency substrate, wrapping each child in an Operation that adds + durable semantics (replay, Close events, coroutineId). +- **Consequences:** The durable combinators depend on Effection's internal + `trap()` behavior (accessed indirectly through `all()` and `race()`). + If `trap()` semantics change in a future Effection version, the error + boundary behavior may change. This is acceptable since `all()` and + `race()` are public API with well-defined error semantics. + +## DEC-021: durableRun accepts Operation, not just Workflow + +- **Phase:** 4 (Structured Concurrency) +- **Date:** 2026-02-28 +- **Context:** `durableRun` originally accepted `() => Workflow` to enforce + that only durable-safe effects are yielded. But `durableAll`/`durableRace` + return `Operation` (they yield infrastructure effects like `useScope`, + `spawn` internally). A workflow that uses combinators yields both + DurableEffect and Effect values, making it `Operation` not `Workflow`. +- **Decision:** Widen `durableRun`'s parameter to + `() => Workflow | Operation`. +- **Rationale:** Type safety is still enforced at the leaf level — `durableCall`, + `durableSleep`, etc. return `Workflow`. But the top-level workflow that + uses combinators naturally returns `Operation`. Requiring `Workflow` + at the top level would force users to cast or use `as any`, which is worse + than accepting the union. Existing `Workflow` code still works without + changes since `Workflow` is a subtype of `Operation`. +- **Consequences:** The type-level guarantee that only durable effects can be + yielded is no longer enforced at the `durableRun` boundary. It is enforced + at the combinator/operation level instead. This is a pragmatic tradeoff. + +## DEC-022: runDurableChild emits Close events in finally for cancellation + +- **Phase:** 4 (Structured Concurrency) +- **Date:** 2026-02-28 +- **Context:** When a child is cancelled (e.g., race loser, sibling of a + failed child), Effection calls `iterator.return()` which triggers + `finally` blocks but not `catch`. The protocol requires Close(cancelled) + events for cancelled coroutines. +- **Decision:** `runDurableChild` tracks whether it completed via ok/err + paths using a `closeEvent` variable. In the `finally` block, if + `closeEvent` is still undefined, the child was cancelled, and a + Close(cancelled) event is emitted. +- **Rationale:** This is the only way to detect cancellation in a generator + without modifying the Effection runtime. The pattern: set `closeEvent` in + try (ok) and catch (err), check for undefined in finally (cancelled). +- **Consequences:** Every child exit path (ok, err, cancelled) writes a + Close event. The `yield* call(() => stream.append(...))` in finally may + itself be interrupted during scope teardown — but this is acceptable for + the in-memory stream. A production stream adapter would need to handle + partial writes. + +## DEC-023: destroy() errors swallowed in durableRun finally block + +- **Phase:** 4 (Structured Concurrency) +- **Date:** 2026-02-28 +- **Context:** `durableRun` calls `await destroy()` in its finally block. + When the workflow fails (e.g., child error propagated up), `destroy()` + may throw "halted" because the scope is in an error state. In JavaScript, + if a `finally` block throws, it replaces the original error from the + catch block. +- **Decision:** Wrap `destroy()` in a try/catch and swallow the error. +- **Rationale:** The original workflow error is more informative than + "halted". Scope cleanup errors are expected when the workflow failed. + The scope's resources are cleaned up regardless. +- **Consequences:** Errors during scope destruction are silently swallowed. + This is acceptable because the scope's destruction is a best-effort + cleanup — the important state (the durable stream) has already been + written to by the catch block before finally runs. + +## DEC-024: Cancelled children replay via suspend(), not throw + +- **Phase:** 4 (Structured Concurrency) +- **Date:** 2026-02-28 +- **Context:** During replay of a `durableRace`, a loser child has + `Close(cancelled)` in the journal. The original implementation threw + a `CancelledError`, but this surfaced as an unexpected race error + rather than silently replaying. +- **Decision:** When `runDurableChild` encounters `Close(cancelled)` during + replay, it calls `yield* suspend()` instead of throwing. The child blocks + until the parent combinator (race) cancels it naturally via Effection's + structured concurrency teardown. +- **Rationale:** In the original live run, the loser was cancelled by + Effection calling `iterator.return()` — it never threw an error; it + simply stopped executing. `suspend()` reproduces this behavior exactly: + the child hangs until cancelled, matching the original execution path. + The `Close(cancelled)` event already exists in the journal, so the + finally block skips re-emitting it (checked via `replayIndex.hasClose()`). +- **Consequences:** Replay of race losers is invisible — they block and + get cancelled just like the original run. No duplicate Close events. + +## DEC-025: Test 27 — dynamic spawn count is not a divergence error + +- **Phase:** 4 (Structured Concurrency) +- **Date:** 2026-02-28 +- **Context:** The protocol specification (§14, test 27) says that replaying + `all([a, b])` with `all([a, b, c])` should produce `DivergenceError`. + However, in our implementation, this succeeds gracefully: children a and b + replay from the journal (their Close events exist), child c executes live + (no journal entries for root.2), and the post-join effects continue normally. +- **Decision:** Our test 27 asserts success, not divergence. The spec's + expected behavior is incorrect for our architecture. +- **Rationale:** Divergence detection operates at the Yield-event level: + when a durable effect (durableCall, durableSleep) is yielded, the replay + index checks if a matching Yield event exists for that coroutineId+cursor. + A new child (root.2) simply has no replay entries, so its effects execute + live — indistinguishable from a partial replay after a crash. There is no + structural check that says "the number of children in an all() must match + the journal." Such a check would be overly restrictive and would prevent + legitimate workflow evolution (adding new parallel branches). +- **Consequences:** Workflows can add new children to `durableAll` without + divergence errors. This is a deliberate relaxation of the spec. The spec + should be updated to reflect this (test 27 verifies graceful handling, + not DivergenceError). + +## DEC-026: Direct HTTP append with raw fetch, not IdempotentProducer + +- **Phase:** 5 (HttpDurableStream) +- **Date:** 2026-02-28 +- **Context:** The `@durable-streams/client` package provides an + `IdempotentProducer` class designed for throughput workloads (fire-and-forget + + background flush). Need to decide whether to use it or raw `fetch()`. +- **Options considered:** + 1. `IdempotentProducer` with `lingerMs=0` and await flush after every append + 2. Raw `fetch()` with manual producer headers +- **Decision:** Raw `fetch()` with manual `Producer-Id`, `Producer-Epoch`, + `Producer-Seq` headers on each POST. +- **Rationale:** Durable execution requires synchronous acknowledgment on + every write (persist-before-resume, spec §5). Setting `lingerMs=0` and + awaiting flush after every append makes the producer pure overhead — it + batches nothing and adds an abstraction layer. Raw fetch captures + `Stream-Next-Offset` from every response, which is needed for future + `tail()` calls. +- **Consequences:** More code in `HttpDurableStream.doAppend()` but full + control over request/response handling. Error types (`StaleEpochError`, + `SequenceGapError`) are still imported from the client package. + +## DEC-027: Promise chain serialization for concurrent appends + +- **Phase:** 5 (HttpDurableStream) +- **Date:** 2026-02-28 +- **Context:** When `durableAll` runs N children, their effects may resolve + in the same tick. Each child's `createDurableEffect.enter()` calls + `stream.append()` — producing concurrent promises. If two POSTs with + seq=5 and seq=6 arrive out of order (HTTP/2 multiplexing), the server + returns 409 (sequence gap). +- **Options considered:** + 1. Mutex/lock around append + 2. Promise chain serialization + 3. Accept 409 and retry with correct seq +- **Decision:** Promise chain serialization. Sequence numbers are assigned + synchronously (before any async work). HTTP calls are chained behind + `this.pending`: `const p = this.pending.then(() => this.doAppend(...)); + this.pending = p.catch(() => {});` +- **Rationale:** Each caller still awaits their own append promise. Ordering + matches seq assignment order. The `p.catch(() => {})` pattern prevents + failed appends from blocking future ones in the chain, but errors still + propagate to the original caller. This is simpler than a mutex and avoids + the complexity of retry logic. +- **Consequences:** Appends execute in strict sequence order. A fatal error + (e.g., StaleEpochError) sets `this.fatalError` so future appends fail-fast + without making HTTP calls. + +## DEC-028: Close event append in finally block is cancellable (best-effort) + +- **Phase:** 5 (HttpDurableStream) +- **Date:** 2026-02-28 +- **Context:** `runDurableChild` (in `combinators.ts`) appends Close + events in a `finally` block via `yield* call(() => stream.append(...))`. + When the parent scope is torn down (e.g., race winner cancels losers), + this async operation can be interrupted. +- **Decision:** Accept that Close event appends in finally blocks are + best-effort. Missing Close events just mean the child re-executes on + replay (idempotent). +- **Rationale:** Effection does not currently expose an uncancellable + context for finally blocks. The in-memory stream completes synchronously + so this is not an issue in tests. For the HTTP adapter, the serialized + POST may be interrupted mid-flight. The protocol handles this gracefully: + a missing Close event means the child has no replay short-circuit, so it + re-executes live on the next run. +- **Consequences:** In rare cases (parent cancellation racing with child + cleanup), a Close event may not be persisted. The workflow remains correct + because re-execution is idempotent. A future enhancement could use an + uncancellable context when Effection exposes one. + +## DEC-029: Track Stream-Next-Offset from every HTTP response + +- **Phase:** 5 (HttpDurableStream) +- **Date:** 2026-02-28 +- **Context:** The Durable Streams server returns a `Stream-Next-Offset` + header on every successful append (200) and on reads. This is an opaque + offset string (e.g., `0000000000000000_0000000000000118`) that represents + the position after the last written event. +- **Decision:** Store `lastOffset` from both reads (`res.offset` from the + client's `stream()` function) and writes (`Stream-Next-Offset` header + from raw fetch responses). +- **Rationale:** The offset is the resumption point for future `tail()` + calls (SSE/long-poll tailing, not yet implemented). Cheap to capture now, + annoying to retrofit later. The field is public (`lastOffset`) for + inspection in tests. +- **Consequences:** `lastOffset` is updated as a side effect of `readAll()` + and `append()`. Nothing consumes it yet, but it's available for the + tailing feature when implemented. + +## DEC-030: durableEach — pre-fetch pattern with context-based state sharing + +- **Phase:** 6 (Durable Iteration) +- **Date:** 2026-02-28 +- **Context:** Need a durable iteration primitive for consuming a + `DurableSource` (e.g., a message queue, paginated API) inside a + Workflow. Each item fetch must be journaled so iteration survives + crashes and replays from the journal. +- **Decision:** Implement `durableEach(name, source)` / `durableEach.next()` + mirroring Effection's `each()` / `each.next()` pattern. Key choices: + 1. **Pre-fetch pattern**: `durableEach()` fetches the first item, + returns a synchronous iterable. `durableEach.next()` fetches + subsequent items. This makes `for...of` work with durable effects. + 2. **Context-based state sharing**: `DurableEachContext` Effection + context stores `{ name, source, current, advanced }`, shared + between `durableEach()` and `durableEach.next()` via `useScope()`. + 3. **`{ value: T } | { done: true }` wrapper**: Stored in journal + to avoid null-as-done ambiguity (null is valid JSON). + 4. **Single fetch helper** (`durableEachFetch`): Both initial and + subsequent fetches use the same helper with description + `{ type: "each", name }`. Same journal format, same replay path. + 5. **Advance guard**: Runtime detection of missing + `yield* durableEach.next()` — iterator throws if re-entered + without advance. Prevents infinite loops on same item. + 6. **Source teardown**: Both effect-level teardown (in + `createDurableEffect`) and scope-level cleanup (via `ensure()`) + call `source.close?.()`. Dual cleanup is safe for idempotent + close functions. + 7. **Operation, not Workflow**: Both `durableEach` and + `durableEach.next()` use `useScope()` / `ensure()`, making them + `Operation` at the type level. Consistent with combinators + (`durableSpawn`, `durableAll`, etc.). +- **Rationale:** Mirrors Effection's `each()` API for developer + familiarity. The pre-fetch pattern is the only way to make `for...of` + work with async durable effects (synchronous iterator protocol + requires the value to be available when `next()` is called). The + `{ value: T } | { done: true }` wrapper prevents the null sentinel + problem documented in the integration spec §12.6. +- **Consequences:** Each iteration produces one Yield event in the + journal. Journal size grows linearly with items consumed. For + long-running streams, a future Continue-As-New feature (§15.2) or + cursor-based checkpointing will bound journal growth. Nested + `durableEach` calls require separate child scopes (inner clobbers + outer context). + +## DEC-031: Divergence API — pluggable policy via Effection's Api pattern + +- **Date:** 2026-03-01 +- **Context:** Divergence detection (description mismatch, continue-past-close) + was hard-coded in `createDurableEffect()` — every mismatch unconditionally + threw `DivergenceError` or `ContinuePastCloseDivergenceError`. Users had no + way to override this behavior (e.g., to switch a coroutine to live execution + when the workflow code has intentionally changed). +- **Decision:** Delegate divergence policy to a `Divergence` API object using + Effection's `Api` pattern with `scope.around()` middleware support. + - The API has one method: `decide(info: DivergenceInfo): DivergenceDecision` + - `DivergenceInfo` is a discriminated union on `kind`: `"description-mismatch"` | + `"continue-past-close"`, carrying context about the divergence + - `DivergenceDecision` has two variants: `{ type: "throw"; error: Error }` | + `{ type: "run-live" }` + - Default behavior is strict: all divergences return `{ type: "throw" }` + - Users override via `scope.around(Divergence, { decide: ([info], next) => ... })` + - Since `durableRun` is an Operation (DEC-032), middleware is installed on + the caller's scope before `yield*`-ing into `durableRun` + - `decide()` is synchronous because it's called from `Effect.enter()`, which + cannot yield. The middleware chain runs synchronously via `Divergence.invoke(scope, "decide", [info])` + - When `run-live` is decided, `replayIndex.disableReplay(coroutineId)` is called + and execution falls through to the live path via a labeled `replay:` block + - The early-return divergence check in `durableRun` is skipped when replay is + disabled for the root coroutine (the Divergence API already approved the change) +- **Implementation notes:** + - Initially (alpha.5) could not use `createApi()` from + `@effection/effection/experimental` due to a circular initialization bug + (`api-internal.ts` imported `useScope` from `scope.ts`, creating: + `api-internal → scope → scope-internal → api → api-internal`). The + workaround was a hand-rolled `Divergence` object using `createContext()` + and manual middleware dispatch mirroring `createApiInternal`'s logic. + - **Fixed in alpha.6/alpha.7:** Charles replaced `yield* useScope()` in + `api-internal.ts` with an inline `GetScope` Effect, breaking the cycle. + The `Divergence` object now uses `createApi()` from the `experimental` + entry point directly — eliminating ~100 lines of workaround code. + - `ReplayIndex` gained `disableReplay(id)`, `isReplayDisabled(id)`, and guards + in `peekYield()`, `hasClose()`, `isFullyReplayed()` to skip replay for disabled + coroutines + - `CoroutineView.scope` in `types.ts` uses Effection's full `Scope` type, + since `Divergence.invoke()` and `scope.expect()` require it +- **Rationale:** Following the same pattern Effection uses for its own built-in + APIs (Scope, Main) ensures composability. Middleware is scope-scoped, so + different workflow runs can have different divergence policies. The `setup` + callback on `durableRun` provides a clean injection point without requiring + callers to manage scopes directly. +- **Consequences:** Divergence handling is now a pluggable policy rather than a + hard-coded behavior. The `run-live` decision path enables future code evolution + scenarios (e.g., "patching" in Temporal's terminology). + +## DEC-032: Open EffectDescription replaces meta field for validation data + +- **Date:** 2026-03-04 +- **Context:** The ReplayGuard design (replay-guard-spec.md) originally proposed + an optional `meta?: Record` field on Yield events to carry + validation metadata (file paths, content hashes) for staleness detection. + Charles objected to adding meta to the stream. Analysis revealed that meta + was solving a problem that doesn't exist: file paths are effect *inputs* + and belong in the effect description; content hashes are effect *outputs* + and belong in result.value. +- **Decision:** Remove `meta` from the Yield event type entirely. Open + `EffectDescription` to allow extra fields beyond `type` and `name` via + an index signature `[key: string]: Json`. Divergence detection continues + to compare only `type` and `name` — extra fields are stored verbatim and + never checked. +- **Rationale:** Inputs belong with the description of what was requested. + Outputs belong with the result of what was produced. This is the natural + separation already established by the protocol. The ReplayGuard middleware + reads `event.description.path` for the file path and + `event.result.value.contentHash` for the recorded hash — no new protocol + fields needed. +- **Consequences:** The Yield event type loses the `meta` field. Effect + implementations that need staleness validation must return rich result + objects that include validation data (e.g., content hash alongside content). + The protocol remains a two-field `{ type, name }` identity check with + open-ended storage for additional context. + +## DEC-033: Operation-native HttpDurableStream via resource + Queue + worker + +- **Date:** 2026-03-04 +- **Context:** The DurableStream interface was made Operation-native (methods + return `Operation` instead of `Promise`) as part of the + operation-native-stream branch. HttpDurableStream still used Promise-based + methods with a Promise chain for serializing concurrent appends (DEC-027). + Simply wrapping the Promise chain with `yield* call()` would work but leaves + Promise-based serialization hidden inside an Operation-native interface — + not truly structured concurrency. +- **Options considered:** + 1. Wrap existing Promise chain with `yield* call()` — minimal change, but + the serialization is still Promise-based under the hood + 2. Channel + spawned worker inside an Effection resource — fully + Operation-native, clean cancellation, structured lifecycle + 3. Hybrid: Queue with deferred/signal per append +- **Decision:** Option 2. Replace the `HttpDurableStream` class with a + `useHttpDurableStream(opts)` resource function that returns + `Operation`. The resource: + 1. Creates the stream on the server (PUT) via `yield* call()` + 2. Creates a `Queue` for serializing appends + 3. Spawns a serial worker that pulls requests from the queue and + executes HTTP POSTs one at a time via `yield* call()` + 4. Uses `withResolvers()` per append so each caller waits for + their specific HTTP POST to complete + 5. Provides the `DurableStream` handle with `readAll()` and `append()` + as generator methods +- **Rationale:** The Queue + worker pattern is idiomatic Effection. The worker's + lifetime is bound to the resource scope — when the scope is torn down (e.g., + workflow finishes), the worker is cancelled and no HTTP requests are left + dangling. Sequence numbers are still assigned synchronously in `append()` + (before the `yield*`), preserving FIFO ordering. The fail-fast pattern is + preserved: `fatalError` is checked before enqueuing, and the worker also + checks it before each POST. `withResolvers()` bridges the worker's + completion back to the specific caller, so errors from a particular append + are propagated to the correct caller. +- **Key primitives used:** + - `resource()` — owns the worker scope and provides the stream handle + - `createQueue()` — FIFO buffer between concurrent callers and the serial worker + - `spawn()` — launches the worker inside the resource scope + - `withResolvers()` — per-append completion signaling + - `call()` — bridges async fetch/HTTP operations into Operations +- **Supersedes:** DEC-027's Promise chain serialization. The FIFO ordering + guarantee and fail-fast semantics are preserved, but the mechanism is now + fully Operation-native. +- **Consequences:** `HttpDurableStream.connect(opts)` is replaced by + `yield* useHttpDurableStream(opts)`. The stream must be created inside an + Effection scope. This is natural since it's always used with `durableRun` + which requires a scope. Tests and demos updated to wrap stream creation + inside `run()`. The `HttpDurableStreamHandle` interface extends + `DurableStream` with the `lastOffset` property for offset tracking. + +## DEC-034: ephemeral() — explicit escape hatch for non-durable Operations in Workflows + +- **Date:** 2026-03-04 +- **Context:** The combinators (`durableAll`, `durableRace`, `durableSpawn`) + accepted `() => Workflow | Operation` as children (per DEC-021). + The `| Operation` part was a type-level loophole — users could pass + bare Operations (containing `sleep()`, `fetch()`, etc.) as children + whose effects wouldn't be journaled, silently breaking replay correctness. + Charles (Effection author) identified that mixing Operations and Workflows + should be a compilation error, with an explicit adapter analogous to + Rust's `unsafe {}` as the only way to opt in. +- **Decision:** Introduce `ephemeral(operation: Operation): Workflow` + as the explicit escape hatch, and tighten combinator child signatures to + accept only `() => Workflow`. + - **`ephemeral()`** wraps a non-durable Operation in a `DurableEffect` that + is transparent to the journal: no Yield event written, no replay index + entry consumed. The Operation runs via `routine.scope.run()` with full + structured concurrency. On replay, the Operation simply re-runs. + - **Combinator signatures** changed from `() => Workflow | Operation` + to `() => Workflow` for `durableAll`, `durableRace`, `durableSpawn`, + and the internal `runDurableChild`. Each combinator self-wraps its + infrastructure effects (useScope, spawn, all, race) in `ephemeral()` + internally, so they return `Workflow` — users never need `ephemeral()` + for standard library combinator calls, including nested ones. + - **`durableRun`** still accepts `() => Workflow | Operation` because + it is the outermost entry point. The dangerous boundary is at the child + level inside combinators, not at `durableRun`'s entry point. + - **`durableEach`** wraps its infrastructure (ensure) in `ephemeral()` + internally and returns `Workflow>`. `durableEach.next()` + is a pure Workflow (no infrastructure effects) — it reads shared state + from a module-level variable rather than Effection context, avoiding + the scope isolation problem that arises when both functions are + individually wrapped in `ephemeral()` (each gets its own child scope, + making context invisible across them). +- **Rationale:** The primary risk is users passing bare non-durable Operations + as children to combinators. By tightening the child signature to + `Workflow`, TypeScript rejects `Operation` children at compile time. + The `ephemeral()` adapter makes the escape explicit and auditable — every + non-durable Operation that participates in a Workflow must go through it. + This is analogous to Rust's `unsafe {}` blocks: the boundary is visible in + the source code, making it easy to audit where durable guarantees are + intentionally relaxed. +- **Implementation:** `ephemeral()` creates a `DurableEffect` with + `description: "ephemeral"` whose `enter()` method runs the wrapped + Operation via `routine.scope.run()`. It never calls `checkReplay()`, + never appends to the stream, and never advances the replay cursor. + Cancellation flows through naturally via Effection's scope hierarchy. +- **Supersedes:** DEC-021's widening of combinator child signatures. + `durableRun`'s parameter type remains widened per DEC-021. +- **Consequences:** Since combinators self-wrap with `ephemeral()` internally, + nested combinator usage works naturally: + ```typescript + yield* durableAll([ + function* () { + const inner = yield* durableAll([...]); // no ephemeral() needed + return inner.join("+") as string; + }, + ]); + ``` + Users only need `ephemeral()` for their own non-durable Operations inside + Workflows — standard library combinators handle it transparently. + `durableEach` uses module-level state (safe due to single-threaded + execution) rather than Effection context to share state between + `durableEach()` and `durableEach.next()`. diff --git a/durable-streams/specs/durable-streams.md b/durable-streams/specs/durable-streams.md new file mode 100644 index 00000000..b53ba63a --- /dev/null +++ b/durable-streams/specs/durable-streams.md @@ -0,0 +1,254 @@ +# Durable Streams as a durable execution log backend + +**Durable Streams provides a strong but not perfect fit for backing a generator-based durable execution log.** The protocol's append-only semantics, monotonic opaque offsets, idempotent producer model with epoch-based zombie fencing, and "catch up then tail" read pattern align well with the core invariants of a Yield/Close event journal. Two critical gaps require application-layer mitigation: the protocol defines durability as a *logical contract* without prescribing fsync or replication, meaning "persist-before-resume" depends entirely on your server implementation; and there is no multi-event atomic append, so causal ordering of Close events relative to parent yields must be enforced by the client through sequencing discipline rather than transactional batches. The protocol emerged from 1.5 years of production use at ElectricSQL for Postgres sync and was publicly released in December 2025 (v0.2.0 shipped January 2026 with idempotent producers). + +--- + +## A. Protocol guarantees map cleanly to execution log requirements + +**Ordered append with monotonic offsets.** Every append to a Durable Stream returns a `Stream-Next-Offset` header containing an opaque, lexicographically sortable token. Offsets are server-generated, strictly monotonically increasing, and immutable by position — bytes at a given offset never change. The protocol mandates that servers serialize validation and append operations per `(stream, producerId)` pair, preventing out-of-order request arrival from corrupting sequence ordering. + +**Prefix-closure with no gaps.** The append-only model inherently provides prefix-closure: reads from any offset return a contiguous sequence of all bytes appended after that position. There is no mechanism to insert, delete, or reorder entries mid-stream. Combined with the durability contract — "once written and acknowledged, bytes persist until the stream is deleted or expired" — this means any acknowledged prefix is permanently sealed. + +**Concurrent readers see a consistent, ordered view.** Multiple readers can consume the same stream simultaneously from different offsets. All read modes (catch-up, long-poll, SSE) deliver data in identical order — the conformance test suite explicitly verifies "streaming equivalence" between SSE and long-poll output. Readers never observe partial appends; each append is atomic at the HTTP request level. + +**Failure modes to design around.** Partial writes during crash are the primary concern. The IMPLEMENTATION_TESTING.md document (drawn from 120 days of reliability hardening at ElectricSQL, fixing 200+ bugs) identifies these critical failure modes: incomplete chunk writes where the server crashes mid-append, partial disk flushes where in-memory state diverges from persisted state, and corrupted chunk files. Implementations should roll back to the last valid boundary on recovery. For the idempotent producer, non-atomic stores have a crash window between persisting the append and updating producer state — the spec explicitly acknowledges this and recommends epoch-bumping as recovery. Retries with duplicate `(producerId, epoch, seq)` tuples return **204 No Content** (idempotent success), making client retries safe within an epoch. + +**"Catch up then tail" is a first-class pattern.** A reader issues `GET ?offset=-1` (or a saved offset) to fetch all historical data, receives a `Stream-Up-To-Date: true` header when caught up, then transitions to `GET ?offset=X&live=long-poll` or `&live=sse` for real-time tailing. The server periodically closes SSE connections (~60 seconds) for CDN compatibility; clients reconnect with their last offset. No server-side session state exists — progress tracking is entirely client-side, which maps well to per-coroutine cursors. + +--- + +## B. Exactly-once semantics work within an epoch but require care across crashes + +**The idempotent producer uses a three-header system.** Every append includes `Producer-Id` (client-chosen identifier), `Producer-Epoch` (monotonically increasing integer), and `Producer-Seq` (per-batch sequence number). All three must be provided together or not at all. The server validates them against stored state per `(stream, producerId)` tuple: + +- **`seq == lastSeq + 1`** → Accept, return 200 OK +- **`seq <= lastSeq`** → Duplicate detected, return 204 No Content (idempotent success) +- **`seq > lastSeq + 1`** → Gap detected, return 409 Conflict with `Producer-Expected-Seq` header +- **`epoch < state.epoch`** → Zombie fenced, return 403 Forbidden with current epoch +- **`epoch > state.epoch` and `seq != 0`** → Invalid, return 400 Bad Request (new epochs must start at seq 0) + +This provides **exactly-once within an epoch**. The sequence number is per-batch (not per-message), and pipelined requests are supported — the server returns `Producer-Seq` in the response confirming the highest accepted sequence, enabling clients to recover pipeline state after reconnection. + +**Crash recovery trades exactly-once for at-least-once across epoch boundaries.** When a producer crashes, it increments its epoch and starts sequence at 0. The server treats this as a new session. If the server uses non-atomic storage (i.e., producer state and log data are not committed in a single transaction), there is a crash window where the append persisted but the sequence counter did not update. Bumping the epoch resolves this but may produce a duplicate of the last pre-crash append. For a durable execution log, this means the **replay engine must be idempotent with respect to a duplicated final Yield event at an epoch boundary**. The spec recommends that persistent stores commit producer state and log appends atomically (e.g., single database transaction) to eliminate this window entirely. + +**No multi-event atomic append exists in the protocol.** Each HTTP POST is one atomic unit. You cannot atomically append a Yield event and a Close event together. For the durable execution use case, this means causal ordering constraints (Close must follow all child yields) must be enforced by **sequencing discipline at the application layer** — append child completion first, await acknowledgment, then append the parent yield that depends on it. The `IdempotentProducer` does support batching via `lingerMs` (accumulating messages over a time window into a single POST), but a batch is a single sequence number — it's atomic at the batch level, not transactional across semantically distinct events. + +--- + +## C. Mapping durable execution invariants requires application-layer contracts + +**"Persist-before-resume" depends on your interpretation of the 200 acknowledgment.** The protocol's durability contract states that acknowledged bytes persist until deletion or expiry, but **does not prescribe fsync, replication factor, or WAL semantics**. The spec explicitly lists in-memory storage as valid. For durable execution, "persist-before-resume" means: append the Yield event, await the 200 response from a server whose storage backend provides crash-durable persistence (file-backed with fsync, database-backed, or the hosted Electric Cloud service), and only then call `generator.next(result)`. The `IdempotentProducer`'s fire-and-forget `append()` with `lingerMs` batching is **not safe for persist-before-resume** — use `producer.flush()` after each critical event to await acknowledgment, or use raw HTTP POSTs with explicit await. + +**Causal ordering requires a stream-per-coroutine or careful global sequencing strategy.** The protocol guarantees total order within a single stream. Two viable architectures exist: + +- **One stream per coroutine**: Each coroutine writes to its own stream. Per-coroutine cursors map directly to per-stream offsets. Causal ordering across coroutines (Close before parent yield) is enforced by the client: append child Close, await ack, then append parent Yield. This provides natural isolation but creates many streams. +- **Single stream with framed events**: All coroutines append to one stream using JSON mode (which preserves message boundaries). Each event includes a `coroutineId` field. The single-stream total order automatically captures causal relationships *if* the client sequences appends correctly. Per-coroutine replay uses a filtered cursor over the shared stream. + +For structured concurrency, the **single-stream model** is likely superior because it naturally captures cross-coroutine causal order in the global offset sequence. The ordering constraint for Close events — that a child's Close must appear in the stream before the parent's Yield that consumes the child's result — is enforced by the runtime: complete the child, append its Close event, await ack, then resume the parent and append its Yield. + +**Client-side buffering strategy must prioritize correctness over throughput.** The recommended approach for durable execution is **synchronous-append mode**: each effect resolution triggers an immediate append with an awaited acknowledgment before the generator resumes. The `lingerMs` batching optimization is only safe for events where ordering is already guaranteed by the client's sequential execution — for example, multiple Yield events within a single coroutine can be batched if the generator will not be resumed until the batch is acknowledged. The `autoClaim: true` option on `IdempotentProducer` provides automatic epoch recovery on restart, suitable for a durable execution scheduler that restarts after a crash. + +--- + +## D. Operational profile suits local dev; production requires hosted or custom server + +**Deployment options span development to production.** The reference Node.js server (`@durable-streams/server`) supports in-memory and file-backed storage for development. A Caddy-based binary ships for macOS, Linux, and Windows, suitable for local dev and light production. The hosted Electric Cloud service (launched January 2026) provides **240K writes/second** for small messages, 15–25 MB/sec sustained throughput, and tested support for 1M concurrent connections per stream. For durable execution, the hosted service or a custom server with a database-backed store (PostgreSQL, SQLite) and atomic producer state commits is the minimum viable production deployment. + +**Observability is minimal at this stage.** The project is early (self-described: "Docs are sparse, guides are coming"). Available hooks include the `onError` callback on `IdempotentProducer`, `Stream-Next-Offset` and `Producer-Seq` response headers for tracking progress, a test UI package (`@durable-streams/test-ui`) for visual stream inspection, and the CLI tool. No Prometheus metrics, OpenTelemetry integration, structured logging, or production dashboards exist yet. For a durable execution system, you'll need to build observability around the HTTP response headers — tracking append latency (time from POST to 200), consumer lag (difference between tail offset and reader offset), and epoch transitions (indicating producer restarts or fencing events). + +**Security delegates to HTTP infrastructure.** The base protocol mandates HTTPS in production but has no built-in authentication. The hosted version uses Bearer token auth. Self-hosted deployments should run behind an API gateway or reverse proxy handling auth. **Stream naming is URL-path-based** — tenant isolation requires either path-prefix scoping (e.g., `/tenant-A/coroutine-123`) or separate server instances. The protocol notes that "sequence numbers are scoped per authenticated writer identity (or per stream, depending on implementation)" — this means auth identity can enforce single-writer semantics if the server implementation supports it. + +--- + +## Durable Streams ↔ durable execution mapping table + +| Execution requirement | Durable Streams feature | Gaps | Mitigation | +|---|---|---|---| +| **Append-only log** | Core model — streams are append-only, immutable by position | None | Direct fit | +| **Monotonic offsets** | Opaque, lexicographically sortable, server-generated offsets | Offsets are opaque (can't derive sequence numbers) | Store logical sequence in event payload; use offset only for resumption | +| **Prefix-closure / no gaps** | Inherent from append-only + acknowledged durability | None | Direct fit | +| **Single writer** | Producer-Id + Epoch fencing (403 on stale epoch) | Protocol allows multiple Producer-Ids per stream | Use one Producer-Id per stream; rely on epoch fencing for failover | +| **Single source of truth** | "Once written and acknowledged, bytes persist until deleted" | Durability is implementation-dependent (no fsync mandate) | Choose a server with crash-durable storage; verify with IMPLEMENTATION_TESTING suite | +| **Persist-before-resume** | 200 response = server considers data durable | No guarantee of fsync/replication at protocol level; fire-and-forget API not safe | Await 200 on every critical append; disable lingerMs batching for Yield/Close events; verify server's durability semantics | +| **Exactly-once Yield recording** | IdempotentProducer deduplicates via (Id, Epoch, Seq) | At-least-once across epoch boundaries for non-atomic stores | Use atomic store; or make replay idempotent w.r.t. duplicate final event at epoch boundary | +| **Causal Close ordering** | Total order within a stream guaranteed | No multi-event atomic append; no cross-stream ordering | Enforce client-side: append child Close → await ack → append parent Yield. Single-stream model preferred | +| **Per-coroutine cursor** | Client-side offset tracking with Stream-Next-Offset | Protocol has no built-in per-entity cursor within a stream | Application layer: store `{coroutineId → lastOffset}` map; filter events during replay | +| **Catch up then tail (replay)** | GET with offset → catch-up; live=long-poll/sse → tail; Stream-Up-To-Date header signals transition | No server-side filtered subscription (e.g., by coroutineId) | Client-side filtering during replay; or use per-coroutine streams | +| **Crash recovery without duplicates** | Epoch bump + autoClaim; seq restart at 0 | One potential duplicate at epoch boundary | Replay engine must tolerate/dedup one duplicate Yield at recovery point | +| **Batch append (multiple Yields)** | lingerMs-based batching in IdempotentProducer | Batch is one sequence number — partial batch failure = full retry | Acceptable for non-critical batching; use synchronous append for Close events | +| **Retention / compaction** | Server MAY implement TTL-based retention | No log compaction (Kafka-style); no snapshot support | Build snapshot/compaction at application layer; use Continue-As-New pattern to bound log size | +| **Auth / tenant isolation** | HTTPS + delegated auth; Bearer tokens on hosted | No built-in auth in protocol | API gateway or reverse proxy; path-based tenant scoping | +| **Observability** | Response headers, onError callback, test UI, conformance tests | No metrics, tracing, or structured logging | Build custom observability layer around HTTP headers and error callbacks | + +--- + +## Recommended integration strategy + +### Producer/session model + +Use **one `IdempotentProducer` per execution run** (i.e., per top-level workflow invocation). Set `Producer-Id` to the workflow execution ID and persist the current epoch in the execution's metadata store. On scheduler restart, load the last known epoch, increment it, and create a new `IdempotentProducer` with `autoClaim: true`. This ensures zombie fencing if a previous scheduler instance is still alive. + +```typescript +const producer = new IdempotentProducer(stream, executionId, { + autoClaim: true, + lingerMs: 0, // Disable batching for durable execution — every append is synchronous + onError: (err) => { + if (err instanceof StaleEpochError) { + // Another scheduler took over this execution — halt gracefully + haltExecution(executionId) + } + }, +}) +``` + +### Append API usage pattern + +For **persist-before-resume correctness**, do not use the fire-and-forget `append()` path. Instead, use explicit flush after each critical event: + +```typescript +// After resolving an effect for a coroutine: +function* runWithDurability(operation, producer) { + const result = yield* executeEffect(operation) + + // 1. Append Yield event + producer.append(JSON.stringify({ + type: "Yield", + coroutineId, + seq: localSeq++, + effect: describeEffect(operation), + result: serializeResult(result), + })) + + // 2. Await durable persistence BEFORE resuming the generator + await producer.flush() + + // 3. Now safe to resume + return result +} +``` + +For **Close events**, the ordering discipline is: + +```typescript +// Child coroutine terminates +producer.append(JSON.stringify({ + type: "Close", + coroutineId: childId, + state: "ok", // or "err" or "cancelled" + value: serializeResult(childResult), +})) +await producer.flush() // Child Close is durable + +// NOW parent can resume and record its Yield that depends on child +producer.append(JSON.stringify({ + type: "Yield", + coroutineId: parentId, + seq: parentSeq++, + effect: { type: "awaitChild", childId }, + result: serializeResult(childResult), +})) +await producer.flush() // Parent Yield is durable +``` + +### Read/catch-up/tail pattern for replay + +Replay reads the full stream from offset `-1` and filters by coroutine ID: + +```typescript +async function replayCoroutine(stream, coroutineId) { + const events = [] + let offset = "-1" + + // Catch-up read — fetch all historical events + while (true) { + const response = await stream.read({ offset, live: false }) + for (const event of response.messages) { + if (event.coroutineId === coroutineId) { + events.push(event) + } + } + offset = response.nextOffset + if (response.upToDate) break + } + + // Replay: feed stored results back into the generator + const gen = createGenerator(coroutineId) + for (const event of events) { + if (event.type === "Yield") { + const yielded = gen.next(deserializeResult(event.result)) + // Verify determinism: yielded.value should match event.effect + assertDeterministic(yielded.value, event.effect) + } else if (event.type === "Close") { + // Coroutine terminated — restore terminal state + return restoreTerminalState(event) + } + } + + // Past end of log — switch to live execution + return { generator: gen, producer, tailOffset: offset } +} +``` + +For **live tailing** (watching for new events during concurrent execution), transition to long-poll after catch-up: + +```typescript +// After catch-up completes, tail for new events +const tailStream = stream.read({ offset: tailOffset, live: "long-poll" }) +for await (const chunk of tailStream) { + processNewEvents(chunk.messages) +} +``` + +### Error handling and retry policy + +- **Transient errors (500, 503, 429)**: Client library retries automatically with Retry-After respect. No application-level handling needed. +- **StaleEpochError (403)**: Another scheduler claimed this execution. Halt the local execution immediately — do not attempt further appends. +- **Sequence gap (409)**: Indicates a client-side bug (skipped a sequence number). Log the error with `Producer-Expected-Seq` and `Producer-Received-Seq` for diagnosis. This should never happen in correct code. +- **Disconnect during append**: Retry with the same `(Id, Epoch, Seq)` tuple. Server returns 204 if the original append succeeded (safe dedup) or 200 if it didn't (first write). +- **Disconnect during read**: Resume from last persisted offset. No data loss possible. + +--- + +## Must-have test checklist for Durable Streams integration + +### Crash during append + +- [ ] **Server crash mid-append**: Append a Yield event, kill the server process before 200 response reaches client. Restart server. Verify: (a) event either fully persisted or fully absent (no partial writes visible to readers), (b) retrying the same `(Id, Epoch, Seq)` returns 200 or 204 correctly. +- [ ] **Client crash after append, before generator resume**: Append succeeds (200 received), client crashes before calling `generator.next()`. On restart, replay from log. Verify: the event appears exactly once in the stream and the generator replays correctly past it. +- [ ] **Client crash during flush**: `producer.flush()` initiated but process dies. On restart with epoch bump, verify: at most one duplicate of the last event exists. Replay engine correctly handles this duplicate. + +### Retry with duplicate prevention + +- [ ] **Retry same sequence number**: Send append with `(Id, epoch=1, seq=5)`, receive 200. Send identical request again. Verify: 204 No Content returned (idempotent success), stream contains the event exactly once. +- [ ] **Retry after network timeout**: Send append, simulate network timeout (no response received). Retry with same `(Id, Epoch, Seq)`. Verify: correct response regardless of whether original append reached the server. +- [ ] **Epoch bump dedup boundary**: Append with `(Id, epoch=1, seq=5)`, simulate ambiguous failure. Restart with `epoch=2, seq=0`. Verify: if the epoch-1 append succeeded, the stream has at most one duplicate; if it failed, the event is absent. +- [ ] **Zombie fencing**: Two producers with same Id, one at epoch 1, one at epoch 2. Verify: epoch-1 producer receives 403 Forbidden on next append; epoch-2 producer operates normally. + +### Reconnect while tailing + +- [ ] **SSE disconnect and resume**: Start SSE tail at offset X. Server closes connection (per 60-second recommendation). Client reconnects at last offset. Verify: no events missed, no duplicates in the received event stream. +- [ ] **Long-poll timeout and resume**: Long-poll returns 200 with empty body (at tail). New event appended. Next long-poll returns the new event. Verify: no gap between catch-up and live. +- [ ] **Network partition during tail**: Simulate extended network outage (>60 seconds). Reconnect. Verify: all events appended during outage are received on catch-up, with correct offset continuity. +- [ ] **Multi-reader consistency**: Two readers tailing the same stream from different offsets. Verify: both see identical event order; neither sees events the other doesn't (eventually). + +### Partial replay from offsets + +- [ ] **Mid-stream replay**: Persist offset after processing event N. Restart and read from that offset. Verify: event N+1 is the first event received (byte-exact resumption, no skip, no repeat of event N). +- [ ] **Per-coroutine filtered replay**: Stream contains interleaved events from 3 coroutines. Replay coroutine B from its logical position 2. Verify: only coroutine B's events 2..N are fed to the generator, in order. +- [ ] **Replay determinism check**: Replay a coroutine through its full log. At each Yield, verify the generator yields the same effect description as recorded. Any mismatch → non-determinism error. +- [ ] **Replay then live transition**: Replay all historical events, verify `Stream-Up-To-Date: true` signals transition, then switch to live tail. Append a new event. Verify: it appears in the live tail without re-reading historical events. + +### Close ordering enforcement + +- [ ] **Child Close before parent Yield**: Spawn a child coroutine. Child completes. Verify: Close event for child appears at a lower offset than the parent's Yield event that consumes the child's result. +- [ ] **Sibling cancellation ordering**: Parent spawns children A and B. A errors. Verify: stream contains A's Close(err), then B's Close(cancelled), then parent's Close(err) — in that causal order. +- [ ] **Nested cleanup event ordering**: Child has a `finally` block that performs an effect. Parent halts child. Verify: child's Close event precedes any finally-block Yield events, and all of these precede the parent's next event. +- [ ] **Concurrent children convergence**: Parent spawns N children. All complete. Verify: all N Close events appear before the parent's Yield that joins them, regardless of completion order among siblings. + +### Protocol edge cases + +- [ ] **Empty stream read**: Read from offset `-1` on an empty stream. Verify: 200 with empty body (or empty JSON array), `Stream-Up-To-Date: true`, valid `Stream-Next-Offset`. +- [ ] **Maximum event size**: Append an event at the server's payload limit. Verify: accepted. Append one byte over the limit. Verify: 413 Payload Too Large. +- [ ] **Stream creation idempotency**: PUT to create a stream twice with the same content type. Verify: second PUT either succeeds idempotently or returns appropriate error; stream data is not corrupted. +- [ ] **Offset opacity**: Never construct, parse, or compare offsets except via lexicographic string comparison. Test that stored offsets from a previous server version still work after server upgrade. \ No newline at end of file diff --git a/durable-streams/specs/effection-integration.md b/durable-streams/specs/effection-integration.md new file mode 100644 index 00000000..06b5fa0d --- /dev/null +++ b/durable-streams/specs/effection-integration.md @@ -0,0 +1,1872 @@ +# Durable Execution for Effection: Architecture Research + +**Status:** Validated through implementation — Tier 1-4 tests passing +**Audience:** Charles, Taras +**Inputs:** Two-event durable execution spec (v2), Effection source, AGENTS.md, Charles's type-constraint feedback, implementation + DECISIONS.md + +--- + +## 1. Executive summary + +This document maps the two-event durable execution protocol onto Effection's +runtime architecture and incorporates Charles's insight that **type-level +constraints** should replace runtime effect classification. The design has +been **validated through implementation** — all four tiers of tests pass: +core replay (Tier 1), divergence detection (Tier 2), structured concurrency +(Tier 3), and deterministic identity (Tier 4). Key conclusions: + +- Effection's reducer does not need to change. +- Durability is implemented entirely within a new `DurableEffect` type whose + `enter()` method handles replay, divergence detection, and persist-before-resume. +- A `Workflow` type constrains generators at compile time so that only + durable-safe effects can be yielded. All `Workflow`s are `Operation`s, but + not all `Operation`s are `Workflow`s. No casts needed at the boundary (DEC-015). +- Structured concurrency combinators (`durableSpawn`, `durableAll`, + `durableRace`) return `Workflow` — they self-wrap their infrastructure + effects in `ephemeral()` and delegate to Effection's native `spawn()`, + `all()`, `race()`. Child signatures are tightened to `() => Workflow`; + bare Operations are rejected at compile time. A shared `runDurableChild` + helper handles DurableContext setup, Close events, and the `suspend()` + trick for replaying cancelled children. +- An `ephemeral(operation: Operation): Workflow` adapter provides + an explicit escape hatch (analogous to Rust's `unsafe {}`) for running + non-durable Operations inside Workflows. It is transparent to the journal + and re-runs on replay. See DEC-034. +- The Durable Streams protocol provides a strong backend fit (see companion + document `durable-streams.md`). + +**Implementation artifacts:** The `@effectionx/durable-streams` package +contains 12 modules (types, replay-index, effect, operations, combinators, +each, ephemeral, run, context, stream, http-stream, serialize) plus `mod.ts` +as the public API barrel. Test files (`*.test.ts`) cover types, replay-index, +durable-run (Tier 1), divergence (Tier 2), structured-concurrency (Tier 3), +deterministic-id (Tier 4), durable-each, ephemeral, and http-stream (backend +adapter). 34 architectural decisions recorded in `DECISIONS.md`. + +--- + +## 2. The protocol (fixed contract) + +The spec defines exactly two event types in an append-only stream: + +```typescript +type DurableEvent = Yield | Close; + +interface Yield { + type: "yield"; + coroutineId: CoroutineId; // e.g. "root.0.1" + description: EffectDescription; // { type, name } + result: Result; +} + +interface Close { + type: "close"; + coroutineId: CoroutineId; + result: Result; // ok | err | cancelled +} + +interface EffectDescription { + type: string; // "call", "sleep", "action", etc. + name: string; // "fetchOrder", "sleep", etc. + [key: string]: Json; // extra fields stored verbatim, never compared +} + +type Result = + | { status: "ok"; value?: Json } + | { status: "err"; error: SerializedError } + | { status: "cancelled" }; +``` + +**Yield** is written after an effect resolves. It records what was requested +(description) and what happened (result). During replay the description is +validated and the result is fed directly to the generator. + +**Close** is written when a coroutine terminates (completed, failed, or +cancelled). Close events are load-bearing for partial replay — they tell the +runtime which scopes completed before a crash and which need re-execution. + +### 2.1 Core invariants from the spec + +| # | Name | Rule | +|---|------|------| +| 1 | Deterministic Identity | Coroutine IDs are stable across runs for same code + same resolutions | +| 2 | Transparency | Generator cannot detect replay vs. live | +| 3 | Fork-Join Across Crash | Join result is independent of which children were replayed | +| 4 | **Persist-Before-Resume** | Durable write MUST complete before `iterator.next()` is called | +| 5 | Divergence Detection | Every replayed effect is validated against journal | +| 6 | Lifetime Containment | child ⊆ parent | +| 7 | Single Parent | Tree, not DAG | +| 8 | Implicit Join | Scope waits for all children | +| 9 | Cancellation Replay Fidelity | Cleanup path matches recorded path | +| 10 | Causal Ordering | Stream order respects causality | +| 11–13 | Stream consistency | Append-only, prefix-closed, monotonic indexing | + +--- + +## 3. How Effection's runtime works (relevant internals) + +### 3.1 The reducer loop + +`lib/reducer.ts` — the synchronous, re-entrant loop that drives all execution: + +```typescript +class Reducer { + reducing = false; + readonly queue = new InstructionQueue(); // min-heap priority queue, shallower scopes first + + reduce = (instruction: Instruction) => { + this.queue.enqueue(instruction); + if (this.reducing) return; // re-entrancy guard + try { + this.reducing = true; + let item = this.queue.dequeue(); + while (item) { + let [, routine, result, _, method] = item; + let iterator = routine.data.iterator; + // Call iterator.next(value), iterator.throw(error), or iterator.return(value) + let next = iterator[method](result.value); + if (!next.done) { + let action = next.value; // the yielded Effect + routine.data.exit = action.enter(routine.next, routine); + } + item = this.queue.dequeue(); + } + } finally { + this.reducing = false; + } + }; +} +``` + +Key properties: +- **Synchronous.** No `await` anywhere. Async effects resolve by calling + `routine.next(result)` from a callback, which re-enters `reduce()`. +- **Re-entrant safe.** If `reduce()` is already running, the instruction is + enqueued and the outer loop picks it up. +- **Priority ordered.** Shallower (parent) scopes run first (FIFO within a tier). + This is structural, not timing-dependent — it's deterministic. + +### 3.2 The Effect interface + +```typescript +interface Effect { + description: string; + enter( + resolve: Resolve>, + routine: Coroutine, + ): (resolve: Resolve>) => void; +} +``` + +Every yielded value from a generator is an `Effect`. The reducer calls +`enter()`, which: +1. Starts the actual work (sets timers, makes requests, etc.) +2. Calls `resolve(result)` when done — this enqueues the next instruction +3. Returns a teardown function called during cancellation/scope exit + +### 3.3 How existing effects use enter() + +**Synchronous / infrastructure effects** call `resolve()` immediately inside +`enter()`: + +```typescript +// useScope() — lib/context.ts +function UseScope(fn: (scope: Scope) => T, description: string): Effect { + return { + description, + enter: (resolve, { scope }) => { + resolve(Ok(fn(scope))); // resolve immediately + return (resolve) => resolve(Ok()); + }, + }; +} +``` + +**Asynchronous / user-facing effects** call `resolve()` later from a callback: + +```typescript +// sleep() — lib/sleep.ts, via action() +function sleep(duration: number): Operation { + return action((resolve) => { + let timeoutId = setTimeout(resolve, duration); // resolve later + return () => clearTimeout(timeoutId); + }); +} +``` + +### 3.4 The type system today + +```typescript +interface Operation { + [Symbol.iterator](): Iterator, T, unknown>; +} +``` + +An `Operation` is anything whose iterator yields `Effect` values. Generator +functions (`function*`) that only do `yield*` to other operations satisfy this. +The `yield*` delegation means the inner generator's yielded `Effect` values +pass through to the outer generator — the reducer sees a flat sequence of +effects regardless of call depth. + +### 3.5 Scope, Context, and Api systems + +- **Scope** (`lib/scope-internal.ts`): tree-structured, owns lifetime and + context. Created via `createScopeInternal(parent)`. Tracks children via + `Children` context. Destruction runs `ensure()` callbacks in reverse order. + +- **Context** (`lib/context.ts`): scope-local key-value storage. Children + inherit from parents. `createContext(name, default?)` creates a typed + context. Accessed via `scope.get()`, `scope.set()`, `scope.expect()`. + +- **Api** (`lib/api.ts`): middleware system for scope-bound operations. + `scope.around(api, middlewares, { at: "min" | "max" })` installs + middleware at different priority layers. Used for `Scope.create`, + `Scope.destroy`, and `Main.main`. + +### 3.6 Task lifecycle + +`createTask()` in `lib/task.ts`: + +1. Creates a child scope via `createScopeInternal(owner)` +2. Creates a `Future` for the task's result +3. Creates a `Delimiter` (error/cancellation boundary) +4. Registers an `ensure()` on the scope that: + - Closes the delimiter + - Resolves or rejects the future + - Propagates errors to the parent boundary +5. Creates a coroutine and returns a `start()` function + +The ensure callback provides a natural lifecycle hook, but the durable +execution implementation uses try/catch/finally in `runDurableChild` +instead (see §8.1). This avoids introducing infrastructure effects into +the child's generator — try/finally is just JavaScript. + +--- + +## 4. Charles's type-constraint architecture (validated) + +The following design was proposed by Charles and has been validated +through implementation. Key confirmation: DEC-009 (`Workflow` = +`Generator, T, unknown>` enforces yield +constraints at compile time) and DEC-015 (`Workflow` is directly +assignable to `Operation` — no casts needed). + +### 4.1 The problem with runtime classification + +The spec's §12 distinguishes "user-facing" from "infrastructure" effects. My +initial analysis proposed classifying them at runtime — e.g., effects that +call `resolve()` synchronously inside `enter()` are infrastructure. + +Charles correctly identified this as fragile. The failure mode is silent: a +misclassified effect gets skipped during replay with no error. Worse, the +classification is implicit — there's no way to verify it statically, and new +effects could be misclassified without anyone noticing. + +### 4.2 The solution: constrain the yield type + +Instead of classifying effects after they're yielded, constrain what can be +yielded at the type level: + +```typescript +interface DurableEffect { + description: string; + effectDescription: EffectDescription; // { type, name } + enter( + resolve: Resolve>, + routine: CoroutineView, + ): (resolve: Resolve>) => void; +} + +type Workflow = Generator, T, unknown>; +``` + +`DurableEffect` is structurally compatible with `Effect` (same shape plus +the extra `effectDescription` field). `Workflow` uses `Generator` (not `Iterable`) +so TypeScript enforces the yield-type constraint at compile time — yielding a +plain `Effect` inside a `Workflow` generator is a type error. + +Key relationships: +- `DurableEffect` is structurally compatible with `Effect` → assignable to `Effect` +- `Workflow` yields `DurableEffect` → every `Workflow` is an `Operation` +- `Operation` yields `Effect` → `Operation` is NOT a `Workflow` +- The reducer processes both identically (it just calls `enter()`) + +### 4.3 What this buys us + +**Compile-time safety.** If you declare `function*(): Workflow`, the +TypeScript compiler rejects `yield*` to any `Operation` that isn't also a +`Workflow`. Using `useAbortSignal()`, `sleep()`, or `each(stream)` inside a +workflow is a type error. + +```typescript +function* badWorkflow(): Workflow { + yield* useAbortSignal(); // TypeError! + yield* sleep(1000); // TypeError! +} +``` + +**No runtime classification.** The reducer doesn't need to know whether an +effect is durable. The `DurableEffect.enter()` method handles its own replay +and persistence logic. The reducer just calls `enter()` as always. + +**No reducer changes.** The existing `Reducer.reduce()` loop is untouched. +It processes `DurableEffect` values the same way it processes any `Effect` — +by calling `enter()`, getting a teardown function, and waiting for `resolve()`. + +**Clear boundary.** Workflow authors know exactly what they can and can't use. +If it compiles, it's durable. There's no hidden gotcha where "this operation +looks safe but actually breaks replay." + +**Uniform Workflow typing.** Combinators (`durableAll`, `durableRace`, +`durableSpawn`) self-wrap their infrastructure effects in `ephemeral()` and +return `Workflow`, so top-level workflows that use combinators can also be +typed as `Workflow`. Child signatures are tightened to `() => Workflow` +— bare Operations are rejected at compile time. Users who intentionally need +non-durable Operations inside a Workflow use `ephemeral()` as an explicit +escape hatch (analogous to Rust's `unsafe {}`). See DEC-034. + +### 4.4 The freeing quality + +Charles's observation: workflow authors don't need to understand whether they're +running durably. The type system enforces it. You write workflows using +workflow-enabled effects, and the compiler guarantees the result is safe for +durable execution. There's no "am I in replay mode?" question — the +`DurableEffect.enter()` handles that transparently. + +--- + +## 5. DurableEffect implementation (validated) + +The following design is implemented in `effect.ts` and tested +across Tier 1-2 (14 tests passing). + +### 5.1 The enter() method does everything + +The central insight: each `DurableEffect` handles its own replay/live dispatch +inside `enter()`. It reads the durable execution context from the scope, +checks the replay index, and either feeds the stored result or executes live +with persistence. + +```typescript +interface DurableContext { + replayIndex: ReplayIndex; + stream: DurableStream; + coroutineId: CoroutineId; + childCounter: number; +} + +const DurableCtx = createContext("@effection/durable"); + +type Executor = ( + resolve: (result: Result) => void, + reject: (error: Error) => void, +) => () => void; + +function createDurableEffect( + desc: EffectDescription, + execute: Executor, +): DurableEffect { + return { + description: `${desc.type}(${desc.name})`, + effectDescription: desc, + enter(resolve, routine) { + const ctx = routine.scope.expect(DurableCtx); + const entry = ctx.replayIndex.peekYield(ctx.coroutineId); + + if (entry) { + // ── REPLAY PATH ── + // §6.2: Validate description match. + // Only `type` and `name` are compared — extra fields on + // EffectDescription are intentionally not compared. + if (entry.description.type !== desc.type || + entry.description.name !== desc.name) { + const cursor = ctx.replayIndex.getCursor(ctx.coroutineId); + resolve({ + ok: false, + error: new DivergenceError( + ctx.coroutineId, cursor, entry.description, desc + ), + }); + return (exit) => exit(VOID_OK); + } + + ctx.replayIndex.consumeYield(ctx.coroutineId); + + // Feed stored result synchronously — no I/O, no side effects. + // Convert from protocol Result to Effection Result. + resolve(protocolToEffection(entry.result)); + return (exit) => exit(VOID_OK); + } + + // No replay entry. Check for continue-past-close divergence (§6.3). + if (ctx.replayIndex.hasClose(ctx.coroutineId)) { + resolve({ + ok: false, + error: new ContinuePastCloseDivergenceError( + ctx.coroutineId, + ctx.replayIndex.yieldCount(ctx.coroutineId), + ), + }); + return (exit) => exit(VOID_OK); + } + + // ── LIVE PATH ── + + function persistAndResolve(result: Result): void { + const event: Yield = { + type: "yield", + coroutineId: ctx.coroutineId, + description: desc, + result, + }; + // Strategy B: buffered write with deferred resume. + ctx.stream.append(event).then( + () => resolve(protocolToEffection(result)), + (err) => resolve({ + ok: false, + error: err instanceof Error ? err : new Error(String(err)), + }), + ); + } + + // Guard against synchronous throws from the executor + let teardown: () => void; + try { + teardown = execute( + (result) => persistAndResolve(result), + (error) => { + persistAndResolve({ + status: "err", + error: serializeError(error), + }); + }, + ); + } catch (e) { + const error = e instanceof Error ? e : new Error(String(e)); + persistAndResolve({ status: "err", error: serializeError(error) }); + return (exit) => exit(VOID_OK); + } + + return (exit) => { + try { teardown(); exit(VOID_OK); } + catch (e) { exit({ ok: false, error: e as Error }); } + }; + }, + }; +} +``` + +### 5.2 How this satisfies spec invariants (verified by tests) + +**Persist-before-resume (§5, hard invariant).** During live execution, +`resolve()` is called inside the `.then()` callback of the stream append. +The generator does not advance until the durable write completes. This is +the spec's "Strategy B: buffered write with deferred resume." If the +append rejects, the error is delivered through Effection's normal error +channel to avoid hanging the generator. + +**Transparency (§4.3).** During replay, `resolve()` is called synchronously +with the stored result (converted via `protocolToEffection()`). The reducer +processes it in the same tick. The generator receives the same value via +the same `iterator.next()` call path. It cannot distinguish replay from live. + +**Divergence detection (§6).** The description comparison happens inside +`enter()` before the stored result is fed. A mismatch raises +`DivergenceError` through the normal error propagation path (via +`resolve({ ok: false, error: ... })`). Additionally, the continue-past-close +check detects when the journal has a Close for this coroutine but the +generator yields additional effects beyond what was recorded. + +**Result conversion.** The protocol uses `{ status: "ok" | "err" | "cancelled" }` +while Effection uses `{ ok: true, value } | { ok: false, error }`. The +`protocolToEffection()` helper bridges this gap in both replay and live paths. + +**Synchronous throw guard.** If the executor throws synchronously (before +returning a teardown function), the error is caught, serialized, and +persisted through the normal `persistAndResolve` path. + +**No reducer changes.** The reducer calls `enter()`, gets a teardown, +waits for `resolve()`. Whether `resolve()` fires synchronously (replay) +or asynchronously (live with persistence) is invisible to the reducer. + +### 5.3 Replay path performance + +During replay, `enter()` is fully synchronous: +1. Read replay index — in-memory map lookup +2. Compare descriptions — two string comparisons +3. Call `resolve()` — enqueues the next reducer instruction +4. Return teardown (no-op) + +The reducer's re-entrancy guard means the enqueued instruction is processed +on the next iteration of the existing `while` loop. There is zero async +overhead during replay. A fully-replayed workflow completes in a single +synchronous reduce cycle. + +--- + +## 6. Workflow-enabled effects (validated) + +Implemented in `operations.ts`. All four effects (`durableSleep`, +`durableCall`, `durableAction`, `versionCheck`) are tested through the +Tier 1-2 test suites. + +### 6.1 durableSleep + +```typescript +function* durableSleep(ms: number): Workflow { + yield createDurableEffect( + { type: "sleep", name: "sleep" }, + (resolve) => { + const id = setTimeout(() => resolve({ status: "ok" }), ms); + return () => clearTimeout(id); + }, + ); +} +``` + +### 6.2 durableCall + +```typescript +function* durableCall( + name: string, + fn: () => Promise, +): Workflow { + return (yield createDurableEffect( + { type: "call", name }, + (resolve) => { + fn().then( + (value) => resolve({ status: "ok", value: value as Json }), + (error) => { + resolve({ + status: "err", + error: serializeError( + error instanceof Error ? error : new Error(String(error)), + ), + }); + }, + ); + return () => {}; + }, + )) as T; +} +``` + +### 6.3 durableAction + +```typescript +function* durableAction( + name: string, + executor: ( + resolve: (value: T) => void, + reject: (error: Error) => void, + ) => () => void, +): Workflow { + return (yield createDurableEffect( + { type: "action", name }, + (protocolResolve, reject) => { + return executor( + (value: T) => + protocolResolve({ status: "ok", value: value as Json }), + reject, + ); + }, + )) as T; +} +``` + +### 6.4 versionGate (§9) + +```typescript +function* versionCheck( + name: string, + opts: { minVersion: number; maxVersion: number }, +): Workflow { + return (yield createDurableEffect( + { type: "version_gate", name }, + (resolve) => { + resolve({ status: "ok", value: opts.maxVersion }); + return () => {}; + }, + )) as number; +} +``` + +### 6.5 Workflow composition + +Workflows compose exactly like operations — via `yield*`: + +```typescript +function* orderWorkflow(orderId: string): Workflow { + let version = yield* versionCheck("add-fraud-check", { minVersion: 0, maxVersion: 1 }); + + if (version >= 1) { + yield* durableCall("fraudCheck", () => fraudCheck(orderId)); + } + + let order = yield* durableCall("fetchOrder", () => fetchOrder(orderId)); + yield* durableCall("chargeCard", () => chargeCard(order.payment)); +} +``` + +--- + +## 7. Coroutine identity (validated) + +### 7.1 Per-parent counter scheme (§3) + +Coroutine IDs are dot-delimited paths assigned deterministically: + +``` +root → "root" + first child of root → "root.0" + second child of root → "root.1" + first child of .1 → "root.1.0" +``` + +### 7.2 Implementation via DurableContext + +The coroutine ID is part of the `DurableContext` stored on each scope. When +a durable scope spawns a child, it increments a per-scope counter and +constructs the child's ID: + +```typescript +interface DurableContext { + replayIndex: ReplayIndex; + stream: DurableStream; + coroutineId: CoroutineId; + childCounter: number; +} +``` + +When `durableSpawn()` creates a child scope: + +```typescript +let parentCtx = scope.expect(DurableCtx); +let childId = `${parentCtx.coroutineId}.${parentCtx.childCounter++}`; +childScope.set(DurableCtx, { + replayIndex: parentCtx.replayIndex, // shared + stream: parentCtx.stream, // shared + coroutineId: childId, + childCounter: 0, +}); +``` + +### 7.3 Why determinism holds + +The spec identifies four properties (§3.2): + +1. **Synchronous reducer.** ✅ Effection's `Reducer.reduce()` is synchronous + with a re-entrancy guard. No async gaps. +2. **Deterministic generators.** ✅ Same inputs → same yields. `for...of` in + `all()` processes operands in array order. +3. **Replay preserves ordering.** ✅ Replayed effects resolve synchronously + inside `DurableEffect.enter()`, so spawns happen in the same order. +4. **Priority ordering is structural.** ✅ `PriorityQueue` orders by scope + depth (the `Priority` context), FIFO within a tier. + +### 7.4 Teardown spawns (§3.4) + +When `ensure()` or `finally` blocks spawn children during scope destruction, +the `childCounter` continues incrementing on the same `DurableContext`. This +is safe because teardown order is deterministic (reverse creation order via +the scope's destructor set in `buildScopeInternal`). + +--- + +## 8. Scope management: spawn, join, Close events (validated) + +Implemented in `combinators.ts`. The actual architecture differs from +earlier sketches — combinators are `Operation` generators (not `DurableEffect`s), +and they delegate to Effection's native `spawn()`, `all()`, and `race()`. + +### 8.1 runDurableChild — the core building block + +All three combinators share a single helper, `runDurableChild()`, that +wraps any child workflow with DurableContext setup and Close event handling. +This is an `Operation` meant to run inside a `spawn()`: + +```typescript +function* runDurableChild( + childWorkflow: () => Workflow | Operation, + childId: string, + parentCtx: DurableContext, +): Operation { + const { replayIndex, stream } = parentCtx; + + // Short-circuit: child already completed in a previous run + if (replayIndex.hasClose(childId)) { + const closeEvent = replayIndex.getClose(childId)!; + if (closeEvent.result.status === "ok") { + return closeEvent.result.value as T; + } else if (closeEvent.result.status === "err") { + throw deserializeError(closeEvent.result.error); + } else { + // Cancelled in previous run — suspend until parent cancels us + yield* suspend(); + return undefined as T; // unreachable + } + } + + // Set child's DurableContext + const scope = yield* useScope(); + scope.set(DurableCtx, { + replayIndex, stream, + coroutineId: childId, + childCounter: 0, + }); + + let closeEvent: Close | undefined; + try { + const result: T = yield* childWorkflow(); + closeEvent = { + type: "close", coroutineId: childId, + result: { status: "ok", value: result as Json }, + }; + return result; + } catch (error) { + closeEvent = { + type: "close", coroutineId: childId, + result: { + status: "err", + error: serializeError( + error instanceof Error ? error : new Error(String(error)), + ), + }, + }; + throw error; + } finally { + if (!closeEvent) { + closeEvent = { + type: "close", coroutineId: childId, + result: { status: "cancelled" }, + }; + } + // Don't re-emit if journal already has this Close + if (!replayIndex.hasClose(childId)) { + yield* call(() => stream.append(closeEvent!)); + } + } +} +``` + +Key design decisions in this helper: + +**Short-circuit on existing Close.** If the journal already has a Close for +this child, `runDurableChild` never runs the workflow. For `ok` and `err`, +it returns/throws immediately. For `cancelled`, it uses `yield* suspend()` +(see §8.4). + +**Close events via try/catch/finally.** The `closeEvent` variable starts +undefined. Normal completion sets it in the try block. Errors set it in +catch. If both are skipped (cancellation via `iterator.return()`), it +stays undefined and finally assigns `cancelled`. This covers all three +terminal states with plain JavaScript. + +**No re-emission guard.** The `if (!replayIndex.hasClose(childId))` check +in finally prevents writing a duplicate Close when replaying a child that +was already completed. Without this, a fully-replayed child that short- +circuits would emit a second Close event. + +**`yield* call()` for stream append.** The finally block uses Effection's +`call()` to await the stream append within generator context, rather than +raw `await`. This keeps the code within Effection's structured concurrency +model. + +### 8.2 durableSpawn + +Spawns a single durable child. Returns `Operation>` (not +`Workflow>`) because it uses infrastructure effects internally: + +```typescript +function* durableSpawn( + childWorkflow: () => Workflow | Operation, +): Operation> { + const scope = yield* useScope(); + const ctx = scope.expect(DurableCtx); + const childId = `${ctx.coroutineId}.${ctx.childCounter++}`; + return yield* spawn(() => runDurableChild(childWorkflow, childId, ctx)); +} +``` + +Uses Effection's `spawn()` directly via `yield*`. The child runs +concurrently in its own scope. `runDurableChild` handles DurableContext +setup and Close events. + +### 8.3 durableAll + +Fork/join — runs all children concurrently, waits for all to complete: + +```typescript +function* durableAll( + workflows: (() => Workflow | Operation)[], +): Operation { + const scope = yield* useScope(); + const ctx = scope.expect(DurableCtx); + + const childOps: Operation[] = workflows.map((workflow) => { + const childId = `${ctx.coroutineId}.${ctx.childCounter++}`; + return { + *[Symbol.iterator]() { + return yield* runDurableChild(workflow, childId, ctx); + }, + }; + }); + + return yield* effectionAll(childOps); +} +``` + +Delegates to Effection's native `all()`, which provides error isolation +via `trap()` internally. When any child fails, remaining siblings are +cancelled — Effection handles this, and `runDurableChild`'s finally block +emits `Close(cancelled)` for each. + +### 8.4 Cancellation replay via suspend() + +When replaying a child that was cancelled in a previous run (journal has +`Close(cancelled)`), the child cannot simply throw or return — that would +change the control flow the parent combinator sees. Instead, `runDurableChild` +calls `yield* suspend()`, which blocks the child forever. + +This works because the parent combinator (`durableRace` or `durableAll` +with a failed sibling) will cancel this child as part of normal structured +concurrency teardown — the same thing that happened in the original run. +The `suspend()` just holds the child alive until that cancellation arrives. + +The `Close(cancelled)` event already exists in the journal, so the finally +block's `if (!replayIndex.hasClose(childId))` guard skips re-emission. + +This is the only correct approach: it makes replay indistinguishable from +live execution from the parent's perspective. The parent's race/all sees +exactly the same behavior — one child completes, the others get cancelled. + +### 8.5 durableRace + +First child to complete wins, others cancelled: + +```typescript +function* durableRace( + workflows: (() => Workflow | Operation)[], +): Operation { + const scope = yield* useScope(); + const ctx = scope.expect(DurableCtx); + + const childOps: Operation[] = workflows.map((workflow) => { + const childId = `${ctx.coroutineId}.${ctx.childCounter++}`; + return { + *[Symbol.iterator]() { + return yield* runDurableChild(workflow, childId, ctx); + }, + }; + }); + + return yield* effectionRace(childOps); +} +``` + +Uses Effection's native `race()`, which spawns all children and returns +the first to complete, cancelling the rest. During replay, the winner +short-circuits from its stored Close(ok), the losers short-circuit into +suspend() from their stored Close(cancelled), and Effection's race cancels +the suspended losers — identical to the original run. + +### 8.6 Close event ordering + +Causal ordering is enforced naturally by the control flow: + +1. Child completes → `runDurableChild` appends Close in finally +2. Child Close is awaited via `yield* call()` +3. Parent resumes only after the child's generator finishes (Effection's + scope lifetime guarantee) +4. Parent's subsequent Yield events are appended after all child Closes + +For `durableAll`: all child Closes precede the parent's post-join effects. +For `durableRace`: winner's Close precedes losers' Close(cancelled), all +precede the parent's next effect. + +--- + +## 9. Replay index (validated) + +Implemented in `replay-index.ts` with 21 unit tests (DEC-013). +Follows the spec §4.1 exactly with no extensions beyond `getCursor()` +and `yieldCount()` diagnostic accessors. + +### 9.1 Structure + +Built from the stream on startup. Provides per-coroutine cursored access: + +```typescript +class ReplayIndex { + private yields = new Map>(); + private cursors = new Map(); + private closes = new Map(); + + constructor(events: DurableEvent[]) { + for (const event of events) { + if (event.type === "yield") { + const list = this.yields.get(event.coroutineId) ?? []; + list.push({ description: event.description, result: event.result }); + this.yields.set(event.coroutineId, list); + } + if (event.type === "close") { + this.closes.set(event.coroutineId, event); + } + } + } + + peekYield(id: CoroutineId) { + const list = this.yields.get(id); + const cursor = this.cursors.get(id) ?? 0; + return list?.[cursor]; + } + + consumeYield(id: CoroutineId) { + const cursor = this.cursors.get(id) ?? 0; + this.cursors.set(id, cursor + 1); + } + + getCursor(id: CoroutineId): number { + return this.cursors.get(id) ?? 0; + } + + hasClose(id: CoroutineId): boolean { + return this.closes.has(id); + } + + getClose(id: CoroutineId): Close | undefined { + return this.closes.get(id); + } + + isFullyReplayed(id: CoroutineId): boolean { + return this.peekYield(id) === undefined && this.hasClose(id); + } + + yieldCount(id: CoroutineId): number { + return this.yields.get(id)?.length ?? 0; + } +} +``` + +### 9.2 Stored as Effection Context + +The replay index is part of `DurableContext`, set on the root scope when +a durable execution begins. All child scopes inherit it (Effection contexts +use prototypal inheritance). Each child scope has its own `coroutineId` and +`childCounter` but shares the same `replayIndex` and `stream`. + +--- + +## 10. Entry point: durableRun (validated) + +Implemented in `run.ts`. Key implementation details beyond the +original design: short-circuits on existing Close event (DEC-016), +checks for early-return divergence after workflow completes, and +emits Close(err) on exceptions. + +The entry point creates a scope, builds the replay index, sets up the +durable context, and runs the workflow: + +```typescript +interface DurableRunOptions { + stream: DurableStream; + coroutineId?: string; +} + +async function durableRun( + workflow: () => Workflow | Operation, + options: DurableRunOptions, +): Promise { + const { stream, coroutineId = "root" } = options; + const events = await stream.readAll(); + const replayIndex = new ReplayIndex(events); + + // Short-circuit: root already completed in a previous run + if (replayIndex.hasClose(coroutineId)) { + const closeEvent = replayIndex.getClose(coroutineId)!; + if (closeEvent.result.status === "ok") return closeEvent.result.value as T; + if (closeEvent.result.status === "err") throw deserializeError(closeEvent.result.error); + throw new Error("Workflow was cancelled"); + } + + const [scope, destroy] = createScope(); + scope.set(DurableCtx, { replayIndex, stream, coroutineId, childCounter: 0 }); + + try { + // Workflow is structurally assignable to Operation — no cast needed + const task = scope.run(workflow); + const result = await task; + + // §6.3: Check for early return divergence + const cursor = replayIndex.getCursor(coroutineId); + const totalYields = replayIndex.yieldCount(coroutineId); + if (cursor < totalYields) { + throw new EarlyReturnDivergenceError(coroutineId, cursor, totalYields); + } + + await stream.append({ + type: "close", coroutineId, + result: { status: "ok", value: result as Json }, + }); + return result; + } catch (error) { + await stream.append({ + type: "close", coroutineId, + result: { status: "err", error: serializeError(error) }, + }); + throw error; + } finally { + try { await destroy(); } catch { /* swallow scope cleanup errors */ } + } +} +``` + +Key details: + +- **Short-circuit on existing Close.** If the journal already has a Close + for the root coroutine, the workflow completed in a previous run. Return + the stored result directly without creating a scope or running the + workflow. This is why full-replay tests show zero effect executions and + zero appends. + +- **`Workflow | Operation` union.** Accepts either type. Combinators + now return `Workflow` (they self-wrap with `ephemeral()`), so the + `Operation` arm is primarily for backward compatibility and edge cases + where users pass a raw Operation at the top level. Structural compatibility + means no cast is needed at the `scope.run()` call site. + +- **Early return divergence check.** After the workflow returns, checks + if the replay index has unconsumed yields. If so, the generator finished + before replaying all journal entries — the code has changed (§6.3). + +- **Swallowing destroy errors.** If the workflow threw, `destroy()` may + also throw "halted". The `try { await destroy() } catch {}` in finally + prevents masking the original error. + +--- + +## 11. What can and cannot be used in workflows (validated) + +### 11.1 Three categories of durable operations + +**Leaf effects** return `Workflow` — they yield a single `DurableEffect` +and are the atomic units of durable execution: + +| Leaf effect | Equivalent Effection operation | +|-------------|-------------------------------| +| `durableSleep(ms)` | `sleep(ms)` | +| `durableCall(name, fn)` | `call(fn)` | +| `durableAction(name, executor)` | `action(executor)` | +| `versionCheck(name, opts)` | (new, no equivalent) | + +**Combinators** return `Workflow` — they self-wrap their infrastructure +effects (`useScope`, `spawn`) in `ephemeral()` and delegate to Effection's +native structured concurrency primitives. Child signatures are tightened to +`() => Workflow`; bare Operations are rejected at compile time. + +| Combinator | Equivalent Effection operation | +|------------|-------------------------------| +| `durableSpawn(workflow)` | `spawn(operation)` | +| `durableAll([...workflows])` | `all([...operations])` | +| `durableRace([...workflows])` | `race([...operations])` | + +Because combinators return `Workflow`, top-level workflows that use them +can also be typed as `Workflow`. The infrastructure effects wrapped in +`ephemeral()` produce no Yield events — only the child workflows' +`DurableEffect` values appear in the journal. + +**Escape hatch** — `ephemeral(operation: Operation): Workflow` wraps +a non-durable Operation so it can be used inside a Workflow. It is transparent +to the journal (no Yield event, no replay index entry) and re-runs on replay. +This is analogous to Rust's `unsafe {}` — every non-durable Operation that +participates in a Workflow must go through `ephemeral()`, making the escape +explicit and auditable. Users rarely need this directly since combinators +self-wrap internally, but it is available for custom infrastructure +Operations. See DEC-034. + +| Escape hatch | Purpose | +|-------------|---------| +| `ephemeral(operation)` | Wrap non-durable Operation for use in Workflow | + +### 11.2 Rejected (type error) + +| Operation | Why it's rejected | +|-----------|-------------------| +| `useAbortSignal()` | Returns a scope-bound resource that doesn't survive replay | +| `each(stream)` | Streams are stateful subscriptions, not serializable | +| `resource(fn)` | Resources hold live state (connections, handles) | +| `on(target, name)` | EventTarget-based, not serializable | +| `sleep(ms)` | Effection's `sleep` — use `durableSleep` instead | +| `call(fn)` | Effection's `call` — use `durableCall` instead | + +### 11.3 The boundary is intentional + +This is the "freeing" quality Charles described. You don't have to think about +whether something is safe for durable execution — the compiler tells you. The +set of workflow-enabled effects is small and explicit. Each one has a clear +contract: it carries a structured description, it handles its own replay, and +its result is JSON-serializable. + +If you intentionally need a non-durable Operation inside a Workflow, `ephemeral()` +makes the boundary visible — every `ephemeral()` call is an auditable point +where durable guarantees are relaxed. This is the same principle as Rust's +`unsafe {}`: the type system enforces safety by default, and the escape hatch +is explicit. + +--- + +## 12. Design questions — status + +Most questions from the initial analysis have been resolved through +implementation. Decisions are recorded in `DECISIONS.md` (29 entries). +Key validations: + +| Question | Status | Decision | +|----------|--------|----------| +| Workflow type constraint | ✅ Resolved | `Generator, T, unknown>` (DEC-009) | +| DurableEffect ↔ Effect compatibility | ✅ Resolved | Structural match, no casts needed (DEC-010, DEC-015) | +| routine.scope accessibility | ✅ Resolved | Confirmed in Effection 4.1 alpha source (DEC-012) | +| Replay/live dispatch location | ✅ Resolved | Inside `enter()`, no reducer changes (DEC-014) | +| Persist-before-resume strategy | ✅ Resolved | Strategy B — async append + deferred resolve (DEC-017) | +| Serialization boundary | ✅ Resolved | `T extends Json` type constraint (DEC-018) | +| DurableStream interface | ✅ Resolved | `readAll()` + `append()`, InMemoryStream for tests, HttpDurableStream for production | +| Terminal divergence detection | ✅ Resolved | Both cases implemented with 3 error classes (DEC-008) | +| durableSpawn implementation | ✅ Resolved | Operations using Effection's native spawn/all/race | +| HTTP backend adapter | ✅ Resolved | Raw fetch writes, promise chain serialization, epoch fencing (DEC-026–029) | +| Batch persistence | ⏳ Deferred | Optimization for concurrent children, not blocking correctness. See §15.1 | +| Durable `each()` | ✅ Resolved | Operation-native `DurableSource`, module-level state, `ephemeral()` wrapping (DEC-030) | +| `ephemeral()` escape hatch | ✅ Resolved | Explicit adapter for non-durable Operations in Workflows (DEC-034) | +| Continue-As-New | ⏳ Future | Journal compaction for long-running loops. Tightly coupled with durableEach. See §15.2 | + +### 12.1 Structured concurrency combinators (resolved) + +The combinators are **`Workflow` generators that self-wrap with `ephemeral()`, +not `DurableEffect`s.** This is a significant departure from the earlier +design sketches. The key insight: combinators don't need to be durable effects +because spawns are not journaled. They're pure scope plumbing that delegates +to Effection's native structured concurrency primitives. The `ephemeral()` +wrapper makes them return `Workflow` so they compose seamlessly with other +durable operations. + +```typescript +// durableSpawn, durableAll, durableRace all return Workflow +function* durableSpawn(op): Workflow> { ... } +function* durableAll(ops): Workflow { ... } +function* durableRace(ops): Workflow { ... } +``` + +**How combinators interact with the type system.** A `Workflow` is +`Generator, T, unknown>` — it constrains what the +generator *yields*. Combinators use infrastructure effects (`useScope()`, +`spawn()`) internally, but wrap them in `ephemeral()` which produces a +`DurableEffect` (transparent to the journal). This means `yield* durableAll(...)` +inside a generator annotated as `Workflow` works — the combinators satisfy +the yield constraint. + +Top-level workflows that use combinators can be typed as `Workflow`: + +```typescript +// This is typed as Workflow — combinators return Workflow +function* myWorkflow(): Workflow { + const prefix = yield* durableCall("step1", () => fetchPrefix()); + const results = yield* durableAll([ + function* () { return yield* durableCall("a", () => fetchA()); }, + function* () { return yield* durableCall("b", () => fetchB()); }, + ]); + return `${prefix}-${results.join(",")}`; +} +``` + +**Child signatures tightened to `() => Workflow`.** Combinators no longer +accept `Operation` children. This is the primary safety boundary — users +cannot accidentally pass bare Operations whose effects wouldn't be journaled. +To intentionally use a non-durable Operation as a child, wrap it in +`ephemeral()`: + +```typescript +yield* durableAll([ + function* () { return yield* durableCall("a", () => fetchA()); }, + function* () { + // Explicit escape hatch — this Operation won't be journaled + return yield* ephemeral(someInfrastructureOperation()); + }, +]); +``` + +**Why not the DurableEffect-in-enter() approach.** The earlier sketch had +durableSpawn as a raw `DurableEffect` calling `scope.spawn()` inside +`enter()`. This had problems: (1) `CoroutineView` didn't expose `spawn()`, +(2) mixing imperative scope calls inside `enter()` is fighting Effection's +design rather than working with it, (3) it couldn't use `all()` or `race()` +which are generator-based. Using Effection's native combinators gets error +isolation (`trap()`), cancellation propagation, and scope lifetime +management for free. + +**The `runDurableChild` helper.** All three combinators share a single +helper that wraps child workflows with DurableContext and Close event +handling. See §8.1 for the full implementation. This is the single point +of responsibility for child lifecycle — DurableContext setup, Close event +short-circuiting, the suspend() trick for cancelled replay, and the +no-re-emission guard. Child signatures are `() => Workflow` — matching +the combinator's public API. + +### 12.2 Serialization boundary (resolved) + +`durableCall` constrains the return type at compile time +(DEC-018). Non-serializable values (Dates, BigInts, class instances) are +rejected by TypeScript. The constraint is intentionally strict — relaxing +later is easy, tightening would be breaking. + +Error serialization is implemented in `serialize.ts` with +`serializeError()` / `deserializeError()`. Custom error properties beyond +`message`, `name`, and `stack` are lost — this matches the spec's +`SerializedError` shape. + +Remaining design space: tagged encoding for Dates/BigInts could be added +as a future `DurableCodec` extension without changing the core protocol. + +### 12.3 DurableStream interface (resolved) and HttpDurableStream backend + +Implemented in `stream.ts` with the minimal interface: + +```typescript +interface DurableStream { + readAll(): Promise; + append(event: DurableEvent): Promise; +} +``` + +`InMemoryStream` implements this for testing, with hooks for tracking +append counts, injecting failures, and observing append ordering (used +by the persist-before-resume test). + +`HttpDurableStream` (`http-stream.ts`) implements this for production +use, backed by HTTP calls to a Durable Streams server. Key design +decisions (DEC-026 through DEC-029): + +- **Raw `fetch()` for writes, not IdempotentProducer** (DEC-026). The + producer's fire-and-forget model is wrong for persist-before-resume. + Raw fetch gives full control over the request/response cycle and + captures `Stream-Next-Offset` from every response. +- **Promise chain serialization for concurrent appends** (DEC-027). + Sequence numbers assigned synchronously, HTTP calls chained behind + `this.pending` to prevent out-of-order arrival. +- **Close events in finally are best-effort** (DEC-028). Missing Close + events mean re-execution on replay, which is idempotent. +- **`lastOffset` tracked from every response** (DEC-029). This is the + resumption point for future `tail()` calls. Not consumed yet but + available for the tailing feature. + +All unexpected HTTP statuses (including transient 500/503) are treated +as fatal, setting `this.fatalError` so future appends fail-fast. This is +the safe choice when the sequence state is uncertain — a failed append +may or may not have persisted, so continuing with the next seq number +risks a 409 gap. A future version could add retry-with-same-seq logic +for transient errors, but that requires careful handling of the ambiguity +window. + +Tests: `smoke.test.ts` — 11 tests against a real +`DurableStreamTestServer`, covering round-trip, empty reads, idempotent +dedup, epoch fencing, full `durableRun`, replay, concurrent appends via +`durableAll`, network errors, offset tracking, fail-fast, and error +preservation. + +#### Verified: `readAll()` response shape ✓ + +`HttpDurableStream.readAll()` uses `stream()` from +`@durable-streams/client@0.2.1`. Both assumptions have been verified +by inspecting the client library source: + +1. **Wire format — confirmed.** `stream()` returns a `StreamResponseImpl`. + `res.json()` returns a proper JSON array (`Content-Type: application/json`), + not NDJSON. `res.json()` → `DurableEvent[]` works correctly. + +2. **Client library API shape — confirmed.** `res.offset` is a prototype + getter on `StreamResponseImpl` that returns the value of the + `Stream-Next-Offset` response header. No need for manual header access. + +Both assumptions hold. The current `readAll()` implementation is correct. +Documented in `http-stream.ts`. + +#### Future interface evolution + +The `DurableStream` interface is deliberately minimal. Future features +add methods without changing existing ones: + +```typescript +interface DurableStream { + readAll(): Promise; // catch-up (exists) + append(event: DurableEvent): Promise; // write (exists) + tail(offset: string): AsyncIterable; // future: SSE/long-poll + readFrom(offset: string): Promise; // future: cursor-based +} +``` + +`readAll()` stays unchanged — it's the startup catch-up for building the +ReplayIndex. `tail()` watches for live events after catch-up (needed for +external workflow observers and multi-worker coordination). `readFrom()` +is cursor-based partial reads (needed for `durableEach` checkpoint +resumption). Both are additive — writes are always HTTP POST regardless +of how you read. The `lastOffset` field already tracked on +`HttpDurableStream` is the resumption point for both. + +The Durable Streams protocol explicitly supports this transition: "catch +up then tail" is a first-class pattern where you read from offset `-1`, +get `Stream-Up-To-Date: true`, then switch to `?live=sse` or +`?live=long-poll` at your last offset. + +### 12.4 Batch persistence (Strategy C) + +During `all()` with multiple children, several effects may resolve in the +same reducer tick (especially during replay-to-live transition). The spec's +Strategy C suggests batching writes. This could be implemented by having +`DurableEffect.enter()` enqueue writes to a buffer on the `DurableContext`, +with the buffer flushed at the end of each reduce cycle. + +### 12.5 Terminal divergence detection (resolved) + +Both cases from §6.3 are implemented and tested (DEC-008): + +1. **Generator finishes early.** Detected in `durableRun()` after the + workflow returns — if `cursor < totalYields`, throws + `EarlyReturnDivergenceError`. Tested in divergence test 9 and 13. + +2. **Generator continues past close.** Detected in + `createDurableEffect.enter()` — when `peekYield()` returns undefined + but `hasClose()` returns true, throws + `ContinuePastCloseDivergenceError`. Tested in divergence test 14. + +Three distinct error classes share `name = "DivergenceError"` for +catch-all handling but carry different diagnostic fields for precise +`instanceof` checks. + +### 12.6 Durable `each()` — design and implementation plan + +Charles's example showed the powerful implication: in a durable workflow, +each iteration of a loop can run on a different VM. This requires a durable +iteration primitive that checkpoints its position after each item. + +#### The `for...of` constraint + +JavaScript's `for...of` calls `iterator.next()` synchronously. Inside a +generator, there is no opportunity to yield a DurableEffect between the +`for...of` calling `next()` and the loop body receiving the value. This +means the naive design — where each `next()` call is itself a DurableEffect +— cannot use `for...of`. + +The solution is the **pre-fetch pattern**: fetch the next item *before* +the `for...of` iterator is re-entered. The synchronous `next()` just +returns an already-fetched value. + +#### User-facing API + +Consistent with Effection's `each()` / `each.next()` pattern: + +```typescript +function* processQueue(): Workflow { + for (let msg of yield* durableEach("queue", source)) { + yield* durableCall("process", () => process(msg)); + yield* durableEach.next(); // checkpoint + pre-fetch next item + // crash here → resume picks up at next message + } +} +``` + +How the cycle works: + +1. `yield* durableEach("queue", source)` — yields a DurableEffect that + fetches item 1 from the source (or replays it from the journal). + Stores state in an Effection context. Returns a synchronous iterable. + +2. `for (let msg of yield* ...)` — calls the iterable's `next()` + synchronously. The iterator is a generator: `while (state.current + !== done) { yield state.current; }`. It yields the pre-fetched + item 1. + +3. Loop body runs — `yield* durableCall(...)` journals the processing. + +4. `yield* durableEach.next()` — reads state from Effection context, + yields a DurableEffect that fetches item 2 from the source (or + replays from journal). Updates `state.current`. Sets + `state.advanced = true`. + +5. Back to `for...of` — calls the iterator's synchronous `next()`, + re-enters the while loop, sees `state.current` is item 2, yields it. + +6. When the source is exhausted, `yield* durableEach.next()` sets + `state.current` to the done sentinel. The while loop exits, + `for...of` sees `{ done: true }`, loop ends. + +#### Advance guard + +Without `yield* durableEach.next()`, the `for...of` spins forever on +the same item — `state.current` never advances. This is the most obvious +footgun in the API, so `durableEach` detects it at runtime using an +`advanced` flag on the shared state (see the implementation sketch +below for the full code). + +The flag cycle: iterator yields → sets `advanced = false` → loop body +→ `yield* durableEach.next()` sets `advanced = true` + fetches → +iterator re-enters while → checks `advanced` → yields next item. If the +iterator is re-entered with `advanced` still false, it throws +immediately with a message telling the developer exactly what to do. + +Edge cases: + +- **`break` or `return` inside the loop.** The iterator isn't + re-entered, so the check never fires. Legitimate early exit works. +- **`continue` without `durableEach.next()`.** The iterator re-enters, + sees `advanced` is false, throws. This is correct — skipping without + checkpointing means a crash would re-deliver the skipped item, + violating the "resume at next unconsumed item" contract. To skip + an item, call `yield* durableEach.next()` before `continue`. + +#### Types + +```typescript +/** Source of items for durable iteration (Operation-native). */ +interface DurableSource { + /** Read the next item, blocking until available. */ + next(): Operation<{ value: T } | { done: true }>; + /** Teardown — called on cancellation or completion. Must be idempotent. */ + close?(): void; +} + +/** State shared between durableEach and durableEach.next(). */ +interface DurableEachState { + name: string; + source: DurableSource; + current: T | typeof DONE; + advanced: boolean; +} +``` + +Note: `DurableSource.next()` returns `{ value: T } | { done: true }` +rather than `T | null` because `null` is valid JSON — a source that +legitimately produces null items would signal false exhaustion with a +null sentinel. + +The optional `close()` method handles teardown on cancellation. Without +it, if the workflow is cancelled while `source.next()` is awaiting +(long-poll on a queue, database cursor), the pending read holds a +connection open indefinitely. The `createDurableEffect` teardown +function should call `source.close?.()`. + +#### Journal shape + +Each `yield* durableEach.next()` and the initial fetch in `yield* durableEach()` +produce identical Yield events: + +``` +[0] yield root { type: "each", name: "queue" } result: { status: "ok", value: { value: msg1 } } +[1] yield root { type: "call", name: "process" } result: { status: "ok" } +[2] yield root { type: "each", name: "queue" } result: { status: "ok", value: { value: msg2 } } +[3] yield root { type: "call", name: "process" } result: { status: "ok" } +[4] yield root { type: "each", name: "queue" } result: { status: "ok", value: { done: true } } +[5] close root result: { status: "ok" } +``` + +The `{ value: T } | { done: true }` wrapper is stored directly in +the result's value field. Position-based divergence detection handles +repeated identical descriptions (`{ type: "each", name: "queue" }`) +correctly — matching is by cursor position, not description uniqueness. + +On replay, stored items are fed back from the journal without +re-reading from the source. The source's `next()` is never called +during replay. + +#### Implementation sketch + +```typescript +// Sentinel for source exhaustion (not exported) +const DONE = Symbol("durableEach.done"); +type ItemOrDone = T | typeof DONE; + +// Module-level state for sharing between durableEach and durableEach.next(). +// Safe because durable execution is single-threaded. +let activeState: DurableEachState | null = null; + +function durableEachFetch( + name: string, + source: DurableSource, +): Workflow> { + return (function* () { + const result = (yield createDurableOperation<{ value: T } | { done: true }>( + { type: "each", name }, + () => source.next(), + )) as { value: T } | { done: true }; + + if ("done" in result) return DONE; + return result.value; + })(); +} + +// Internal: returns Operation> because ensure() is infrastructure +function* _durableEachOp( + name: string, + source: DurableSource, +): Operation> { + yield* ensure(() => { source.close?.(); }); + + const first: ItemOrDone = yield* durableEachFetch(name, source); + + // Store state in module-level slot for durableEach.next() to access + const state: DurableEachState = { + name, + source, + current: first, + advanced: true, + }; + activeState = state as DurableEachState; + + return { + *[Symbol.iterator]() { + try { + while (!isDone(state.current)) { + if (!state.advanced) { + throw new Error( + `durableEach("${name}"): yield* durableEach.next() must be ` + + `called before the next iteration.` + ); + } + state.advanced = false; + yield state.current as T; + } + } finally { + activeState = null; + source.close?.(); + } + }, + }; +} + +// Public API: wraps in ephemeral() to return Workflow> +function* _durableEach( + name: string, + source: DurableSource, +): Workflow> { + return yield* ephemeral(_durableEachOp(name, source)); +} + +// Static method — pure Workflow, no infrastructure effects +durableEach.next = function* (): Workflow { + if (activeState === null) { + throw new Error("durableEach.next(): no active durableEach iteration."); + } + const state = activeState as DurableEachState; + state.current = yield* durableEachFetch(state.name, state.source); + state.advanced = true; +}; +``` + +Key design choices: + +- **Module-level state sharing.** State is stored in a module-level + `activeState` variable (not Effection context). This is safe because + durable execution is single-threaded — only one coroutine runs at a + time. This avoids the scope isolation problem that arises when both + `durableEach()` and `durableEach.next()` are individually wrapped in + `ephemeral()`: each `ephemeral()` call creates an isolated child scope + via `scope.run()`, making Effection context set in one child invisible + to the other. + +- **`durableEach` wraps in `ephemeral()`; `durableEach.next()` does not.** + `durableEach` uses `ensure()` (an infrastructure Operation), so it + needs `ephemeral()` to satisfy the `Workflow` return type. + `durableEach.next()` only reads module-level state and calls + `durableEachFetch` (a pure `Workflow`), so it is itself a pure `Workflow` + with no `ephemeral()` needed. + +- **Both return `Workflow`.** Unlike the previous design where both + returned `Operation`, the current implementation returns `Workflow` + — `durableEach` via `ephemeral()` wrapping, `durableEach.next()` natively. + This means they compose cleanly inside `Workflow`-annotated generators. + +- **Operation-native `DurableSource.next()`.** The source interface uses + `next(): Operation<...>` instead of `next(): Promise<...>`. This enables + full structured concurrency — cancellation of the scope cancels the + in-flight `source.next()` call via Effection's normal teardown. + +- **Symbol sentinel for exhaustion.** `DONE` is a private Symbol, + not `null` or `undefined`. Cannot collide with any JSON value from + the source. + +- **Source teardown in effect teardown.** The `createDurableEffect` + teardown function calls `source.close?.()`, so cancellation during + a pending `source.next()` can clean up (abort fetch, close cursor, + release connection). + +- **durableEachFetch is the only DurableEffect.** Both the initial + fetch (inside `durableEach`) and subsequent fetches (inside + `durableEach.next()`) go through the same helper. Same effect + description, same journal format, same replay path. + +#### Three approaches to checkpointing (background) + +The implementation above uses **Option A: yield-per-item**. Two +alternative approaches exist for future consideration: + +**Option B: Cursor checkpoint.** Instead of recording each item, +record a cursor/offset that represents "I've processed up to here." +On replay, the runtime reads from the cursor position, not from the +start. Requires the source to support cursor-based reads — which maps +to the Durable Streams `readFrom(offset)` pattern or any external +system with offset semantics (Kafka consumer offsets, database +sequences, SQS receipt handles). Smaller journals, faster replay. +But the source must be re-readable from a position, which not all +sources support (transient webhook streams, one-shot HTTP responses). + +**Option C: Hybrid with Continue-As-New.** Record items in the +journal (like A), but periodically compact by starting a new execution +with a fresh journal. After N iterations, `durableRun` returns a +continuation token (cursor position + accumulated state), and the +scheduler starts a new execution seeded with that token. Works with +any source. Bounds journal growth. But requires Continue-As-New as +a separate feature (see §15.2). + +Option A is the right starting point: no new `DurableStream` methods +needed, no new features required. The unbounded journal limitation is +acceptable for initial use cases with bounded iteration counts (process +a batch of N items, not an infinite stream). Add `readFrom(offset)` and +Continue-As-New as follow-on work when journal size becomes a practical +constraint. + +#### Interaction with durableAll + +When durableEach feeds items into parallel processing pipelines: + +```typescript +function* fanOut(): Workflow { + const batch: Json[] = []; + for (let msg of yield* durableEach("queue", source)) { + batch.push(msg); + if (batch.length === 10) { + yield* durableAll(batch.map(m => + function*() { yield* durableCall("process", () => process(m)); } + )); + batch.length = 0; + } + yield* durableEach.next(); + } +} +``` + +This produces bursts of concurrent appends (10 children resolving in +the same tick), making batch persistence (Strategy C, §15.1) a +performance concern. Without it, each child's Yield event is a separate +HTTP POST awaited sequentially via the promise chain. + +#### Interaction with Effection's `each()` + +Effection's `each(subscription)` consumes streams within structured +concurrency using a channel-based protocol. `durableEach` mirrors the +same API pattern — `each()` returns an iterable, `each.next()` is a +static method that advances via context — but cannot wrap `each()` +directly because `each()` yields infrastructure effects +(`Effect`, not `DurableEffect`). The type constraint +rejects it. + +`durableEach` re-implements the pre-fetch pattern using `useScope()` +and Effection contexts, matching `each()`'s ergonomics while staying +within the durable type system. The two serve different purposes: +Effection's `each()` is for reactive stream consumption within a scope; +`durableEach` is for durable checkpoint-based consumption that survives +crashes. + +--- + +## 13. Summary of what doesn't change + +| Component | Changes? | Notes | +|-----------|----------|-------| +| `Reducer` | No | Unchanged. Calls `enter()` on effects as always. | +| `Effect` | No | Unchanged. `DurableEffect` extends it. | +| `Operation` | No | Unchanged. `Workflow` is a subtype. | +| `Scope` / `ScopeInternal` | No | Unchanged. Durable context stored via existing Context system. | +| `Context` | No | Unchanged. Used to store `DurableContext`. | +| `Api.around()` | No | Unchanged. Not used — Close events handled via try/finally in `runDurableChild`. | +| `PriorityQueue` | No | Unchanged. Deterministic ordering enables replay. | +| `createTask()` | No | Unchanged. Durable variant wraps it with context setup. | +| `spawn()`, `all()`, `race()` | No | Unchanged. Durable combinators delegate to them directly. | + +--- + +## 14. Progress and next steps + +### Completed (Tier 1-4 + HTTP backend) + +1. ~~Validate type system~~ — `Workflow` rejects `Operation` usage at + compile time (DEC-009, `types.test.ts`). +2. ~~Implement `createDurableEffect`~~ — Replay/live dispatch in `enter()` + (`effect.ts`). +3. ~~Implement `ReplayIndex`~~ — Spec-compliant, 21 tests + (`replay-index.ts`, `replay-index.test.ts`). +4. ~~Implement workflow effects~~ — `durableSleep`, `durableCall`, + `durableAction`, `versionCheck` (`operations.ts`). +5. ~~Implement `durableRun`~~ — Entry point with in-memory stream + (`run.ts`). +6. ~~Run Tier 1 tests~~ — Golden run, full replay, crash-at-N, + persist-before-resume, actor handoff — all passing + (`durable-run.test.ts`). +7. ~~Run Tier 2 tests~~ — All divergence detection cases passing + (`divergence.test.ts`). +8. ~~Implement `durableSpawn`, `durableAll`, `durableRace`~~ — Workflow + generators that self-wrap infrastructure in `ephemeral()` and delegate + to Effection's native spawn/all/race, with shared `runDurableChild` + helper. Child signatures tightened to `() => Workflow` (`combinators.ts`). +9. ~~Run Tier 3 tests~~ — Fork/join, nested scopes, race with + cancellation, error propagation, partial replay — all passing + (`structured-concurrency.test.ts`). +10. ~~Run Tier 4 tests~~ — Deterministic coroutine IDs across runs, + live vs replay, nested hierarchical IDs, race IDs — all passing + (`deterministic-id.test.ts`). +11. ~~Durable Streams backend adapter~~ — `HttpDurableStream` + (`http-stream.ts`) with raw fetch writes, promise chain + serialization, epoch fencing, offset tracking. 11 tests against + real server (`smoke.test.ts`). DEC-026 through DEC-029. + +### ~~Immediate: verify `readAll()` response shape~~ ✓ Complete + +Verified in a prior session. `stream()` from `@durable-streams/client@0.2.1` +returns `StreamResponseImpl` where `res.json()` returns a JSON array and +`res.offset` is a prototype getter for the `Stream-Next-Offset` header. +Both assumptions confirmed correct. See §12.3 for details. + +12. ~~Implement `durableEach`~~ — Durable iteration primitive with + Operation-native `DurableSource`, module-level state sharing, + `ephemeral()` wrapping. 10 tests passing (`each.ts`, + `durable-each.test.ts`). DEC-030. +13. ~~Implement `ephemeral()`~~ — Explicit escape hatch for non-durable + Operations inside Workflows. Transparent to the journal. 6 tests + passing (`ephemeral.ts`, `ephemeral.test.ts`). DEC-034. + +### Future improvements + +15. **Batch persistence (Strategy C).** Optimize concurrent child effects + by batching writes within a single reduce cycle. Becomes a performance + concern when `durableEach` feeds items into `durableAll` parallel + processing. See §12.4 and §15.1. + +16. **Continue-As-New.** Periodic journal compaction for long-running + `durableEach` loops. Bounds journal growth. See §15.2. + +17. **SSE/long-poll tailing.** `tail(offset)` method on `DurableStream` + for watching live events — needed for external workflow observers + and multi-worker coordination. See §12.3 on future interface + evolution. Additive, no changes to existing methods. + +--- + +## 15. Future architecture considerations + +### 15.1 Batch persistence (Strategy C) + +During `all()` with multiple children, several effects may resolve in the +same reducer tick (especially during replay-to-live transition). The spec's +Strategy C suggests batching writes — accumulating Yield events into a +buffer on the `DurableContext` and flushing at the end of each reduce cycle. + +The current implementation uses Strategy B (async append + deferred resolve) +on every individual effect. For the HTTP backend, this means each child's +Yield event is a separate HTTP POST, serialized by the promise chain +(DEC-027). With N concurrent children, that's N sequential round-trips. + +Strategy C would batch these into a single HTTP POST using the Durable +Streams `lingerMs`-style batching (or a single POST with multiple JSON +messages). The batch is one sequence number — atomic at the batch level. +This amortizes latency but requires changes to `createDurableEffect`: +instead of calling `stream.append()` directly, it would enqueue to a buffer +and the buffer would flush after the synchronous reduce cycle completes. + +The ordering constraint for batching: Close events must still be appended +strictly after the child's Yield events (causal ordering, spec §8). Within +a batch of sibling Yield events, ordering doesn't matter — they're from +independent coroutines. + +Not blocking for correctness. Only relevant for throughput with concurrent +children. + +### 15.2 Continue-As-New + +For `durableEach` loops processing unbounded streams, the journal grows +without limit. Continue-As-New is the standard solution from Temporal and +similar systems: after N iterations (or N bytes of journal), the runtime +terminates the current execution and starts a new one seeded with a +continuation token — the current cursor position plus any accumulated state. + +This requires: + +- **durableRun recognizing a continuation signal.** The workflow returns + a special value or throws a `ContinueAsNew` error that `durableRun` + catches. Instead of writing `Close(ok)`, it writes a continuation + marker and returns the seed state. +- **A scheduling layer.** Something outside `durableRun` that creates + a new stream, seeds the new execution, and links the executions for + observability. This might be a `DurableScheduler` or just a loop + around `durableRun`. +- **Stream lifecycle management.** Old streams can be archived or deleted + after the continuation starts. The Durable Streams protocol supports + TTL-based retention but no compaction — Continue-As-New is the + compaction strategy. + +Continue-As-New is a significant feature. It touches `durableRun` (the +continuation signal), the stream interface (creating new streams), and +potentially a new scheduler layer. Design it alongside `durableEach` +since they're tightly coupled — `durableEach` without Continue-As-New +is limited to bounded iteration counts. + +### 15.3 Uncancellable contexts for Close events + +`runDurableChild` appends Close events in a `finally` block via +`yield* call(() => stream.append(...))`. During parent scope teardown, +this async operation can be interrupted by Effection's cancellation +(DEC-028). The protocol handles this gracefully — a missing Close just +means re-execution on replay — but it's a correctness gap for +observability (the journal may not reflect the child's actual terminal +state). + +If Effection adds an uncancellable context in a future version (an +`uncancellable(() => ...)` wrapper that suppresses `iterator.return()` +during execution), `runDurableChild`'s finally block should use it. +This would guarantee Close events are always persisted, eliminating +the re-execution window. + +### 15.4 Stream naming conventions + +The HTTP adapter uses `${baseUrl}/${streamId}` as the URL. In a +multi-tenant or multi-workflow-type deployment, a naming convention +prevents collisions: + +- `workflows/${workflowType}/${executionId}` — per-execution stream + with type namespace +- `tenant/${tenantId}/workflows/${type}/${id}` — multi-tenant isolation + +The Durable Streams protocol uses URL-path-based naming with no built-in +namespacing. Tenant isolation requires path-prefix scoping or separate +server instances. Worth deciding before production deployment but not +blocking for development. + +### 15.5 Transient error retry for HTTP appends + +The current `HttpDurableStream` treats all unexpected HTTP statuses +(including 500, 503) as fatal. This is the safe choice when the +sequence state is uncertain — a failed append may or may not have +persisted on the server. + +A future version could add retry-with-same-seq logic for transient +errors: + +1. On 500/503/timeout, retry the same `(Id, Epoch, Seq)` tuple. +2. If the server returns 200, the retry succeeded (first write). +3. If the server returns 204, the original append did persist + (idempotent success). +4. Both outcomes are safe — the seq counter doesn't advance until + the append is confirmed. + +This requires distinguishing transient errors (retry-safe) from +permanent errors (StaleEpochError, SequenceGapError — fatal). The +current code's `fatalError` flag would need to become a discriminated +error state. diff --git a/durable-streams/specs/protocol-specification.md b/durable-streams/specs/protocol-specification.md new file mode 100644 index 00000000..8478a8c5 --- /dev/null +++ b/durable-streams/specs/protocol-specification.md @@ -0,0 +1,1046 @@ +# Two-Event Durable Execution Protocol for Generator-Based Structured Concurrency + +**Status:** Draft Specification +**Scope:** Effection-style generator runtime with append-only durable stream + +--- + +## 1. Overview + +This specification defines a durable execution protocol for generator-based +structured concurrency runtimes. The protocol records every observable effect +and its resolution to an append-only stream using exactly two event types. +On restart, the runtime replays stored resolutions into generators +deterministically, then transitions seamlessly to live execution for +any effects not yet in the stream. + +The protocol is designed for runtimes where: + +- Workflows are expressed as generator functions (`function*` / `yield*`). +- A synchronous reducer loop drives generators by calling `iterator.next(value)`, + `iterator.throw(error)`, and `iterator.return()`. +- Concurrency is tree-structured: every task has exactly one parent scope, + and child lifetimes are contained within parent lifetimes. + +### 1.1 Design principles + +**Generators are effect description machines.** A generator never performs side +effects directly. It yields descriptions of desired effects. The reducer +interprets descriptions during live execution and bypasses them during replay. +The generator cannot distinguish the two modes. + +**The stream is the single source of truth.** All durable state is encoded in +the append-only event stream. No external index, database, or in-memory +structure is authoritative — they are all derived from the stream. + +**Every stream entry is self-contained.** Each event carries all information +needed to interpret it. There are no cross-references between events, no +link-by-ID patterns, and no events that require a future event to become +meaningful. + +**Structure enables durability.** Tree-structured concurrency constrains the +execution model so that task lifetimes nest strictly. This makes the execution +trace a well-ordered tree that can be checkpointed, replayed, and cancelled +deterministically. Without structure, replay of concurrent tasks is +ambiguous. + +### 1.2 Non-goals + +This protocol does not address: + +- Unstructured `async`/`await` or callback-first orchestration. +- Reactive streams that do not model execution control flow. +- Best-effort or heuristic replay. +- Transport, encoding, or storage layer specifics (the stream is abstract). +- Distributed coordination between multiple workers (single-writer assumed). + +### 1.3 Terminology + +**Reducer.** The synchronous loop that drives generators. It dequeues +instructions from a priority queue, calls `iterator.next(value)` to +advance the generator, and processes the yielded effect. The reducer is +not a scheduler — it makes no decisions about which coroutine to run +next. The next instruction is always the next item in the queue, ordered +by scope depth (shallower scopes first — parent effects are entered +before child effects), FIFO within a tier. This determinism is what +makes replay possible. + +**Runtime.** The broader system that manages the lifecycle of scopes, +coroutines, and the durable stream. The runtime encompasses the reducer +but also handles concerns outside the reduce loop: scope creation and +destruction, cancellation propagation, stream persistence, and the +replay index. When this specification says "the runtime does X," it +means the implementation as a whole is responsible, not necessarily the +reduce loop specifically. + +**Coroutine.** A generator instance being driven by the reducer. Each +coroutine has a unique ID (§3), an iterator, and belongs to exactly one +scope. + +**Scope.** A structured concurrency boundary that owns zero or more +child coroutines. Scopes enforce lifetime containment (§7.1). + +**Stream.** The append-only sequence of durable events. The single +source of truth for replay. + +--- + +## 2. Event types + +The protocol defines exactly two event types. Every event in the stream is +one of these. + +```typescript +type DurableEvent = Yield | Close; +``` + +### 2.1 `Yield` — an effect was executed and resolved + +```typescript +interface Yield { + type: "yield"; + coroutineId: CoroutineId; + description: EffectDescription; + result: Result; +} +``` + +A `Yield` event is written **after** an effect resolves. It records both what +was requested (the description) and what the outcome was (the result). During +replay, the description is used for divergence detection and the result is +fed directly to the generator via `iterator.next(value)` or +`iterator.throw(error)`. + +A `Yield` event is never written for infrastructure effects (scope creation, +scope middleware, internal bookkeeping). Only user-facing effects — those +that represent observable interactions with the outside world — produce +`Yield` events. The classification of an effect as infrastructure vs. +user-facing is determined by the runtime, not by this protocol. + +### 2.2 `Close` — a coroutine reached a terminal state + +```typescript +interface Close { + type: "close"; + coroutineId: CoroutineId; + result: Result; +} +``` + +A `Close` event is written when a coroutine terminates. The three terminal +states are: + +- **Completed** (`status: "ok"`): the generator returned normally. +- **Failed** (`status: "err"`): an unhandled error propagated out of the generator. +- **Cancelled** (`status: "cancelled"`): the coroutine was halted by its parent + or by an external signal. + +`Close` events cannot be derived from `Yield` events because: + +- Cancellation produces no yield — the coroutine is halted externally. +- Return values are not captured by any yield (the generator's final + `return` statement does not yield). +- Unhandled errors that kill a coroutine may not correspond to any + specific effect resolution. + +`Close` events are **load-bearing for partial replay.** When the runtime +resumes from a partial stream after a crash, `Close` events tell the +runtime which scopes completed before the crash and which require +re-execution. Without `Close` events, the runtime would have to infer +completion from the absence of further yields — which is ambiguous (a +coroutine with no further yields might be completed, or might be +mid-execution at the crash point). + +### 2.3 Shared types + +```typescript +/** Dot-delimited hierarchical path. See §3. */ +type CoroutineId = string; + +/** + * Structured effect identity for divergence detection. + * See §6 for matching rules. + * + * Only `type` and `name` are compared during divergence detection. + * Extra fields beyond `type` and `name` are stored verbatim in the + * journal but never compared. They exist for runtime use (e.g., + * replay guards reading input parameters like file paths). + */ +interface EffectDescription { + /** Effect category. E.g., "call", "sleep", "action", "spawn", "resource". */ + type: string; + /** Stable name within the category. E.g., function name, resource label. */ + name: string; + /** Extra fields stored verbatim, never compared during divergence detection. */ + [key: string]: Json; +} + +type Result = + | { status: "ok"; value?: Json } + | { status: "err"; error: SerializedError } + | { status: "cancelled" }; + +/** Any JSON-serializable value. */ +type Json = string | number | boolean | null | Json[] | { [key: string]: Json }; + +interface SerializedError { + message: string; + name?: string; + stack?: string; +} +``` + +### 2.4 Why `EffectDescription` is structured, not a flat string + +A flat description string (e.g., `"action"`) conflates effect type and +identity. When multiple effects share the same description, divergence +detection cannot distinguish reordering from correct execution. + +For example, in a workflow that yields six `"action"` effects (three +keypresses followed by three queue reads), reordering a keypress and a +queue read produces a journal where every position still shows `"action"` — +divergence is not caught. + +The structured `{ type, name }` representation provides two levels of +discrimination: + +- **Type mismatch** (e.g., `"call"` vs. `"sleep"`) is always a hard + divergence error. +- **Name mismatch** (e.g., `call("fetchOrder")` vs. `call("chargeCard")`) + is a hard divergence error by default, with the option of a configurable + warning-only mode for specific name changes during migration. + +Effect arguments are intentionally excluded from the description. Argument +changes between versions are generally safe (the generator handles whatever +value it receives) and checking them produces false-positive divergence +errors during legitimate refactors. + +--- + +## 3. Coroutine identity + +### 3.1 Deterministic per-parent counter scheme + +Coroutine IDs are not recorded in the stream. They are assigned +deterministically at runtime using a per-parent creation counter: + +``` +root → "root" + first child of root → "root.0" + second child of root → "root.1" + first child of .1 → "root.1.0" + third child of root → "root.2" +``` + +The ID of a coroutine is the dot-concatenation of its ancestor chain, +where each segment is the zero-based creation index within the parent scope. + +### 3.2 Why this works + +The determinism of this scheme depends on three properties of the runtime: + +**Property 1: The reduce loop is synchronous.** The reducer processes +instructions in a synchronous `while` loop with a re-entrant guard. When +an effect resolves synchronously (as all replayed effects do), its +resolution is enqueued and processed in the same loop iteration. There +are no async gaps where scheduling non-determinism could alter ordering. + +**Property 2: Generators are deterministic.** A generator function given +the same arguments and the same sequence of values fed via `next()` produces +the same sequence of yields. Iteration constructs like `for...of` in `all()` +always process their operands in array order. Each `spawn` within such a +loop is processed in deterministic order within the synchronous reduce loop. + +**Property 3: Replay preserves ordering.** During replay, every user-facing +effect resolves synchronously (the stored result is fed back immediately). +This means the reduce loop processes spawns in the same order as the original +run. The per-parent counter increments in an identical sequence. + +**Property 4: Priority ordering is structural.** If the runtime uses a +priority queue ordered by scope depth (lower depth = higher priority), +priorities are determined by code structure (nesting depth), not by timing. +Shallower scopes are dequeued first — when a parent and child effect are +enqueued in the same tick, the parent is always entered first. Instructions +at the same priority are processed in FIFO order within their tier. + +### 3.3 Formal requirement + +> **INVARIANT (Deterministic Identity):** For any generator function `G` +> and any sequence of effect resolutions `R₁, R₂, ..., Rₙ`, the sequence +> of coroutine IDs assigned during execution of `G` with resolutions `R` is +> identical across all executions. Two runs of the same generator with the +> same resolution sequence must produce the same set of coroutine IDs in +> the same order. + +### 3.4 Spawn during teardown and `ensure()` + +When a scope is destroyed, the runtime runs teardown logic (`ensure()` +blocks, resource destructors) in reverse creation order. If teardown code +spawns new children, the per-parent counter continues incrementing +from where it left off. + +This is safe because teardown order is determined by the scope tree +structure (reverse creation order), which is itself deterministic +(see §3.2). The counter path is therefore identical across live and +replay runs, even for children spawned during teardown. + +> **REQUIREMENT:** Implementations MUST test the following scenario +> explicitly: an `ensure()` block inside a scope spawned by `all()` +> that itself spawns a child. The coroutine ID of the child spawned +> during teardown must be identical between live execution and replay. + +--- + +## 4. The replay loop + +### 4.1 Replay index + +The replay index is a derived, in-memory structure built from the stream +on startup. It provides per-coroutine cursored access to yield events +and keyed access to close events: + +```typescript +class ReplayIndex { + private yields = new Map>(); + private cursors = new Map(); + private closes = new Map(); + + constructor(events: DurableEvent[]) { + for (const event of events) { + if (event.type === "yield") { + const list = this.yields.get(event.coroutineId) ?? []; + list.push({ description: event.description, result: event.result }); + this.yields.set(event.coroutineId, list); + } + if (event.type === "close") { + this.closes.set(event.coroutineId, event); + } + } + } + + /** + * Returns the next unconsumed yield for this coroutine, + * or undefined if the cursor is past the end. + */ + peekYield(coroutineId: CoroutineId): { description: EffectDescription; result: Result } | undefined { + const list = this.yields.get(coroutineId); + const cursor = this.cursors.get(coroutineId) ?? 0; + return list?.[cursor]; + } + + /** Advances the cursor for this coroutine by one position. */ + consumeYield(coroutineId: CoroutineId): void { + const cursor = this.cursors.get(coroutineId) ?? 0; + this.cursors.set(coroutineId, cursor + 1); + } + + /** Returns true if a close event exists for this coroutine. */ + hasClose(coroutineId: CoroutineId): boolean { + return this.closes.has(coroutineId); + } + + /** Returns the close event for this coroutine, or undefined. */ + getClose(coroutineId: CoroutineId): Close | undefined { + return this.closes.get(coroutineId); + } + + /** + * Returns true if the cursor for this coroutine has been + * fully consumed AND a close event exists. This means the + * coroutine completed in a previous run and can be + * treated as fully replayed. + */ + isFullyReplayed(coroutineId: CoroutineId): boolean { + return this.peekYield(coroutineId) === undefined && this.hasClose(coroutineId); + } +} +``` + +### 4.2 Effect handling: replay vs. live + +When a generator yields a user-facing effect, the reducer determines +the execution mode for that specific effect based on the replay index +state for the yielding coroutine: + +``` +generator yields effect with description D + │ + ├─ entry = replayIndex.peekYield(coroutineId) + │ + ├─ if entry exists: + │ ├─ REPLAY PATH + │ ├─ Compare D against entry.description (see §6) + │ │ ├─ match → continue + │ │ └─ mismatch → raise DivergenceError (see §6.2) + │ ├─ replayIndex.consumeYield(coroutineId) + │ ├─ Feed entry.result to generator: + │ │ ├─ status "ok" → iterator.next(entry.result.value) + │ │ └─ status "err" → iterator.throw(deserialize(entry.result.error)) + │ └─ effect.enter() is NOT called + │ + └─ if no entry: + ├─ LIVE PATH + ├─ effect.enter(callback) + │ ... effect runs asynchronously or synchronously ... + │ effect resolves with result R + ├─ stream.append({ type: "yield", coroutineId, description: D, result: R }) + │ ↑ DURABLE WRITE — must complete before next step (see §5) + ├─ Feed R to generator: + │ ├─ status "ok" → iterator.next(R.value) + │ └─ status "err" → iterator.throw(deserialize(R.error)) + └─ continue +``` + +### 4.3 Replay-to-live transition + +The transition from replay to live execution occurs **per-coroutine** when +the replay index cursor for that coroutine exceeds the available entries. +There is no global mode switch. At any given moment, some coroutines may +be replaying while others are executing live. + +This per-coroutine transition is what enables partial replay of scope trees +that span crash boundaries (see §4.4). + +> **INVARIANT (Transparency):** The generator produces identical behavior +> whether driven by replay or live execution. The reducer is the only +> component that differs between modes. No mechanism exists for a generator +> to detect which mode is active. + +### 4.4 Partial replay of scope trees + +When the runtime resumes from a stream produced by a crashed previous run, +the scope tree may be partially complete: some children finished, others +did not. The runtime reconstructs the tree as follows: + +1. **Coroutines with `Close` events** are fully replayed. Their yields are + fed from the index, and their terminal state is read from the `Close` + event. The generator for a fully-closed coroutine may be instantiated + and driven through its replay to reconstruct any in-memory state the + parent needs from the child's return value, or the return value may be + read directly from the `Close` event's result if the parent accesses + it only through a join. + +2. **Coroutines with `Yield` events but no `Close` event** are partially + replayed. Their recorded yields are fed from the index (replay mode), + then execution continues live from the first unrecorded effect. The + generator transitions seamlessly at the per-coroutine boundary. + +3. **Coroutines with no events at all** were never reached in the previous + run (the crash occurred before they were spawned). They execute entirely + live. + +4. **The parent's join** waits for all children regardless of their replay + status. Children in cases 1 and 3 above may complete quickly (or + instantly, for case 1), while children in case 2 block on their first + live effect. The join does not distinguish replayed from live children. + +> **INVARIANT (Fork-Join Across Crash Boundaries):** A parent's join +> must produce the same result regardless of which children were fully +> replayed, partially replayed, or executed entirely live. The final +> result depends only on the children's results, not on how those +> results were obtained. + +--- + +## 5. The persist-before-resume invariant + +This section describes the single most critical correctness property of +the protocol. + +> **HARD INVARIANT (Persist-Before-Resume):** A `Yield` event recording +> the resolution of effect N MUST be durably persisted before +> `iterator.next(resultN)` or `iterator.throw(errorN)` is called to +> resume the generator. Violation of this invariant creates +> unrecoverable replay gaps. + +### 5.1 Why this is a hard invariant + +If the generator advances past a yield point whose resolution is not in +the stream, a crash at any subsequent point makes correct replay +impossible. The stream records effects 0 through N−1. On restart, the +reducer replays those N effects and then expects effect N from the +generator. But the generator, given the same resolutions for 0 through +N−1, will yield effect N — which the reducer executes live. This +re-executes effect N, potentially causing duplicate side effects +(double charges, duplicate messages, etc.). + +### 5.2 Interaction with the synchronous reduce loop + +The synchronous reduce loop creates a tension with async persistence. +The loop calls `iterator.next(value)` synchronously, but a durable write +to a remote log or disk is inherently asynchronous. + +Implementations MUST use one of the following strategies: + +**Strategy A: Synchronous local write.** Write the event to a local +write-ahead log (WAL) with `fsync` before resuming the generator. +The reduce loop blocks on the write. This is simple but adds latency +per effect. + +**Strategy B: Buffered write with deferred resume.** Instead of +resuming the generator synchronously after effect resolution, enqueue +the resume as a pending instruction. Perform the durable write +asynchronously. Only process the pending resume instruction after +durability is confirmed. This preserves the synchronous reduce loop's +structure while allowing async I/O. + +**Strategy C: Batch write at tick boundary.** When multiple effects +resolve in the same reducer tick (e.g., several children in a +`fork/join` completing simultaneously during replay), batch their +`Yield` events into a single durable write. Resume all generators +only after the batch is persisted. This amortizes write latency +across concurrent children. + +> **REQUIREMENT:** Implementations MUST document which persistence +> strategy they use and MUST include a test that verifies a crash +> between effect resolution and the next `iterator.next()` call does +> not advance the generator past a non-durable point (see §9, Test 6). + +--- + +## 6. Divergence detection + +### 6.1 Matching rules + +During replay, each yielded effect's `EffectDescription` is compared +against the corresponding journal entry's description. The comparison +uses only the `type` and `name` fields — extra fields on +`EffectDescription` beyond `type` and `name` are never compared during +divergence detection: + +| Yielded | Recorded | Result | +|---------|----------|--------| +| Same type, same name | — | **Match.** Consume entry, feed result. Extra fields are ignored. | +| Different type | — | **Hard divergence.** Always fatal. | +| Same type, different name | — | **Hard divergence** (default). Configurable to warning for specific migration scenarios. | + +> **INVARIANT (Divergence Detection):** During replay, every yielded +> effect MUST be validated against the corresponding journal entry +> before its stored result is fed to the generator. A mismatch in +> effect type is always a fatal `DivergenceError`. + +### 6.2 `DivergenceError` + +A `DivergenceError` is raised when the replay index entry at the current +cursor position does not match the effect yielded by the generator. The +error includes: + +```typescript +class DivergenceError extends Error { + coroutineId: CoroutineId; + position: number; // cursor position within the coroutine + expected: EffectDescription; // from the journal + actual: EffectDescription; // from the generator +} +``` + +A `DivergenceError` is **not recoverable**. The workflow cannot continue +because the generator's execution path has diverged from the recorded +history. The runtime MUST halt the workflow and surface the error to +the operator. + +### 6.3 Terminal divergence cases + +Beyond per-effect matching, the reducer detects two additional +divergence conditions: + +**Generator finishes early.** The generator returns `{ done: true }` +while the replay index still has unconsumed entries for this coroutine. +This means the current code produces fewer effects than the recorded +run — effects were removed without a version gate. + +**Journal exhausted with close but generator continues.** The replay +index has a `Close` event for this coroutine but the generator has not +finished after consuming all recorded yields. This means the current +code produces more effects than the recorded run — effects were added +without a version gate. + +Both cases raise `DivergenceError`. + +### 6.4 What is NOT checked + +- **Extra description fields.** Any fields on `EffectDescription` beyond + `type` and `name` are stored in the journal but never compared during + divergence detection. They exist for runtime use (e.g., staleness + validation by replay guards) and are safe to change between versions. +- **Effect arguments / inputs.** Changes to arguments between versions + are generally safe. The generator handles whatever value it receives + from the resolution. +- **Result values.** The stored result is fed to the generator as-is. + The generator's handling of the value is its own concern. +- **Timing.** Wall-clock time is not part of the protocol. + +--- + +## 7. Structured concurrency semantics + +### 7.1 Lifetime invariants + +> **INVARIANT (Lifetime Containment):** A child coroutine cannot outlive +> its parent scope. When a scope exits (by completion, error, or +> cancellation), all of its children MUST have terminated. + +> **INVARIANT (Single Parent):** Every coroutine has exactly one parent +> scope. The coroutine graph is always a tree, never a DAG. + +> **INVARIANT (Implicit Join):** A scope does not exit until all of its +> children have terminated. There is no mechanism to detach a child from +> its parent. + +### 7.2 Cancellation + +Cancellation propagates **downward** from parent to children, never +upward. + +When a scope is cancelled, the runtime cancels its children in **reverse +creation order** (last-created child first). This is a post-order traversal +of the subtree rooted at the cancelled scope: leaves are cancelled before +their parents. + +For each cancelled coroutine, the runtime calls `iterator.return()`. This +triggers `finally` blocks in the generator, allowing cleanup. If cleanup +code yields effects, those effects are executed and journaled normally +(in live mode) or replayed (if the cancellation itself is being replayed). + +``` +cancel(scope) + │ + ├─ for child in scope.children.reverse(): + │ cancel(child) // recurse, leaf-first + │ + ├─ step = scope.iterator.return(CANCELLED) + │ + ├─ while not step.done: // cleanup may yield effects + │ effect = step.value + │ result = handleEffect(effect) // replay or live, per §4.2 + │ step = scope.iterator.next(result) + │ + └─ stream.append({ type: "close", coroutineId, result: { status: "cancelled" } }) +``` + +> **INVARIANT (Cancellation Replay Fidelity):** Cancellation events in +> the journal MUST be replayed by calling `iterator.return()` at the +> same journal position, driving the generator through its stored +> cleanup path. The sequence of effects yielded during cleanup MUST +> match the recorded cleanup effects. + +### 7.3 Error propagation + +Unhandled errors in children propagate **upward** to the parent scope. +The parent's error-handling policy determines the response: + +- **Fail-fast (default):** On first child error, cancel all siblings, + then propagate the error to the parent's parent. +- **Fail-complete:** Collect results from all children. If any failed, + propagate an aggregate error after all children terminate. +- **Error boundary:** The parent catches the error via `try/catch` in + the generator. Siblings are not cancelled. Execution continues. + +The policy is determined by the concurrency combinator (`race` uses +fail-fast, `all` may use either, `spawn` within `try/catch` enables +error boundaries). The policy is not recorded in the journal — it is +a property of the code, which is deterministic. + +### 7.4 Deterministic shutdown ordering + +When a scope exits for any reason, cleanup proceeds in this order: + +1. Cancel all children in reverse creation order (§7.2). +2. Run the scope's own `ensure()` / `finally` blocks. +3. Write the scope's `Close` event to the stream. +4. Deliver the scope's result to the parent (via the parent's + pending join or the resolution of a `spawn` effect). + +This ordering is deterministic because creation order is deterministic +(§3.2) and reverse creation order is therefore also deterministic. + +--- + +## 8. Causal ordering + +> **INVARIANT (Causal Ordering):** Events in the stream MUST appear in +> an order consistent with causal dependency. If event A causally +> depends on event B (e.g., a parent's yield that consumes a child's +> return value depends on the child's `Close` event), then B MUST +> appear before A in the stream. + +The synchronous reduce loop guarantees this during live execution: +a parent cannot yield an effect that depends on a child's result until +the child has completed and its `Close` event has been appended. During +replay, the stream is read in append order, so causal ordering is +preserved by construction. + +This invariant is what makes the per-coroutine cursor model correct. +Each coroutine's cursor advances independently, but the cursors advance +in an order consistent with the global causal order — a parent's cursor +never advances past a point that depends on a child result that hasn't +been replayed yet. + +--- + +## 9. Version gates + +When workflow code changes in a way that alters the effect sequence +(adding, removing, or reordering yields), a **version gate** allows a +single codebase to handle both in-flight (old) and new workflow +instances. + +### 9.1 Mechanism + +A version gate is itself a yielded effect: + +```typescript +const version = yield* versionCheck("add-fraud-check", { minVersion: 0, maxVersion: 1 }); +``` + +On first execution (live mode), the reducer records a `Yield` event +with description `{ type: "version_gate", name: "add-fraud-check" }` +and result `{ status: "ok", value: 1 }` (the max version). + +On replay, the stored version determines which code path executes: + +```typescript +function* orderWorkflow(orderId: string) { + const version = yield* versionCheck("add-fraud-check", { minVersion: 0, maxVersion: 1 }); + + if (version >= 1) { + yield* call(fraudCheck, orderId); // new step, v1+ + } + + const order = yield* call(fetchOrder, orderId); + yield* call(chargeCard, order.payment); +} +``` + +### 9.2 Lifecycle + +Version gates follow a three-phase lifecycle: + +1. **Add gate.** Deploy code with both old (v0) and new (v1) paths. + In-flight workflows replay with v0. New workflows execute with v1. +2. **Deprecate old path.** Once all v0 workflows have completed, the + v0 code path is dead but retained for safety. +3. **Remove gate.** Once all v1 workflows from the deprecation era + have completed, remove the gate and the old code path entirely. + +### 9.3 Alternative: immutable deployments + +Instead of version gates, the runtime may support immutable deployments +where in-flight workflows always resume on the original code version. +In this model, version gates are unnecessary — the runtime routes each +workflow to the deployment endpoint that created it. + +This specification supports both approaches. The choice is an +operational concern, not a protocol concern — the stream format is +identical either way. + +--- + +## 10. Race semantics and cancellation ordering in the journal + +`race()` creates a scope with multiple children where the first child +to complete determines the result. Remaining children are cancelled. + +### 10.1 Journal structure for a race + +Consider `race([op1, op2])` where `op1` wins after `op2` has partially +executed: + +``` +[0] yield root.0.1 { type: "call", name: "step1" } result: ok ... // op2's first effect +[1] yield root.0.0 { type: "call", name: "fetch" } result: ok ... // op1 completes +[2] close root.0.0 result: ok // op1 done — op1 wins +[3] close root.0.1 result: cancelled // op2 cancelled +[4] close root.0 result: ok // race scope returns op1's result +``` + +The interleaving of events from `root.0.0` and `root.0.1` reflects +the actual execution order. The journal preserves this interleaving. + +### 10.2 Replay of races with partial children + +During replay, the reducer processes events in stream order. When it +encounters `close root.0.1 cancelled` in the journal, it knows to +call `iterator.return()` on `root.0.1`'s generator after replaying +that coroutine's recorded yields. + +The per-coroutine cursor model handles this correctly because: + +1. `root.0.1` has one yield entry (at position 0 of its cursor). +2. After replaying that yield, the cursor is exhausted. +3. The `Close` event with `status: "cancelled"` tells the reducer + to call `iterator.return()` rather than waiting for more yields. + +> **REQUIREMENT:** The runtime MUST use `Close` events with +> `status: "cancelled"` as the trigger for cancellation during replay. +> The cancellation point for a coroutine during replay is determined by +> the position of its `Close(cancelled)` event relative to its last +> `Yield` event — the coroutine is cancelled after its last recorded +> yield has been replayed. + +### 10.3 Interleaving and the per-coroutine cursor model + +The per-coroutine cursor model groups yields by coroutine ID, which +discards the original interleaving information from the flat stream. +This is acceptable because interleaving order between sibling coroutines +does not affect replay correctness — each coroutine's yield sequence is +independent, and the reducer feeds results from each coroutine's own +cursor. + +The only ordering that matters across coroutines is **causal** ordering +(§8), which is encoded in the `Close` events: a parent cannot proceed +past a join until all children have `Close` events. + +--- + +## 11. Stream format + +### 11.1 Logical structure + +The stream is an ordered, append-only sequence of `DurableEvent` values. +Each event occupies a unique position (offset) in the stream. Offsets +are monotonically increasing integers starting from 0. + +``` +offset 0: DurableEvent +offset 1: DurableEvent +offset 2: DurableEvent +... +``` + +### 11.2 Physical encoding + +This specification does not prescribe a physical encoding. Implementations +may use JSON, MessagePack, Protobuf, or any other format that can +faithfully represent the `DurableEvent` type. The encoding must preserve: + +- Event type (`"yield"` or `"close"`) +- Coroutine ID (string) +- Effect description (for `Yield` events): type and name fields +- Result (status, value, error as applicable) +- Append order (events must be readable in the order they were written) + +### 11.3 Stream consistency + +> **INVARIANT (Append-Only):** Events are only appended, never updated +> or deleted. + +> **INVARIANT (Prefix-Closed):** If event at offset N exists, events +> at offsets 0 through N−1 must also exist. There are no gaps. + +> **INVARIANT (Monotonic Indexing):** The event at offset N was the +> (N+1)th event written to the stream. + +### 11.4 Example: sequential workflow + +Workflow: +```typescript +function* pipeline() { + yield* sleep(2000); + const result = yield* call(transform, "alpha"); + return result; +} +``` + +Stream: +``` +[0] yield root.0 { type: "sleep", name: "sleep" } result: { status: "ok" } +[1] yield root.0 { type: "call", name: "transform" } result: { status: "ok", value: "ALPHA" } +[2] close root.0 result: { status: "ok", value: "ALPHA" } +[3] close root result: { status: "ok", value: "ALPHA" } +``` + +### 11.5 Example: fork/join with `all()` + +Workflow: +```typescript +function* parallel() { + const [a, b] = yield* all([ + call(fetchUser, "alice"), + call(fetchUser, "bob"), + ]); + yield* call(merge, a, b); + return "done"; +} +``` + +Stream: +``` +[0] yield root.0.0 { type: "call", name: "fetchUser" } result: { status: "ok", value: { name: "alice" } } +[1] yield root.0.1 { type: "call", name: "fetchUser" } result: { status: "ok", value: { name: "bob" } } +[2] close root.0.0 result: { status: "ok", value: { name: "alice" } } +[3] close root.0.1 result: { status: "ok", value: { name: "bob" } } +[4] yield root.0 { type: "call", name: "merge" } result: { status: "ok", value: "merged" } +[5] close root.0 result: { status: "ok", value: "done" } +[6] close root result: { status: "ok", value: "done" } +``` + +### 11.6 Example: race with cancellation + +Workflow: +```typescript +function* timeout() { + return yield* race([ + call(fetchData), + sleep(5000), + ]); +} +``` + +`fetchData` wins: +``` +[0] yield root.0.0 { type: "call", name: "fetchData" } result: { status: "ok", value: { data: 42 } } +[1] close root.0.0 result: { status: "ok", value: { data: 42 } } +[2] close root.0.1 result: { status: "cancelled" } +[3] close root.0 result: { status: "ok", value: { data: 42 } } +[4] close root result: { status: "ok", value: { data: 42 } } +``` + +Timeout wins (fetchData was slow, had partially executed): +``` +[0] yield root.0.0 { type: "call", name: "fetchData.step1" } result: { status: "ok", value: ... } +[1] yield root.0.1 { type: "sleep", name: "sleep" } result: { status: "ok" } +[2] close root.0.1 result: { status: "ok" } +[3] close root.0.0 result: { status: "cancelled" } +[4] close root.0 result: { status: "ok" } +[5] close root result: { status: "ok" } +``` + +--- + +## 12. Infrastructure effects + +Not all effects are recorded. The runtime distinguishes between +**user-facing effects** (which are durable) and **infrastructure effects** +(which are not). + +### 12.1 Classification + +**User-facing effects** interact with the outside world or represent +decisions that must be preserved across restarts. Examples: + +- `call(fn, ...args)` — invoke an external function +- `sleep(ms)` — wait for a duration +- `action(description)` — wait for an external event + +**Infrastructure effects** manage the runtime's internal structure. +Examples: + +- `useScope()` — obtain a reference to the current scope +- Scope middleware hooks +- Internal bookkeeping (counter management, queue operations) + +### 12.2 Rule + +Infrastructure effects are executed live during both replay and live +modes. They are never recorded in the stream and never appear in the +replay index. Their results are deterministic by construction (they +depend only on the runtime's internal state, which is reconstructed +identically during replay because the user-facing effects that shaped +it are replayed). + +> **REQUIREMENT:** The boundary between user-facing and infrastructure +> effects MUST be documented by the runtime implementation. An effect +> that is classified as infrastructure MUST NOT have observable external +> side effects and MUST produce the same result during replay as during +> live execution. + +--- + +## 13. Invariant summary + +For reference, the complete set of invariants defined in this specification: + +| # | Name | Section | Scope | +|---|------|---------|-------| +| 1 | Deterministic Identity | §3.3 | Coroutine IDs are stable across runs | +| 2 | Transparency | §4.3 | Generator cannot detect replay vs. live | +| 3 | Fork-Join Across Crash Boundaries | §4.4 | Join result is independent of replay status | +| 4 | Persist-Before-Resume | §5 | Durable write before generator advance | +| 5 | Divergence Detection | §6.1 | Every replayed effect is validated | +| 6 | Lifetime Containment | §7.1 | child ⊆ parent | +| 7 | Single Parent | §7.1 | Tree, not DAG | +| 8 | Implicit Join | §7.1 | Scope waits for all children | +| 9 | Cancellation Replay Fidelity | §7.2 | Cleanup path matches recorded path | +| 10 | Causal Ordering | §8 | Stream order respects causality | +| 11 | Append-Only | §11.3 | No mutation or deletion | +| 12 | Prefix-Closed | §11.3 | No gaps in the stream | +| 13 | Monotonic Indexing | §11.3 | Sequential offsets | + +--- + +## 14. Test plan + +### Tier 1 — Core replay correctness + +These tests MUST pass for the protocol to be considered implemented. + +| # | Test | Procedure | Verify | +|---|------|-----------|--------| +| 1 | **Golden run** | Execute workflow end-to-end with no interruption. | Stream contains expected events; final result is correct. | +| 2 | **Full replay** | Replay entire stream against same code. | No effects re-executed; no divergence; same result. | +| 3 | **Crash before first effect** | Provide empty stream to workflow. | All effects execute live; stream matches golden run. | +| 4 | **Crash at position N** | Provide first N events from golden stream. | First N effects replayed (not re-executed); remaining execute live; same result. | +| 5 | **Crash after last effect** | Provide all `Yield` events but no `Close` events. | All effects replayed; close events written; same result. | +| 6 | **Persist-before-resume verification** | Inject crash between effect resolution and `iterator.next()`. | On resume, the resolved effect is in the stream; no replay gap; no duplicate execution. | +| 7 | **Actor handoff** | Process A writes first N events, terminates. Process B reads stream, resumes. | B replays N events (none re-executed), continues live; correct result. | + +### Tier 2 — Divergence detection + +| # | Test | Procedure | Verify | +|---|------|-----------|--------| +| 8 | **Added step** | Record stream with code v1. Replay with v2 that adds an effect before existing ones. | `DivergenceError` at position 0 with expected vs. actual descriptions. | +| 9 | **Removed step** | Record with v1. Replay with v2 that removes an effect. | `DivergenceError` at the position where removed effect was expected. | +| 10 | **Reordered steps** | Record with v1. Replay with v2 that swaps two effects. | `DivergenceError` at first swapped position. | +| 11 | **Type mismatch** | Record a `call` effect. Replay code yields `sleep` at same position. | `DivergenceError` citing type mismatch. | +| 12 | **Name mismatch** | Record `call("fetchOrder")`. Replay yields `call("chargeCard")`. | `DivergenceError` citing name mismatch. | +| 13 | **Generator finishes early** | Record stream with 5 yields + close. Replay code produces only 3 yields then returns. | `DivergenceError`: generator completed with unconsumed journal entries. | +| 14 | **Generator continues past close** | Record stream with close after 3 yields. Replay code produces 5 yields. | `DivergenceError`: journal shows close but generator hasn't finished. | + +### Tier 3 — Structured concurrency + +| # | Test | Procedure | Verify | +|---|------|-----------|--------| +| 15 | **Fork/join — all children complete before crash** | Parent forks 3 children, all complete; crash before parent's post-join effect. | All child results replayed from stream; parent continues live. | +| 16 | **Fork/join — partial completion** | 2 of 3 children complete before crash. | 2 replayed, 1 re-executes from its last yield; correct join result. | +| 17 | **Nested scopes** | Crash inside doubly-nested scope. | Scope tree reconstructed; inner scope partially replayed; outer scope waits correctly. | +| 18 | **Cancellation propagation** | Cancel parent while children run. | Children cancelled in reverse order; `finally` blocks run; `Close(cancelled)` events written. | +| 19 | **Cancellation replay** | Replay a stream that contains cancellation events. | `iterator.return()` called at correct positions; cleanup effects replayed; no divergence. | +| 20 | **Error in child — sibling cancellation** | Child A throws; siblings cancelled. | Siblings cancelled in reverse order; error propagated to parent; cleanup recorded. | +| 21 | **Error boundary** | Child error caught by parent `try/catch`. | Parent catches error; siblings NOT cancelled; execution continues. | +| 22 | **Race — winner cancels losers** | `race([op1, op2])`, op1 wins. | op2 receives `Close(cancelled)`; race returns op1's result. | +| 23 | **Race replay with partial loser** | Replay race where loser partially executed. | Loser's partial yields replayed, then cancelled at correct point. | + +### Tier 4 — Deterministic identity + +| # | Test | Procedure | Verify | +|---|------|-----------|--------| +| 24 | **Stable IDs across runs** | Execute workflow twice with same inputs/resolutions. | Coroutine IDs identical in both runs. | +| 25 | **Stable IDs: live vs. replay** | Execute workflow live. Execute same workflow via full replay. | Coroutine IDs identical. | +| 26 | **Spawn during teardown** | `ensure()` block inside `all()` child spawns a new child. | Teardown child's coroutine ID identical between live and replay. | +| 27 | **Dynamic spawn count divergence** | Record stream with `all([a, b])`. Replay with `all([a, b, c])`. | `DivergenceError` (c produces yields not in journal, or parent's post-join effect mismatches). | + +### Tier 5 — Versioning + +| # | Test | Procedure | Verify | +|---|------|-----------|--------| +| 28 | **Version gate — old workflow** | Stream from v0 code. Replay with code containing version gate. | Gate reads v0; old path executes; no divergence. | +| 29 | **Version gate — new workflow** | Execute with version gate, no prior stream. | Gate records v1; new path executes; stream contains version marker. | + +### Tier 6 — Edge cases + +| # | Test | Procedure | Verify | +|---|------|-----------|--------| +| 30 | **Empty workflow** | Workflow with zero yields. | Completes immediately; stream has only `Close` events; replay is no-op. | +| 31 | **Effect that throws** | Effect resolves with error; generator catches. | Stream records error result; replay injects via `iterator.throw()`; catch path executes. | +| 32 | **Truncated stream** | Remove last N events from valid stream. | Runtime detects incomplete state; re-executes from truncation point; no silent corruption. | +| 33 | **Systematic crash-point sweep** | For workflow with M yields, run M+1 crash-resume cycles. | Every crash point produces correct final result; replayed effects not re-executed. | + +### Tier 7 — Property-based tests + +| # | Property | Strategy | +|---|----------|----------| +| 34 | **Crash-resume equivalence** | For any workflow and any set of crash points, the final result equals the uninterrupted result. | +| 35 | **Journal monotonicity** | After any operation sequence, stream length is non-decreasing and offsets are sequential. | +| 36 | **Replay idempotency** | Replaying the same stream K times produces the same effect sequence every time. | +| 37 | **Random workflow fuzzing** | Generate random workflow shapes (varying depth, fork count, error/cancel injection). Verify all invariants hold. | diff --git a/durable-streams/stream.ts b/durable-streams/stream.ts new file mode 100644 index 00000000..0ada9003 --- /dev/null +++ b/durable-streams/stream.ts @@ -0,0 +1,90 @@ +/** + * DurableStream interface and in-memory implementation. + * + * The interface is intentionally abstract — protocol-specification.md §11 + * does not prescribe a physical encoding or transport. + */ + +import type { Operation } from "effection"; +import type { DurableEvent } from "./types.ts"; + +function cloneEvent(event: DurableEvent): DurableEvent { + return structuredClone(event); +} + +function cloneEvents(events: DurableEvent[]): DurableEvent[] { + return events.map(cloneEvent); +} + +/** + * Abstract interface for the append-only durable event stream. + * + * Implementations must guarantee: + * - Append-only (events are never updated or deleted) + * - Prefix-closed (no gaps) + * - Monotonic indexing (sequential offsets) + * - Durability (once append resolves, the event persists) + */ +export interface DurableStream { + /** Read all events in the stream, in append order. */ + readAll(): Operation; + + /** + * Append an event to the stream. + * The returned operation completes only after the event is durably persisted. + */ + append(event: DurableEvent): Operation; +} + +/** + * In-memory DurableStream implementation for testing. + * + * Provides optional hooks for: + * - Tracking append calls (to verify no re-execution during replay) + * - Injecting failures (for persist-before-resume testing) + */ +export class InMemoryStream implements DurableStream { + private events: DurableEvent[] = []; + + /** Count of append calls, useful for verifying replay doesn't re-execute. */ + appendCount = 0; + + /** If set, append() will reject with this error. */ + injectFailure: Error | null = null; + + /** Optional callback invoked on each append, before persistence. */ + onAppend: ((event: DurableEvent) => void) | null = null; + + constructor(initialEvents: DurableEvent[] = []) { + this.events = cloneEvents(initialEvents); + } + + // deno-lint-ignore require-yield + *readAll(): Operation { + return cloneEvents(this.events); + } + + // deno-lint-ignore require-yield + *append(event: DurableEvent): Operation { + if (this.injectFailure) { + throw this.injectFailure; + } + const cloned = cloneEvent(event); + this.onAppend?.(cloneEvent(cloned)); + this.events.push(cloned); + this.appendCount++; + } + + /** Get a snapshot of current events (for test assertions). */ + snapshot(): DurableEvent[] { + return cloneEvents(this.events); + } + + /** Reset the stream (for test setup). */ + reset(events: DurableEvent[] = []): void { + this.events = cloneEvents(events); + this.appendCount = 0; + this.injectFailure = null; + this.onAppend = null; + } +} diff --git a/durable-streams/structured-concurrency.test.ts b/durable-streams/structured-concurrency.test.ts new file mode 100644 index 00000000..18e9d078 --- /dev/null +++ b/durable-streams/structured-concurrency.test.ts @@ -0,0 +1,656 @@ +/** + * Tier 3 tests — structured concurrency. + * + * Tests 15-23 from the protocol specification. These validate that + * durableAll, durableRace, and durableSpawn correctly handle child + * scope lifecycles, Close events, cancellation, and replay. + */ + +import { describe, it } from "@effectionx/bdd"; +import { expect } from "expect"; +import { + InMemoryStream, + type Json, + type Workflow, + durableAll, + durableCall, + durableRace, + durableRun, +} from "./mod.ts"; + +// --------------------------------------------------------------------------- +// Helpers +// --------------------------------------------------------------------------- + +/** Track which functions were actually called during live execution. */ +function createCallTracker() { + const calls: string[] = []; + return { + calls, + fn(name: string, value: T): () => Promise { + return () => { + calls.push(name); + return Promise.resolve(value); + }; + }, + }; +} + +describe("structured concurrency", () => { + // --------------------------------------------------------------------------- + // Test 15: Fork/join — all children complete (golden run) + // --------------------------------------------------------------------------- + + it("all: golden run — all children execute live", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + const result = yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("fetchA", tracker.fn("fetchA", "alpha")); + }, + function* () { + return yield* durableCall("fetchB", tracker.fn("fetchB", "beta")); + }, + ]); + return `${results[0]}-${results[1]}`; + }, + { stream }, + ); + + expect(result).toBe("alpha-beta"); + expect(tracker.calls.sort()).toEqual(["fetchA", "fetchB"]); + + // Verify stream structure: child yields, child Closes, root Close + const events = stream.snapshot(); + + const yieldEvents = events.filter((e) => e.type === "yield"); + const closeEvents = events.filter((e) => e.type === "close"); + + expect(yieldEvents.length).toBe(2); + expect(closeEvents.length).toBe(3); // root.0, root.1, root + + // Child coroutine IDs should be root.0 and root.1 + const childCloses = closeEvents.filter((e) => e.coroutineId !== "root"); + const childIds = childCloses.map((e) => e.coroutineId).sort(); + expect(childIds).toEqual(["root.0", "root.1"]); + + // Root Close should be last + expect(closeEvents[closeEvents.length - 1]!.coroutineId).toBe("root"); + }); + + // --------------------------------------------------------------------------- + // Test 15b: Fork/join — full replay + // --------------------------------------------------------------------------- + + it("all: full replay — returns stored result without re-executing", function* () { + // First: golden run + const stream = new InMemoryStream(); + const tracker1 = createCallTracker(); + + yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("fetchA", tracker1.fn("fetchA", "alpha")); + }, + function* () { + return yield* durableCall("fetchB", tracker1.fn("fetchB", "beta")); + }, + ]); + return `${results[0]}-${results[1]}`; + }, + { stream }, + ); + + // Second: replay with the same stream + const tracker2 = createCallTracker(); + const replayStream = new InMemoryStream(stream.snapshot()); + + const result = yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("fetchA", tracker2.fn("fetchA", "WRONG")); + }, + function* () { + return yield* durableCall("fetchB", tracker2.fn("fetchB", "WRONG")); + }, + ]); + return `${results[0]}-${results[1]}`; + }, + { stream: replayStream }, + ); + + // Result from stored Close event + expect(result).toBe("alpha-beta"); + + // No effects re-executed + expect(tracker2.calls).toEqual([]); + }); + + // --------------------------------------------------------------------------- + // Test 16: Fork/join — partial completion (crash after some children) + // --------------------------------------------------------------------------- + + it("all: partial replay — completed children replayed, incomplete re-execute", function* () { + // Golden run to capture full stream + const goldenStream = new InMemoryStream(); + const goldenTracker = createCallTracker(); + + yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall( + "fetchA", + goldenTracker.fn("fetchA", "alpha"), + ); + }, + function* () { + return yield* durableCall( + "fetchB", + goldenTracker.fn("fetchB", "beta"), + ); + }, + function* () { + return yield* durableCall( + "fetchC", + goldenTracker.fn("fetchC", "gamma"), + ); + }, + ]); + return results.join("-"); + }, + { stream: goldenStream }, + ); + + // Simulate crash: keep events for root.0 and root.1 only + // (child 2's events and root Close are dropped) + const allEvents = goldenStream.snapshot(); + const partialEvents = allEvents.filter((e) => { + if (e.coroutineId === "root") return false; + if (e.coroutineId === "root.2") return false; + return true; + }); + + const partialStream = new InMemoryStream(partialEvents); + const tracker = createCallTracker(); + + const result = yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("fetchA", tracker.fn("fetchA", "WRONG")); + }, + function* () { + return yield* durableCall("fetchB", tracker.fn("fetchB", "WRONG")); + }, + function* () { + return yield* durableCall("fetchC", tracker.fn("fetchC", "gamma")); + }, + ]); + return results.join("-"); + }, + { stream: partialStream }, + ); + + // Children 0,1 replayed from stored Close. Child 2 executed live. + expect(result).toBe("alpha-beta-gamma"); + + // Only fetchC was actually called + expect(tracker.calls).toEqual(["fetchC"]); + }); + + // --------------------------------------------------------------------------- + // Test 17: Nested scopes — inner all inside outer all + // --------------------------------------------------------------------------- + + it("all: nested — inner all inside outer all", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + const result = yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + // Child 0: has its own nested all + const inner = yield* durableAll([ + function* () { + return yield* durableCall("innerA", tracker.fn("innerA", "a")); + }, + function* () { + return yield* durableCall("innerB", tracker.fn("innerB", "b")); + }, + ]); + return inner.join("+") as string; + }, + function* () { + return yield* durableCall("outerB", tracker.fn("outerB", "B")); + }, + ]); + return results.join("-"); + }, + { stream }, + ); + + expect(result).toBe("a+b-B"); + expect(tracker.calls.sort()).toEqual(["innerA", "innerB", "outerB"]); + + // Verify nested coroutine IDs + const events = stream.snapshot(); + const coroutineIds = [...new Set(events.map((e) => e.coroutineId))].sort(); + expect(coroutineIds).toEqual([ + "root", + "root.0", + "root.0.0", + "root.0.1", + "root.1", + ]); + }); + + // --------------------------------------------------------------------------- + // Test 18: Race — first to complete wins, others cancelled + // --------------------------------------------------------------------------- + + it("race: golden run — first to complete wins", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + const result = yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + return yield* durableCall("fast", tracker.fn("fast", "winner")); + }, + function* () { + yield* durableCall( + "slow-step1", + tracker.fn("slow-step1", "partial"), + ); + return yield* durableCall( + "slow-step2", + tracker.fn("slow-step2", "would-not-reach"), + ); + }, + ]); + }, + { stream }, + ); + + expect(result).toBe("winner"); + + // Verify Close events for the winner + const events = stream.snapshot(); + const closeEvents = events.filter((e) => e.type === "close"); + + const winnerClose = closeEvents.find((e) => e.coroutineId === "root.0"); + expect(winnerClose !== undefined).toBe(true); + if (winnerClose?.type === "close") { + expect(winnerClose.result.status).toBe("ok"); + } + }); + + // --------------------------------------------------------------------------- + // Test 19: Race full replay + // --------------------------------------------------------------------------- + + it("race: full replay — returns stored result without re-executing", function* () { + const stream = new InMemoryStream(); + const tracker1 = createCallTracker(); + + const result1 = yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + return yield* durableCall("fast", tracker1.fn("fast", "winner")); + }, + function* () { + yield* durableCall( + "slow-step1", + tracker1.fn("slow-step1", "partial"), + ); + return yield* durableCall( + "slow-step2", + tracker1.fn("slow-step2", "loser"), + ); + }, + ]); + }, + { stream }, + ); + + // Replay + const tracker2 = createCallTracker(); + const replayStream = new InMemoryStream(stream.snapshot()); + + const result2 = yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + return yield* durableCall("fast", tracker2.fn("fast", "WRONG")); + }, + function* () { + yield* durableCall( + "slow-step1", + tracker2.fn("slow-step1", "WRONG"), + ); + return yield* durableCall( + "slow-step2", + tracker2.fn("slow-step2", "WRONG"), + ); + }, + ]); + }, + { stream: replayStream }, + ); + + expect(result2).toBe(result1); + expect(tracker2.calls).toEqual([]); + }); + + // --------------------------------------------------------------------------- + // Test 20: Error in child — siblings cancelled, error propagated + // --------------------------------------------------------------------------- + + it("all: child error — siblings cancelled, error propagated", function* () { + const stream = new InMemoryStream(); + + try { + yield* durableRun( + function* () { + const results = yield* durableAll([ + function* () { + return yield* durableCall("good", () => + Promise.resolve("ok"), + ); + }, + function* () { + yield* durableCall("failStep", () => + Promise.reject(new Error("child-boom")), + ); + return "unreachable"; + }, + ]); + return results.join("-"); + }, + { stream }, + ); + throw new Error("expected error"); + } catch (e) { + expect(e).toBeInstanceOf(Error); + expect((e as Error).message).toContain("child-boom"); + } + + // The stream should contain Close(err) for the failing child + const events = stream.snapshot(); + const errCloses = events.filter( + (e) => e.type === "close" && e.result.status === "err", + ); + expect(errCloses.length >= 1).toBe(true); + }); + + // --------------------------------------------------------------------------- + // Test 21: Error boundary — parent catches child error + // --------------------------------------------------------------------------- + + it("all: error boundary — parent catches child error", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + const result = yield* durableRun( + function* () { + try { + yield* durableAll([ + function* () { + return yield* durableCall( + "good", + tracker.fn("good", "ok"), + ); + }, + function* (): Workflow { + yield* durableCall("failStep", () => + Promise.reject(new Error("child-caught")), + ); + return "unreachable"; + }, + ]); + } catch { + // Error caught — continue + } + const recovery = yield* durableCall( + "recovery", + tracker.fn("recovery", "recovered"), + ); + return recovery; + }, + { stream }, + ); + + expect(result).toBe("recovered"); + expect(tracker.calls.includes("recovery")).toBe(true); + }); + + // --------------------------------------------------------------------------- + // Test 22: Race — winner Close(ok), loser gets Close event + // --------------------------------------------------------------------------- + + it("race: winner Close(ok), loser gets Close(cancelled)", function* () { + const stream = new InMemoryStream(); + + const result = yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + return yield* durableCall("instant", () => Promise.resolve("won")); + }, + function* () { + return yield* durableCall( + "never", + () => + new Promise(() => { + /* never resolves */ + }), + ); + }, + ]); + }, + { stream }, + ); + + expect(result).toBe("won"); + + const events = stream.snapshot(); + const closeEvents = events.filter((e) => e.type === "close"); + + // Winner Close(ok) + const winnerClose = closeEvents.find((e) => e.coroutineId === "root.0"); + expect(winnerClose !== undefined).toBe(true); + if (winnerClose?.type === "close") { + expect(winnerClose.result.status).toBe("ok"); + } + + // Loser Close(cancelled) — Effection cancels losers, runDurableChild + // detects this via the undefined closeEvent path in finally. + const loserClose = closeEvents.find((e) => e.coroutineId === "root.1"); + expect(loserClose !== undefined).toBe(true); + if (loserClose?.type === "close") { + expect(loserClose.result.status).toBe("cancelled"); + } + + // Causal ordering: child Closes before root Close + const rootCloseIdx = closeEvents.findIndex((e) => e.coroutineId === "root"); + const childCloseIdxs = closeEvents + .map((e, i) => (e.coroutineId !== "root" ? i : -1)) + .filter((i) => i >= 0); + for (const childIdx of childCloseIdxs) { + expect(childIdx < rootCloseIdx).toBe(true); + } + }); + + // --------------------------------------------------------------------------- + // Test 23: Race replay — full replay (all events including root Close) + // --------------------------------------------------------------------------- + + it("race: full replay — returns stored result without re-executing", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + const result1 = yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + const a = yield* durableCall("winA", tracker.fn("winA", "a")); + return yield* durableCall("winB", tracker.fn("winB", `${a}-b`)); + }, + function* () { + yield* durableCall("loseA", tracker.fn("loseA", "x")); + return yield* durableCall("loseB", tracker.fn("loseB", "y")); + }, + ]); + }, + { stream }, + ); + + // Replay with complete journal (root Close present) + const replayStream = new InMemoryStream(stream.snapshot()); + const tracker2 = createCallTracker(); + + const result2 = yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + yield* durableCall("winA", tracker2.fn("winA", "WRONG")); + return yield* durableCall("winB", tracker2.fn("winB", "WRONG")); + }, + function* () { + yield* durableCall("loseA", tracker2.fn("loseA", "WRONG")); + return yield* durableCall("loseB", tracker2.fn("loseB", "WRONG")); + }, + ]); + }, + { stream: replayStream }, + ); + + expect(result2).toBe(result1); + expect(tracker2.calls).toEqual([]); + }); + + // --------------------------------------------------------------------------- + // Test 23b: Race partial replay — root Close stripped, cancelled losers replayed + // --------------------------------------------------------------------------- + + it("race: partial replay — cancelled loser replays via suspend", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + return yield* durableCall("fast", tracker.fn("fast", "winner")); + }, + function* () { + yield* durableCall("slowStep", tracker.fn("slowStep", "partial")); + return yield* durableCall( + "slowStep2", + tracker.fn("slowStep2", "never"), + ); + }, + ]); + }, + { stream }, + ); + + // Simulate crash: strip root Close but keep everything else + // (winner Close(ok), loser Close(cancelled), winner yield, loser yield) + const allEvents = stream.snapshot(); + const partialEvents = allEvents.filter((e) => e.coroutineId !== "root"); + + const partialStream = new InMemoryStream(partialEvents); + const tracker2 = createCallTracker(); + + // Replay: winner replays from Close(ok), loser sees Close(cancelled) + // and suspends (blocks until parent race cancels it naturally). + const result = yield* durableRun( + function* () { + return yield* durableRace([ + function* () { + return yield* durableCall("fast", tracker2.fn("fast", "WRONG")); + }, + function* () { + yield* durableCall("slowStep", tracker2.fn("slowStep", "WRONG")); + return yield* durableCall( + "slowStep2", + tracker2.fn("slowStep2", "WRONG"), + ); + }, + ]); + }, + { stream: partialStream }, + ); + + // Winner's result replayed from journal + expect(result).toBe("winner"); + // No effects re-executed — winner replayed from Close, loser suspended + expect(tracker2.calls).toEqual([]); + }); + + // --------------------------------------------------------------------------- + // Mixed: durableCall then durableAll + // --------------------------------------------------------------------------- + + it("mixed: durableCall then durableAll", function* () { + const stream = new InMemoryStream(); + const tracker = createCallTracker(); + + const result = yield* durableRun( + function* () { + const prefix = yield* durableCall( + "prefix", + tracker.fn("prefix", "PRE"), + ); + const results = yield* durableAll([ + function* () { + return yield* durableCall("A", tracker.fn("A", "a")); + }, + function* () { + return yield* durableCall("B", tracker.fn("B", "b")); + }, + ]); + return `${prefix}-${results.join(",")}`; + }, + { stream }, + ); + + expect(result).toBe("PRE-a,b"); + + // Replay + const replayStream = new InMemoryStream(stream.snapshot()); + const tracker2 = createCallTracker(); + + const result2 = yield* durableRun( + function* () { + const prefix = yield* durableCall( + "prefix", + tracker2.fn("prefix", "WRONG"), + ); + const results = yield* durableAll([ + function* () { + return yield* durableCall("A", tracker2.fn("A", "WRONG")); + }, + function* () { + return yield* durableCall("B", tracker2.fn("B", "WRONG")); + }, + ]); + return `${prefix}-${results.join(",")}`; + }, + { stream: replayStream }, + ); + + expect(result2).toBe("PRE-a,b"); + expect(tracker2.calls).toEqual([]); + }); +}); diff --git a/durable-streams/tsconfig.json b/durable-streams/tsconfig.json new file mode 100644 index 00000000..20877bcf --- /dev/null +++ b/durable-streams/tsconfig.json @@ -0,0 +1,14 @@ +{ + "extends": "../tsconfig.json", + "compilerOptions": { + "outDir": "dist", + "rootDir": "." + }, + "include": ["**/*.ts"], + "exclude": ["**/*.test.ts", "dist"], + "references": [ + { + "path": "../bdd" + } + ] +} diff --git a/durable-streams/types.test.ts b/durable-streams/types.test.ts new file mode 100644 index 00000000..5e863584 --- /dev/null +++ b/durable-streams/types.test.ts @@ -0,0 +1,108 @@ +/** + * Type-level tests for Workflow and DurableEffect. + * + * These tests verify that TypeScript correctly enforces the constraint + * that only DurableEffect values can be yielded inside a Workflow generator. + * + * Tests are runtime no-ops — they only validate at compile time. + */ + +import { describe, it } from "@effectionx/bdd"; +import type { Operation } from "effection"; +import { expect } from "expect"; +import type { + DurableEffect, + EffectionResult, + Resolve, + Workflow, +} from "./types.ts"; + +// --------------------------------------------------------------------------- +// Helper: create a minimal DurableEffect for testing +// --------------------------------------------------------------------------- + +function testDurableEffect(value: T): DurableEffect { + return { + description: "test", + effectDescription: { type: "test", name: "test" }, + enter(resolve: Resolve>) { + resolve({ ok: true, value }); + return (exit: Resolve>) => { + exit({ ok: true, value: undefined }); + }; + }, + }; +} + +// A Workflow-compatible operation that yields a DurableEffect +function durableOp(): Workflow { + return (function* () { + return (yield testDurableEffect(42)) as number; + })(); +} + +// Another Workflow-compatible operation +function anotherDurableOp(): Workflow { + return (function* () { + return (yield testDurableEffect("hello")) as string; + })(); +} + +describe("Workflow types", () => { + // --------------------------------------------------------------------------- + // POSITIVE: These should compile + // --------------------------------------------------------------------------- + + it("Workflow can yield DurableEffect values", function* () { + // This function compiles — DurableEffect is accepted as yield type + function* _myWorkflow(): Workflow { + const x = (yield testDurableEffect(42)) as number; + return x; + } + expect(true).toBe(true); // runtime no-op + }); + + it("Workflow can yield* to another Workflow", function* () { + // yield* delegation between Workflows should work + function* _myWorkflow(): Workflow { + const n: number = yield* durableOp(); + const s: string = yield* anotherDurableOp(); + return `${n}-${s}`; + } + expect(true).toBe(true); + }); + + it("Workflow is assignable to Operation", function* () { + // DurableEffect extends Effect structurally, so Workflow generators + // produce iterators compatible with Operation's expected iterator type. + function* _myWorkflow(): Workflow { + return (yield testDurableEffect(42)) as number; + } + + // This assignment should compile: Workflow → Operation + const _op: Operation = { + [Symbol.iterator]: _myWorkflow, + }; + expect(true).toBe(true); + }); +}); + +// --------------------------------------------------------------------------- +// NEGATIVE: These SHOULD NOT compile (commented out with expected errors) +// --------------------------------------------------------------------------- + +// Uncomment any of these to verify they produce type errors: + +// import { sleep, useScope } from "effection"; +// +// function* _badWorkflow1(): Workflow { +// yield* sleep(1000); +// // ^ Type error: Iterator, void, unknown> is not +// // assignable to Iterator, ...> +// } +// +// function* _badWorkflow2(): Workflow { +// yield* useScope(); +// // ^ Type error: same reason — useScope returns Operation, +// // which yields Effect, not DurableEffect +// } diff --git a/durable-streams/types.ts b/durable-streams/types.ts new file mode 100644 index 00000000..438aeff8 --- /dev/null +++ b/durable-streams/types.ts @@ -0,0 +1,155 @@ +/** + * Protocol types for the two-event durable execution protocol. + * + * Protocol types (Json, Result, DurableEvent, etc.) are the fixed contract + * defined by protocol-specification.md and do not depend on Effection. + * + * Effection integration types (CoroutineView, DurableEffect, Workflow) are + * in the second section and bridge the protocol with Effection's runtime. + */ + +import type { Result as EffectionResult, Resolve, Scope } from "effection"; + +/** Any JSON-serializable value. */ +export type Json = + | string + | number + | boolean + | null + | Json[] + | { [key: string]: Json }; + +// biome-ignore lint/suspicious/noConfusingVoidType: Workflows may intentionally return no value. +export type WorkflowValue = Json | void; + +/** Serialized error for durable storage. */ +export interface SerializedError { + message: string; + name?: string; + stack?: string; +} + +/** Result of an effect or coroutine. */ +export type Result = + | { status: "ok"; value?: Json } + | { status: "err"; error: SerializedError } + | { status: "cancelled" }; + +/** Dot-delimited hierarchical coroutine path. See spec §3. */ +export type CoroutineId = string; + +/** + * Structured effect identity for divergence detection. + * See spec §6 for matching rules. + * + * Only `type` and `name` are compared during divergence detection. + * Extra fields beyond `type` and `name` are stored verbatim in the + * journal but never compared. They exist for runtime use (e.g., + * replay guards reading input parameters like file paths). + */ +export interface EffectDescription { + /** Effect category. E.g., "call", "sleep", "action", "spawn", "resource". */ + type: string; + /** Stable name within the category. E.g., function name, resource label. */ + name: string; + /** Extra fields stored verbatim, never compared during divergence detection. */ + [key: string]: Json; +} + +/** + * A Yield event — an effect was executed and resolved. + * Written after an effect resolves. Records both what was requested + * (description) and what the outcome was (result). See spec §2.1. + * + * Replay guards access `description.*` for input fields (e.g., file path) + * and `result.value.*` for output fields (e.g., content hash). There is + * no separate metadata field — inputs belong in the effect description, + * outputs belong in the result. + */ +export interface Yield { + type: "yield"; + coroutineId: CoroutineId; + description: EffectDescription; + result: Result; +} + +/** + * A Close event — a coroutine reached a terminal state. + * Written when a coroutine terminates (completed, failed, or cancelled). + * See spec §2.2. + */ +export interface Close { + type: "close"; + coroutineId: CoroutineId; + result: Result; +} + +/** The two event types that make up the durable stream. */ +export type DurableEvent = Yield | Close; + +// --------------------------------------------------------------------------- +// Effection integration types +// +// EffectionResult and Resolve are re-exported from Effection. +// EffectionResult is an alias for Effection's Result, renamed to +// avoid collision with our protocol's Result type. +// --------------------------------------------------------------------------- + +/** + * Effection's internal Result type, re-exported under a distinct name to + * avoid collision with the protocol's Result type. + * + * Effection uses { ok: true, value: T } | { ok: false, error: Error }. + * The protocol uses { status: "ok" | "err" | "cancelled" }. + */ +export type { EffectionResult, Resolve }; + +/** + * View of Effection's Coroutine — the fields we need from enter(). + * + * The full Coroutine type is internal to Effection (@ignore), but + * enter() receives it. We need `scope` to read DurableContext and + * to invoke the Divergence API via Api.invoke(scope, ...). + */ +export interface CoroutineView { + scope: Scope; +} + +/** + * A DurableEffect extends Effection's Effect interface with a structured + * `effectDescription` for divergence detection and replay. + * + * The `enter()` signature matches Effection's Effect exactly: + * enter(resolve: Resolve>, routine: Coroutine): + * (resolve: Resolve>) => void + * + * DurableEffect is structurally assignable to Effect because it has + * the same shape plus the extra `effectDescription` field. We use + * CoroutineView (a narrower type than Coroutine) so that contravariance + * keeps the assignment valid while documenting our minimal dependency. + */ +export interface DurableEffect { + /** Human-readable description (for Effection's Effect interface). */ + description: string; + /** Structured description for divergence detection (spec §6). */ + effectDescription: EffectDescription; + /** Enter the effect — handles replay/live dispatch internally. */ + enter( + resolve: Resolve>, + routine: CoroutineView, + ): (resolve: Resolve>) => void; +} + +/** + * A Workflow is a generator that only yields DurableEffect values. + * + * Every Workflow is structurally compatible with Operation because + * DurableEffect extends Effect (it has all required fields). + * TypeScript's covariant yield type means Generator + * is assignable to Iterator. + * + * Uses Generator (not Iterable) so TypeScript enforces the yield type + * at compile time — yielding a plain Effect inside a Workflow generator + * is a type error. + */ +export type Workflow = Generator, T, unknown>; diff --git a/effect-ts/effect-runtime.ts b/effect-ts/effect-runtime.ts index 03dd28a2..d14fe64b 100644 --- a/effect-ts/effect-runtime.ts +++ b/effect-ts/effect-runtime.ts @@ -1,5 +1,5 @@ import { type Effect, type Exit, Layer, ManagedRuntime } from "effect"; -import { type Operation, action, call, resource } from "effection"; +import { type Operation, action, resource, until } from "effection"; /** * A runtime for executing Effect programs inside Effection operations. @@ -103,15 +103,57 @@ export function makeEffectRuntime( layer ?? Layer.empty, ) as ManagedRuntime.ManagedRuntime; + interface PendingExecution { + abort: () => void; + settled: Promise; + } + + const pending = new Set(); + + function startManaged(runPromise: (signal: AbortSignal) => Promise) { + const controller = new AbortController(); + let done = false; + + const execution = { + abort: () => { + if (!done) { + controller.abort(); + } + }, + settled: Promise.resolve(), + } as PendingExecution; + + const promise = runPromise(controller.signal); + + execution.settled = promise + .then( + () => undefined, + () => undefined, + ) + .finally(() => { + done = true; + pending.delete(execution); + }); + + pending.add(execution); + + return { promise, abort: execution.abort, signal: controller.signal }; + } + const run: EffectRuntime["run"] = ( effect: Effect.Effect, ) => { return action((resolve, reject) => { - const controller = new AbortController(); - managedRuntime - .runPromise(effect, { signal: controller.signal }) - .then(resolve, reject); - return () => controller.abort(); + const { promise, abort, signal } = startManaged((signal) => + managedRuntime.runPromise(effect, { signal }), + ); + + promise.then(resolve, (error) => { + if (!signal.aborted) { + reject(error); + } + }); + return abort; }); }; @@ -119,18 +161,29 @@ export function makeEffectRuntime( effect: Effect.Effect, ) => { return action>((resolve, reject) => { - const controller = new AbortController(); - managedRuntime - .runPromiseExit(effect, { signal: controller.signal }) - .then(resolve, reject); - return () => controller.abort(); + const { promise, abort, signal } = startManaged((signal) => + managedRuntime.runPromiseExit(effect, { signal }), + ); + + promise.then(resolve, (error) => { + if (!signal.aborted) { + reject(error); + } + }); + return abort; }); }; try { yield* provide({ run, runExit }); } finally { - yield* call(() => managedRuntime.dispose()); + const active = Array.from(pending); + for (const execution of active) { + execution.abort(); + } + + yield* until(Promise.all(active.map((execution) => execution.settled))); + yield* until(managedRuntime.dispose()); } }); } diff --git a/effect-ts/package.json b/effect-ts/package.json index 7a461579..6c2aae52 100644 --- a/effect-ts/package.json +++ b/effect-ts/package.json @@ -1,7 +1,7 @@ { "name": "@effectionx/effect-ts", "description": "Bidirectional interop between Effect-TS and Effection", - "version": "0.1.2", + "version": "0.1.3", "keywords": ["effection", "effectionx", "interop", "effect-ts", "effect"], "type": "module", "main": "./dist/mod.js", diff --git a/package.json b/package.json index 69f5361e..f8e75a22 100644 --- a/package.json +++ b/package.json @@ -30,6 +30,9 @@ "peerDependencyRules": { "ignoreMissing": [], "allowAny": [] + }, + "overrides": { + "effection": "4.1.0-alpha.7" } }, "volta": { diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 16e6d53e..90340eb0 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -4,6 +4,9 @@ settings: autoInstallPeers: true excludeLinksFromLockfile: false +overrides: + effection: 4.1.0-alpha.7 + importers: .: @@ -18,8 +21,8 @@ importers: specifier: ^7 version: 7.7.1 effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 expect: specifier: ^29 version: 29.7.0 @@ -48,8 +51,8 @@ importers: specifier: workspace:* version: link:../tinyexec effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 generatorics: specifier: ^1 version: 1.1.0 @@ -70,8 +73,8 @@ importers: version: link:../test-adapter devDependencies: effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 chain: devDependencies: @@ -79,8 +82,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 context-api: devDependencies: @@ -88,8 +91,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 converge: dependencies: @@ -101,8 +104,30 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 + + durable-streams: + dependencies: + '@durable-streams/client': + specifier: ^0.2.1 + version: 0.2.1 + devDependencies: + '@durable-streams/server': + specifier: ^0.2.1 + version: 0.2.1(@tanstack/db@0.5.30(typescript@5.9.3)) + '@effectionx/bdd': + specifier: workspace:* + version: link:../bdd + '@effectionx/fs': + specifier: workspace:* + version: link:../fs + effection: + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 + expect: + specifier: ^29 + version: 29.7.0 effect-ts: devDependencies: @@ -113,8 +138,8 @@ importers: specifier: ^3 version: 3.19.14 effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 fetch: devDependencies: @@ -122,8 +147,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 fs: devDependencies: @@ -131,8 +156,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 fx: devDependencies: @@ -140,8 +165,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 jsonl-store: devDependencies: @@ -149,8 +174,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 node: devDependencies: @@ -158,8 +183,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 process: dependencies: @@ -186,8 +211,8 @@ importers: specifier: ^6 version: 6.0.6 effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 raf: devDependencies: @@ -198,8 +223,8 @@ importers: specifier: ^1 version: 1.2.0 effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 scope-eval: devDependencies: @@ -207,8 +232,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 signals: dependencies: @@ -220,8 +245,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 stream-helpers: dependencies: @@ -242,8 +267,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 stream-yaml: dependencies: @@ -258,8 +283,8 @@ importers: specifier: workspace:* version: link:../stream-helpers effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 task-buffer: devDependencies: @@ -273,14 +298,14 @@ importers: specifier: ^8 version: 8.1.5 effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 test-adapter: devDependencies: effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 timebox: devDependencies: @@ -288,8 +313,8 @@ importers: specifier: workspace:* version: link:../bdd effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 tinyexec: dependencies: @@ -298,8 +323,8 @@ importers: version: 0.3.2 devDependencies: effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 vitest: dependencies: @@ -314,8 +339,8 @@ importers: specifier: workspace:* version: link:../process effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 vitest: specifier: ^3 version: 3.2.4(@types/node@22.19.7)(yaml@2.8.2) @@ -354,8 +379,8 @@ importers: specifier: workspace:* version: link:../stream-helpers effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 websocket: devDependencies: @@ -366,8 +391,8 @@ importers: specifier: ^8 version: 8.18.1 effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 ws: specifier: ^8 version: 8.19.0 @@ -391,8 +416,8 @@ importers: specifier: workspace:* version: link:../converge effection: - specifier: ^4 - version: 4.0.0 + specifier: 4.1.0-alpha.7 + version: 4.1.0-alpha.7 packages: @@ -457,6 +482,20 @@ packages: cpu: [x64] os: [win32] + '@durable-streams/client@0.2.1': + resolution: {integrity: sha512-+mGdK6TuDR9fJPo8jw6DufPfoUv6g+27xoPES76GXQc6y3val9Oe/SK2o2FV9sqqLSE19HEUSxTp0D6CZebfZw==} + engines: {node: '>=18.0.0'} + + '@durable-streams/server@0.2.1': + resolution: {integrity: sha512-34Ss6CJIJJk0i8Leo9hhNqfbiAAfthSBWiYb13wYu4Wn72vJHorrCKH5Qr0stzZR7jp+fjdbZEnim4lExxEOjg==} + engines: {node: '>=18.0.0'} + + '@durable-streams/state@0.2.1': + resolution: {integrity: sha512-8piCcw/y1Jz6QJn0vOf+yhovFlp3xvRbzWRWtF51ndlorofJi3pOfCXnEINuvSueuNMIJosmKOg0AEvpMKfoCw==} + engines: {node: '>=18.0.0'} + peerDependencies: + '@tanstack/db': '>=0.5.0' + '@esbuild/aix-ppc64@0.27.2': resolution: {integrity: sha512-GZMB+a0mOMZs4MpDbj8RJp4cw+w1WV5NYD6xzgvzUJ5Ek2jerwfO2eADyI6ExDSUED+1X8aMbegahsJi+8mgpw==} engines: {node: '>=18'} @@ -616,6 +655,9 @@ packages: '@essentials/raf@1.2.0': resolution: {integrity: sha512-AWJvpprE2o7ATMb7HBYMVUVmPJBCt2wZp2rY7d+rAcNSMvzLbDepy9KFeqqrPZh+s9aIpbw1LgmuAW7kuRFgrQ==} + '@harperfast/extended-iterable@1.0.3': + resolution: {integrity: sha512-sSAYhQca3rDWtQUHSAPeO7axFIUJOI6hn1gjRC5APVE1a90tuyT8f5WIgRsFhhWA7htNkju2veB9eWL6YHi/Lw==} + '@jest/expect-utils@29.7.0': resolution: {integrity: sha512-GlsNBWiFQFCVi9QVSx7f5AgMeLxe9YCCs5PuP2O2LdjDAA8Jh9eX7lA1Jq/xdXw3Wb3hyvlFNfZIfcRetSzYcA==} engines: {node: ^14.15.0 || ^16.10.0 || >=18.0.0} @@ -631,6 +673,78 @@ packages: '@jridgewell/sourcemap-codec@1.5.5': resolution: {integrity: sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==} + '@lmdb/lmdb-darwin-arm64@3.5.1': + resolution: {integrity: sha512-tpfN4kKrrMpQ+If1l8bhmoNkECJi0iOu6AEdrTJvWVC+32sLxTARX5Rsu579mPImRP9YFWfWgeRQ5oav7zApQQ==} + cpu: [arm64] + os: [darwin] + + '@lmdb/lmdb-darwin-x64@3.5.1': + resolution: {integrity: sha512-+a2tTfc3rmWhLAolFUWRgJtpSuu+Fw/yjn4rF406NMxhfjbMuiOUTDRvRlMFV+DzyjkwnokisskHbCWkS3Ly5w==} + cpu: [x64] + os: [darwin] + + '@lmdb/lmdb-linux-arm64@3.5.1': + resolution: {integrity: sha512-aoERa5B6ywXdyFeYGQ1gbQpkMkDbEo45qVoXE5QpIRavqjnyPwjOulMkmkypkmsbJ5z4Wi0TBztON8agCTG0Vg==} + cpu: [arm64] + os: [linux] + + '@lmdb/lmdb-linux-arm@3.5.1': + resolution: {integrity: sha512-0EgcE6reYr8InjD7V37EgXcYrloqpxVPINy3ig1MwDSbl6LF/vXTYRH9OE1Ti1D8YZnB35ZH9aTcdfSb5lql2A==} + cpu: [arm] + os: [linux] + + '@lmdb/lmdb-linux-x64@3.5.1': + resolution: {integrity: sha512-SqNDY1+vpji7bh0sFH5wlWyFTOzjbDOl0/kB5RLLYDAFyd/uw3n7wyrmas3rYPpAW7z18lMOi1yKlTPv967E3g==} + cpu: [x64] + os: [linux] + + '@lmdb/lmdb-win32-arm64@3.5.1': + resolution: {integrity: sha512-50v0O1Lt37cwrmR9vWZK5hRW0Aw+KEmxJJ75fge/zIYdvNKB/0bSMSVR5Uc2OV9JhosIUyklOmrEvavwNJ8D6w==} + cpu: [arm64] + os: [win32] + + '@lmdb/lmdb-win32-x64@3.5.1': + resolution: {integrity: sha512-qwosvPyl+zpUlp3gRb7UcJ3H8S28XHCzkv0Y0EgQToXjQP91ZD67EHSCDmaLjtKhe+GVIW5om1KUpzVLA0l6pg==} + cpu: [x64] + os: [win32] + + '@microsoft/fetch-event-source@2.0.1': + resolution: {integrity: sha512-W6CLUJ2eBMw3Rec70qrsEW0jOm/3twwJv21mrmj2yORiaVmVYGS4sSS5yUwvQc1ZlDLYGPnClVWmUUMagKNsfA==} + + '@msgpackr-extract/msgpackr-extract-darwin-arm64@3.0.3': + resolution: {integrity: sha512-QZHtlVgbAdy2zAqNA9Gu1UpIuI8Xvsd1v8ic6B2pZmeFnFcMWiPLfWXh7TVw4eGEZ/C9TH281KwhVoeQUKbyjw==} + cpu: [arm64] + os: [darwin] + + '@msgpackr-extract/msgpackr-extract-darwin-x64@3.0.3': + resolution: {integrity: sha512-mdzd3AVzYKuUmiWOQ8GNhl64/IoFGol569zNRdkLReh6LRLHOXxU4U8eq0JwaD8iFHdVGqSy4IjFL4reoWCDFw==} + cpu: [x64] + os: [darwin] + + '@msgpackr-extract/msgpackr-extract-linux-arm64@3.0.3': + resolution: {integrity: sha512-YxQL+ax0XqBJDZiKimS2XQaf+2wDGVa1enVRGzEvLLVFeqa5kx2bWbtcSXgsxjQB7nRqqIGFIcLteF/sHeVtQg==} + cpu: [arm64] + os: [linux] + + '@msgpackr-extract/msgpackr-extract-linux-arm@3.0.3': + resolution: {integrity: sha512-fg0uy/dG/nZEXfYilKoRe7yALaNmHoYeIoJuJ7KJ+YyU2bvY8vPv27f7UKhGRpY6euFYqEVhxCFZgAUNQBM3nw==} + cpu: [arm] + os: [linux] + + '@msgpackr-extract/msgpackr-extract-linux-x64@3.0.3': + resolution: {integrity: sha512-cvwNfbP07pKUfq1uH+S6KJ7dT9K8WOE4ZiAcsrSes+UY55E/0jLYc+vq+DO7jlmqRb5zAggExKm0H7O/CBaesg==} + cpu: [x64] + os: [linux] + + '@msgpackr-extract/msgpackr-extract-win32-x64@3.0.3': + resolution: {integrity: sha512-x0fWaQtYp4E6sktbsdAqnehxDgEc/VwM7uLsRCYWaiGu0ykYdZPiS8zCWdnjHwyiumousxfBm4SO31eXqwEZhQ==} + cpu: [x64] + os: [win32] + + '@neophi/sieve-cache@1.5.0': + resolution: {integrity: sha512-9T3nD5q51X1d4QYW6vouKW9hBSb2Tb/wB/2XoTr4oP5SCGtp3a7aTHHewQFylred1B21/Bhev6gy4x01FPBcbQ==} + engines: {node: '>=18'} + '@rollup/rollup-android-arm-eabi@4.55.1': resolution: {integrity: sha512-9R0DM/ykwfGIlNu6+2U09ga0WXeZ9MRC2Ter8jnz8415VbuIykVuc6bhdrbORFZANDmTDvq26mJrEVTl8TdnDg==} cpu: [arm] @@ -768,6 +882,20 @@ packages: '@standard-schema/spec@1.1.0': resolution: {integrity: sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==} + '@tanstack/db-ivm@0.1.17': + resolution: {integrity: sha512-DK7vm56CDxNuRAdsbiPs+gITJ+16tUtYgZg3BRTLYKGIDsy8sdIO7sQFq5zl7Y+aIKAPmMAbVp9UjJ75FTtwgQ==} + peerDependencies: + typescript: '>=4.7' + + '@tanstack/db@0.5.30': + resolution: {integrity: sha512-kgBbmGrWSrDdKPzl465OEGy2Epyc6HsoI2U/jUrMSHuGgUEtgowpnBCTyPJCGBrHL8+bYgASFV4ULsFn4L6pLg==} + peerDependencies: + typescript: '>=4.7' + + '@tanstack/pacer-lite@0.2.1': + resolution: {integrity: sha512-3PouiFjR4B6x1c969/Pl4ZIJleof1M0n6fNX8NRiC9Sqv1g06CVDlEaXUR4212ycGFyfq4q+t8Gi37Xy+z34iQ==} + engines: {node: '>=18'} + '@types/chai@5.2.3': resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==} @@ -906,6 +1034,10 @@ packages: resolution: {integrity: sha512-h5k/5U50IJJFpzfL6nO9jaaumfjO/f2NjK/oYB2Djzm4p9L+3T9qWpZqZ2hAbLPuuYq9wrU08WQyBTL5GbPk5Q==} engines: {node: '>=6'} + detect-libc@2.1.2: + resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==} + engines: {node: '>=8'} + diff-sequences@29.6.3: resolution: {integrity: sha512-EjePK1srD3P08o2j4f0ExnylqRs5B9tJjcp9t1krH2qRi8CCdsYfwe9JgSLurFBWwq4uOlipzfk5fHNvwFKr8Q==} engines: {node: ^14.15.0 || ^16.10.0 || >=18.0.0} @@ -913,8 +1045,8 @@ packages: effect@3.19.14: resolution: {integrity: sha512-3vwdq0zlvQOxXzXNKRIPKTqZNMyGCdaFUBfMPqpsyzZDre67kgC1EEHDV4EoQTovJ4w5fmJW756f86kkuz7WFA==} - effection@4.0.0: - resolution: {integrity: sha512-eW2yqhyBdey4k8lkp7hpiev2FSHvJvQqvaIebI3EGikHZvfUWvNy7SmkwOnJa6WcsUtSh7VHUwdjHTbV++8M9w==} + effection@4.1.0-alpha.7: + resolution: {integrity: sha512-ir7qM7vQXOo7eiuNxaD6FgwUetdy3XMB3dwtzUxp9ytiZl6abZUrXwM3vMqKWdMW4cW1KZzJqSEuXofJFscwFw==} engines: {node: '>= 16'} es-module-lexer@1.7.0: @@ -944,6 +1076,9 @@ packages: resolution: {integrity: sha512-h5+1OzzfCC3Ef7VbtKdcv7zsstUQwUDlYpUTvjeUsJAssPgLn7QzbboPtL5ro04Mq0rPOsMzl7q5hIbRs2wD1A==} engines: {node: '>=8.0.0'} + fastq@1.20.1: + resolution: {integrity: sha512-GGToxJ/w1x32s/D2EKND7kTil4n8OVk/9mycTc4VDza13lOvpUZTGX3mFSCtV9ksdGBVzvsyAVLM6mHFThxXxw==} + fdir@6.5.0: resolution: {integrity: sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==} engines: {node: '>=12.0.0'} @@ -957,6 +1092,10 @@ packages: resolution: {integrity: sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg==} engines: {node: '>=8'} + fractional-indexing@3.2.0: + resolution: {integrity: sha512-PcOxmqwYCW7O2ovKRU8OoQQj2yqTfEB/yeTYk4gPid6dN5ODRfU1hXd9tTVZzax/0NkO7AxpHykvZnT1aYp/BQ==} + engines: {node: ^14.13.1 || >=16.0.0} + fsevents@2.3.3: resolution: {integrity: sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==} engines: {node: ^8.16.0 || ^10.6.0 || >=11.0.0} @@ -1013,6 +1152,10 @@ packages: js-tokens@9.0.1: resolution: {integrity: sha512-mxa9E9ITFOt0ban3j6L5MpjwegGz6lBQmM1IJkWeBZGcMxto50+eWdjC/52xDbS2vy0k7vIMK0Fe2wfL9OQSpQ==} + lmdb@3.5.1: + resolution: {integrity: sha512-NYHA0MRPjvNX+vSw8Xxg6FLKxzAG+e7Pt8RqAQA/EehzHVXq9SxDqJIN3JL1hK0dweb884y8kIh6rkWvPyg9Wg==} + hasBin: true + loupe@3.2.1: resolution: {integrity: sha512-CdzqowRJCeLU72bHvWqwRBBlLcMEtIvGrlvef74kMnV2AolS9Y8xUv1I0U/MNAWMhBlKIoyuEgoJ0t/bbwHbLQ==} @@ -1026,11 +1169,28 @@ packages: ms@2.1.3: resolution: {integrity: sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==} + msgpackr-extract@3.0.3: + resolution: {integrity: sha512-P0efT1C9jIdVRefqjzOQ9Xml57zpOXnIuS+csaB4MdZbTdmGDLo8XhzBG1N7aO11gKDDkJvBLULeFTo46wwreA==} + hasBin: true + + msgpackr@1.11.8: + resolution: {integrity: sha512-bC4UGzHhVvgDNS7kn9tV8fAucIYUBuGojcaLiz7v+P63Lmtm0Xeji8B/8tYKddALXxJLpwIeBmUN3u64C4YkRA==} + nanoid@3.3.11: resolution: {integrity: sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==} engines: {node: ^10 || ^12 || ^13.7 || ^14 || >=15.0.1} hasBin: true + node-addon-api@6.1.0: + resolution: {integrity: sha512-+eawOlIgy680F0kBzPUNFhMZGtJ1YmqM6l4+Crf4IkImjYrO/mqPwRMh352g23uIaQKFItcQ64I7KMaJxHgAVA==} + + node-gyp-build-optional-packages@5.2.2: + resolution: {integrity: sha512-s+w+rBWnpTMwSFbaE0UXsRlg7hU4FjekKU4eyAih5T8nJuNZT1nNsskXpxmeqSK9UzkBl6UgRlnKc8hz8IEqOw==} + hasBin: true + + ordered-binary@1.6.1: + resolution: {integrity: sha512-QkCdPooczexPLiXIrbVOPYkR3VO3T6v2OyKRkR1Xbhpy7/LAVXwahnRCgRp78Oe/Ehf0C/HATAxfSr6eA1oX+w==} + path-key@3.1.1: resolution: {integrity: sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q==} engines: {node: '>=8'} @@ -1074,6 +1234,10 @@ packages: remeda@2.33.4: resolution: {integrity: sha512-ygHswjlc/opg2VrtiYvUOPLjxjtdKvjGz1/plDhkG66hjNjFr1xmfrs2ClNFo/E6TyUFiwYNh53bKV26oBoMGQ==} + reusify@1.1.0: + resolution: {integrity: sha512-g6QUff04oZpHs0eG5p83rFLhHeV00ug/Yf9nZM6fLeUrPguBTkTQOdpAWWspMh55TZfVQDPaN3NQJfbVRAxdIw==} + engines: {iojs: '>=1.0.0', node: '>=0.10.0'} + rollup@4.55.1: resolution: {integrity: sha512-wDv/Ht1BNHB4upNbK74s9usvl7hObDnvVzknxqY/E/O3X6rW1U1rV1aENEfJ54eFZDTNo7zv1f5N4edCluH7+A==} engines: {node: '>=18.0.0', npm: '>=8.0.0'} @@ -1102,6 +1266,9 @@ packages: resolution: {integrity: sha512-g9Q1haeby36OSStwb4ntCGGGaKsaVSjQ68fBxoQcutl5fS1vuY18H3wSt3jFyFtrkx+Kz0V1G85A4MyAdDMi2Q==} engines: {node: '>=8'} + sorted-btree@1.8.1: + resolution: {integrity: sha512-395+XIP+wqNn3USkFSrNz7G3Ss/MXlZEqesxvzCRFwL14h6e8LukDHdLBePn5pwbm5OQ9vGu8mDyz2lLDIqamQ==} + source-map-js@1.2.1: resolution: {integrity: sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==} engines: {node: '>=0.10.0'} @@ -1268,6 +1435,9 @@ packages: jsdom: optional: true + weak-lru-cache@1.2.2: + resolution: {integrity: sha512-DEAoo25RfSYMuTGc9vPJzZcZullwIqRDSI9LOy+fkCJPi6hykCnfKaXTuPBDuXAUcqHXyOgFtHNp/kB2FjYHbw==} + web-worker@1.5.0: resolution: {integrity: sha512-RiMReJrTAiA+mBjGONMnjVDP2u3p9R1vkcGz6gDIrOMT3oGuYwX2WRMYI9ipkphSuE5XKEhydbhNEJh4NY9mlw==} @@ -1350,6 +1520,26 @@ snapshots: '@biomejs/cli-win32-x64@1.9.4': optional: true + '@durable-streams/client@0.2.1': + dependencies: + '@microsoft/fetch-event-source': 2.0.1 + fastq: 1.20.1 + + '@durable-streams/server@0.2.1(@tanstack/db@0.5.30(typescript@5.9.3))': + dependencies: + '@durable-streams/client': 0.2.1 + '@durable-streams/state': 0.2.1(@tanstack/db@0.5.30(typescript@5.9.3)) + '@neophi/sieve-cache': 1.5.0 + lmdb: 3.5.1 + transitivePeerDependencies: + - '@tanstack/db' + + '@durable-streams/state@0.2.1(@tanstack/db@0.5.30(typescript@5.9.3))': + dependencies: + '@durable-streams/client': 0.2.1 + '@standard-schema/spec': 1.1.0 + '@tanstack/db': 0.5.30(typescript@5.9.3) + '@esbuild/aix-ppc64@0.27.2': optional: true @@ -1430,6 +1620,8 @@ snapshots: '@essentials/raf@1.2.0': {} + '@harperfast/extended-iterable@1.0.3': {} + '@jest/expect-utils@29.7.0': dependencies: jest-get-type: 29.6.3 @@ -1449,6 +1641,49 @@ snapshots: '@jridgewell/sourcemap-codec@1.5.5': {} + '@lmdb/lmdb-darwin-arm64@3.5.1': + optional: true + + '@lmdb/lmdb-darwin-x64@3.5.1': + optional: true + + '@lmdb/lmdb-linux-arm64@3.5.1': + optional: true + + '@lmdb/lmdb-linux-arm@3.5.1': + optional: true + + '@lmdb/lmdb-linux-x64@3.5.1': + optional: true + + '@lmdb/lmdb-win32-arm64@3.5.1': + optional: true + + '@lmdb/lmdb-win32-x64@3.5.1': + optional: true + + '@microsoft/fetch-event-source@2.0.1': {} + + '@msgpackr-extract/msgpackr-extract-darwin-arm64@3.0.3': + optional: true + + '@msgpackr-extract/msgpackr-extract-darwin-x64@3.0.3': + optional: true + + '@msgpackr-extract/msgpackr-extract-linux-arm64@3.0.3': + optional: true + + '@msgpackr-extract/msgpackr-extract-linux-arm@3.0.3': + optional: true + + '@msgpackr-extract/msgpackr-extract-linux-x64@3.0.3': + optional: true + + '@msgpackr-extract/msgpackr-extract-win32-x64@3.0.3': + optional: true + + '@neophi/sieve-cache@1.5.0': {} + '@rollup/rollup-android-arm-eabi@4.55.1': optional: true @@ -1536,6 +1771,21 @@ snapshots: '@standard-schema/spec@1.1.0': {} + '@tanstack/db-ivm@0.1.17(typescript@5.9.3)': + dependencies: + fractional-indexing: 3.2.0 + sorted-btree: 1.8.1 + typescript: 5.9.3 + + '@tanstack/db@0.5.30(typescript@5.9.3)': + dependencies: + '@standard-schema/spec': 1.1.0 + '@tanstack/db-ivm': 0.1.17(typescript@5.9.3) + '@tanstack/pacer-lite': 0.2.1 + typescript: 5.9.3 + + '@tanstack/pacer-lite@0.2.1': {} + '@types/chai@5.2.3': dependencies: '@types/deep-eql': 4.0.2 @@ -1676,6 +1926,8 @@ snapshots: deep-eql@5.0.2: {} + detect-libc@2.1.2: {} + diff-sequences@29.6.3: {} effect@3.19.14: @@ -1683,7 +1935,7 @@ snapshots: '@standard-schema/spec': 1.1.0 fast-check: 3.23.2 - effection@4.0.0: {} + effection@4.1.0-alpha.7: {} es-module-lexer@1.7.0: {} @@ -1736,6 +1988,10 @@ snapshots: dependencies: pure-rand: 6.1.0 + fastq@1.20.1: + dependencies: + reusify: 1.1.0 + fdir@6.5.0(picomatch@4.0.3): optionalDependencies: picomatch: 4.0.3 @@ -1744,6 +2000,8 @@ snapshots: dependencies: to-regex-range: 5.0.1 + fractional-indexing@3.2.0: {} + fsevents@2.3.3: optional: true @@ -1802,6 +2060,23 @@ snapshots: js-tokens@9.0.1: {} + lmdb@3.5.1: + dependencies: + '@harperfast/extended-iterable': 1.0.3 + msgpackr: 1.11.8 + node-addon-api: 6.1.0 + node-gyp-build-optional-packages: 5.2.2 + ordered-binary: 1.6.1 + weak-lru-cache: 1.2.2 + optionalDependencies: + '@lmdb/lmdb-darwin-arm64': 3.5.1 + '@lmdb/lmdb-darwin-x64': 3.5.1 + '@lmdb/lmdb-linux-arm': 3.5.1 + '@lmdb/lmdb-linux-arm64': 3.5.1 + '@lmdb/lmdb-linux-x64': 3.5.1 + '@lmdb/lmdb-win32-arm64': 3.5.1 + '@lmdb/lmdb-win32-x64': 3.5.1 + loupe@3.2.1: {} magic-string@0.30.21: @@ -1815,8 +2090,32 @@ snapshots: ms@2.1.3: {} + msgpackr-extract@3.0.3: + dependencies: + node-gyp-build-optional-packages: 5.2.2 + optionalDependencies: + '@msgpackr-extract/msgpackr-extract-darwin-arm64': 3.0.3 + '@msgpackr-extract/msgpackr-extract-darwin-x64': 3.0.3 + '@msgpackr-extract/msgpackr-extract-linux-arm': 3.0.3 + '@msgpackr-extract/msgpackr-extract-linux-arm64': 3.0.3 + '@msgpackr-extract/msgpackr-extract-linux-x64': 3.0.3 + '@msgpackr-extract/msgpackr-extract-win32-x64': 3.0.3 + optional: true + + msgpackr@1.11.8: + optionalDependencies: + msgpackr-extract: 3.0.3 + nanoid@3.3.11: {} + node-addon-api@6.1.0: {} + + node-gyp-build-optional-packages@5.2.2: + dependencies: + detect-libc: 2.1.2 + + ordered-binary@1.6.1: {} + path-key@3.1.1: {} pathe@2.0.3: {} @@ -1849,6 +2148,8 @@ snapshots: remeda@2.33.4: {} + reusify@1.1.0: {} + rollup@4.55.1: dependencies: '@types/estree': 1.0.8 @@ -1894,6 +2195,8 @@ snapshots: slash@3.0.0: {} + sorted-btree@1.8.1: {} + source-map-js@1.2.1: {} stack-utils@2.0.6: @@ -2039,6 +2342,8 @@ snapshots: - tsx - yaml + weak-lru-cache@1.2.2: {} + web-worker@1.5.0: {} which@2.0.2: diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 1047611c..7bc94864 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -5,6 +5,7 @@ packages: - "context-api" - "converge" # deno-deploy excluded - deprecated Deno-only package + - "durable-streams" - "effect-ts" - "fetch" - "fs" diff --git a/tsconfig.json b/tsconfig.json index 7b917b58..171cebdc 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -17,6 +17,7 @@ { "path": "chain" }, { "path": "context-api" }, { "path": "converge" }, + { "path": "durable-streams" }, { "path": "effect-ts" }, { "path": "fetch" }, { "path": "fs" },