Best Practices for Service Lifecycles and Dependency Injection (Executors/Workflows)

[Marcus-Kanon]

Hello there!

This framework seems really promising. I'm currently integrating it into a microservice that continuously processes messages from an event queue and I have some questions about the idiomatic way to handle service setup and lifecycle.

I've registered my executors and workflow as singletons, like so:

builder.Services.AddSingleton<IChatClientFactory, ChatClientFactory>();
builder.Services.AddSingleton(sp =>
    sp.GetRequiredService<IChatClientFactory>().CreateClient());

builder.Services.AddSingleton<StartExecutor>();
builder.Services.AddSingleton<TransformExecutor>();
builder.Services.AddSingleton<OptimistExecutor>();
builder.Services.AddSingleton<CynicExecutor>();
builder.Services.AddSingleton<AnalysisAggregatorExecutor>();

builder.Services.AddSingleton<IWorkflowFactory, WorkflowFactory>();
builder.Services.AddSingleton(sp =>
    sp.GetRequiredService<IWorkflowFactory>().CreateWorkflow());

This approach works well, provided I keep the executors stateless and rely on the context for state management.

My questions are:

• Is using a singleton lifetime for executors and workflows the recommended approach?
• Are there established best practices or preferences for what should be instantiated on every run versus registered as a singleton?
• Are there any potential issues or things to watch out for with my current setup?

Thank you!

[lokitoth]

Assuming that your executors are stateless, singletons are probably a good way of managing them. For "pure" function executors (input => output, no side-effects), this is almost certainly the case.

Once you start thinking about state it becomes a little more complicated and depends a little on what concurrency mode you intend to use when executing the workflow. To allow developers to begin quickly with minimal boilerplate, we allow them to pass Executor instances directly to the WorkflowBuilder. If those instances contain state on the object, however, it could cause bad behaviour if used simultaneously from multiple threads or multiple concurrent runs.

To protect against this, we introduced first the ability to run Workflows serially across multiple runs by letting Executors implement a "reset" protocol using IResettableExecutor. While this mode can work well for local applications, or other single-user processes, it behaves poorly when used in web hosting conditions. To that end we introduced the fully concurrent mode, but this requires a bit more care on the part of the workflow developer: To be able to use a Workflow concurrently, all executors provided as instances must declare themselves to be "cross-run-shareable", which (generally) means they have no on-object state, or are provided as factories, so every run can create a separate instance of the executor.

What this means for hosting in DI:

• If your executor has no on-object state, having it be a singleton is perfectly reasonable. Under the hood it will still be wrapped in a "factory", but that factory will always resolve to the same instance. I would recommend marking such executors as "cross-run-shareable" (one of the base constructor arguments), so you can execute multiple concurrent runs.

• If your executor does have on-object state, I would recommend having it be provided as a factory to the Workflow and implement the IResettableExecutor interface to be able to get to an initial state so the workflow can be used sequentially in multiple runs. Another thing to consider doing in this case is overriding the OnCheckpointingAsync and OnCheckpointRestoredAsync methods to ensure Checkpointing works correctly.

• Building a Workflow is not an expensive operation right now, as there is very little validation that can be performed in a synchronous context. Bigger workflows will take longer to build though, so using a single instance may be advantageous from a performance standpoint. In a web context I would recommend running in fully concurrent mode, so make sure either all executors are provided as factories, or they are "cross-run-shareable".

One last thing to note: By default, using the AsAgent extension to wrap a Workflow in an AIAgent uses the "concurrent" mode by default if the workflow supports it, which is another reason to strive for concurrent-compatible workflows.

[Marcus-Kanon]

Very informative! Thank you for taking your time to answer that. Will definitely keep these things in mind as I go forward.