Enhancement: Optimize dependency resolution overhead (TaskGroup bottleneck)

[maximsakhno]

Summary

Benchmarks show a significant performance degradation in dependency resolution when increasing the number of flat dependencies. Specifically, moving from 1 to 2 dependencies results in a sharp drop in RPS — down to ~36%.

Investigation suggests this is due to the overhead introduced by TaskGroup usage for parallel resolution.

Basic Example

Here is a reproduction scenario with two empty dependencies and handlers using one vs. two dependencies:

from litestar import get, Litestar

async def dep0() -> int:
    return 42

async def dep1() -> int:
    return 42

@get("/deps-1")
async def handler1(
    val0: int,
) -> str:
	# Benchmark result: 41,451 req/s
    return "OK"

@get("/deps-2")
async def handler2(
    val0: int,
    val1: int,
) -> str:
    # Benchmark result: 26,492 req/s 
    # (Performance drops to 36% compared to "/deps-1")
    return "OK"

Visualizing the drop:

The full benchmark reproduction is available here: https://github.com/maximsakhno/di-benchmarks

Drawbacks and Impact

Optimizing the dependency resolution logic will prevent the performance cliff and substantially increase RPS for endpoints with multiple dependencies, making the framework much more efficient in real-world scenarios with complex dependency trees.

Unresolved questions

No response

[maximsakhno]

I've analyzed the source code and I believe I have a solution to fix this.

Root Cause: Looking at Litestar's dependency resolution logic, a TaskGroup is used to resolve batches of dependencies in parallel: https://github.com/litestar-org/litestar/blob/v2.19.0/litestar/_kwargs/kwargs_model.py#L390

As noted in the issue description, the TaskGroup overhead is the main culprit for the RPS drop.

Proposed Solution: I propose moving away from TaskGroup for this context and resolving dependencies sequentially.

Currently, Litestar employs Kahn's algorithm (https://github.com/litestar-org/litestar/blob/v2.19.0/litestar/_kwargs/dependencies.py#L77) to organize dependencies into batches for concurrent execution. Since these batches are no longer necessary for a sequential approach, I suggest simplifying this logic by replacing Kahn's algorithm with a topological sort based on Depth-First Search (DFS).

A DFS-based approach is:

1. Simpler to implement and maintain.
2. Naturally produces a linear sequence of dependencies, which aligns perfectly with sequential execution.

Here is a conceptual example of the sequential approach:

async def resolve_dependencies(self, connection: ASGIConnection, kwargs: dict[str, Any]) -> DependencyCleanupGroup:
    cleanup_group = DependencyCleanupGroup()
    for dependency in self.sequential_dependencies:
        await resolve_dependency(dependency, connection, kwargs, cleanup_group)
    return cleanup_group

I would like to implement this change and submit a PR if the maintainers agree with this direction. Could you please assign this issue to me?

[provinzkraut]

This is only true for dependencies that do no work. The assumption taken in Litestars implementation is that dependencies actually do something; In particular, that async dependencies perform async tasks, i.e. io-bound work.

Your benchmark measure only the overhead, but not the real world usage. Before implementing any changes, I'd like to see a benchmark that involves dependencies with io-bound async loads (e.g. connecting to a database).

[maximsakhno]

Regarding the benchmark from the issue: even there, the CPU utilization was at 100%. A 36% drop in RPS is significant because it represents pure dependency resolution logic overhead on top of Uvicorn (IO + HTTP parsing) and Litestar’s core logic. The cost of TaskGroup management impacts the CPU budget regardless of whether the dependencies themselves are doing heavy work.

However, to address your request, I conducted a new benchmark involving actual database interactions.

Methodology

I created endpoints with 1, 2, and 4 dependencies. To isolate the overhead of dependency resolution, I kept the total work constant across all scenarios (4 DB queries per request):

• 1 Dependency: performs 4 DB queries.

• 2 Dependencies: perform 2 DB queries each (4 total).

• 4 Dependencies: perform 1 DB query each (4 total).

async def fetch_data(num_queries: int = 1) -> None:
    for _ in range(num_queries):
        await pool.fetchval("SELECT 42")

async def dep11() -> int:
    await fetch_data(num_queries=4)
    return 42


async def dep21() -> int:
    await fetch_data(num_queries=2)
    return 42


async def dep22() -> int:
    await fetch_data(num_queries=2)
    return 42
    
@get("/deps-1")
async def handler1(
    val11: int, 
) -> str:
    return "OK"

@get("/deps-2")
async def handler2(
    val21: int, 
    val22: int, 
) -> str:
    return "OK"
    
app = Litestar(
    route_handlers=[
        handler1, 
        handler2,
    ],
    dependencies={
        "val11": Provide(dep11),
        "val21": Provide(dep21),
        "val22": Provide(dep22),
    },
    lifespan=[manage_db_pool],
)

I tested two implementations of the dependency resolution logic:

1. Current: The original implementation using TaskGroup.

2. Proposed: An optimization where dependencies are simply awaited.

# Current:
async def resolve_dependencies(self, connection: ASGIConnection, kwargs: dict[str, Any]) -> DependencyCleanupGroup:
    cleanup_group = DependencyCleanupGroup()
    for batch in self.dependency_batches:
        if len(batch) == 1:
            await resolve_dependency(next(iter(batch)), connection, kwargs, cleanup_group)
        else:
            try:
                async with create_task_group() as task_group:
                    for dependency in batch:
                        task_group.start_soon(resolve_dependency, dependency, connection, kwargs, cleanup_group)
            except _ExceptionGroup as excgroup:
                raise excgroup.exceptions[0] from excgroup  # type: ignore[attr-defined]

    return cleanup_group

# Proposed:
async def resolve_dependencies(self, connection: ASGIConnection, kwargs: dict[str, Any]) -> DependencyCleanupGroup:
    cleanup_group = DependencyCleanupGroup()
    for batch in self.dependency_batches:
        for dependency in batch:
            await resolve_dependency(dependency, connection, kwargs, cleanup_group)

    return cleanup_group

Results

Even with the TaskGroup implementation handling IO-bound work, splitting the logic into multiple dependencies incurs a heavy cost:

• 1 → 2 Dependencies: An 11% drop in RPS.

• 1 → 4 Dependencies: A 17% drop in RPS.

Key Observation: During these tests, neither the database nor wrk were the bottleneck (they were not at 100% load). The Litestar application process was the bottleneck, pinned at 100% CPU.

Considering the application is already doing "heavy" work (HTTP handling + framework logic + 4 network DB queries), an 11-17% performance penalty solely due to TaskGroup overhead is very expensive. As shown in the graphs, removing TaskGroup in favor of awaits recovers this RPS.

Real World Context

It is also worth noting that in practice, not all dependencies perform IO. A large portion of dependencies are CPU-bound: instantiating repositories, setting up services, or extracting/transforming data (headers, query params, etc.). Only a subset strictly performs heavy IO (DB/API calls).

Therefore, this benchmark represents the "Heavy IO" edge case. Real-world usage lies somewhere between the "No-op" benchmark and this "Heavy IO" benchmark. In both extremes, the TaskGroup logic introduces significant, measurable overhead.