You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(task): implement GraphResolver with fan-in cycle pruning and migrate monorepo to Task System v3 (#976)
* feat(task): implement 4-phase GraphResolver, concrete edge tracking, and memory lifecycle in LocalRunner
* refactor(inspection): modernize inspection taskbase and runner for edge attributes
* refactor(task): migrate inspection tasks across monorepo to new task system
* feat(task): implement priority-based fan-in pruning and cycle detection in GraphResolver
* fix(review): address review comments on formtask aliases, inventory task multi-stage, and cycle tests
* feat(task): support stage-aware fan-in binding in TaskGraphMetadata and TaskSet
* feat(task): split cyclic multi-stage consumer tasks into DAG stages in GraphResolver
* test(inspection): verify end-to-end multi-stage inventory aggregation with cyclic producer
* fix(task): resolve LocalRunner stage tracking data race and improve naming
* refactor(task): eliminate dead BoundReferenceIDsWithTag and normalize task-scoped tag metadata
* refactor(task): decompose cycle resolution helpers and add edge routing tests
* refactor(task): clean up taskset doc comment, runner identifiers, and taskid parameter naming
* refactor(task): decompose cycle resolution helpers and extract graph utilities into graphresolver_graphutil.go
* test(task): add upstream PtP edge routing test for split multi-stage consumer
* refactor(task): eliminate obsolete refToTask and refToImplID maps in cycle resolution
* refactor(task): decompose cycle resolution and stage expansion functions
* refactor(task): standardize PointToPoint identifiers and fix inverted test description
* refactor(task): rename TaskEdge endpoints to SourceImplID and TargetImplID
* refactor(task): extract stage expansion and rerouting into graphresolver_stage.go
* fix(task): reroute fan-in edges from split producer and ensure deterministic stage edges
* refactor(task): align stage identifier names and test mock field naming
* refactor(task): extract stage tests into graphresolver_stage_test.go and co-locate stageTask
* fix(task): prevent premature deletion of self-loop stage results and add runner test
* refactor(task): decompose rerouteFanInEdges in graphresolver_stage.go
* refactor(task): standardize remainingStagesByRefID and boundFanInRefIDsByTaskImpl identifiers
* test(task): add unit tests for graphresolver_graphutil.go
* fix(task): replace scalar cmp.Diff with direct equality in runner_test.go and add stage assertions
* refactor(task): eliminate abbreviations and align identifiers in stage resolution and tests
* refactor(task): rename rem and remDep to remainingDependents in LocalRunner
* refactor(task): align stage edge identifiers and non-split naming in stage resolution
* refactor(task): flatten control flow nesting in LocalRunner cleanup
* refactor(task): fix point-to-point doc comment and disambiguate split consumer IDs
* refactor(task): rename BoundReferenceIDsForTaskWithTag to BoundReferenceIDsForTaskImplWithTag
* refactor(task): align stageTaskPairs, producer identifiers, and table-driven graphutil tests
* refactor(task): prune cyclic fan-in candidate edges into single-stage DAG and unify dependency scopes
Copy file name to clipboardExpand all lines: docs/en/khi-task-system-concept.md
+2-6Lines changed: 2 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -112,7 +112,7 @@ In KHI, connections (edges) between tasks in the DAG are represented by the `Dep
112
112
Represents a direct 1-to-1 dependency on a specific task reference (`taskID.Ref()`). Downstream tasks read the upstream task's return value using `coretask.GetTaskResult(ctx, ref)`.
113
113
-**Tag Fan-In (`TagReference[T]`)**:
114
114
Represents a 1-to-N aggregated dependency. Producer tasks declare the tags they provide using the `coretask.ProvidesTag(tag, opts...)` label option. You can optionally specify `coretask.WithTagPriority(priority)` to assign precedence to the producer's contribution (default: 100, where lower numerical values indicate higher precedence). A consumer task declares a dependency on the tag using `tag.Ref()`. During execution, the consumer retrieves a combined slice of results (`[]T`) from all active producer tasks using `coretask.GetTaskResultsWithTag(ctx, tag.Ref())`. This allows new log parsers or metadata producers to be added without modifying downstream consumer tasks.
115
-
When cross-inventory dependencies between multiple producers cause circular dependencies, pure aggregator tasks annotated with `coretask.AllowMultiStageExecution()` can be split into multiple execution stages by the graph resolver to automatically resolve cycles. For details on prerequisites and resolution mechanisms, see [6. Prerequisites of Fan-In Cycles and Graph Stabilization via Priority](#6-prerequisites-of-fan-in-cycles-and-graph-stabilization-via-priority).
115
+
When cross-inventory dependencies between multiple producers cause circular dependencies, the graph resolver deterministically prunes candidate fan-in edges that form cycles, automatically resolving the circular dependency into a single-stage DAG. For details on prerequisites and resolution mechanisms, see [6. Prerequisites of Fan-In Cycles and Graph Stabilization via Priority](#6-prerequisites-of-fan-in-cycles-and-graph-stabilization-via-priority).
116
116
117
117
#### 2. Edge Kind: Data vs Order-Only
118
118
@@ -200,11 +200,7 @@ KHI achieves an always unique, deterministic, and stable graph through the follo
200
200
Producer tasks declare their contribution certainty and priority using `ProvidesTag(tag, WithTagPriority(priority))` (default: 100, where lower numerical values indicate higher precedence).
201
201
- For example: A parser providing definitive metadata early in execution has high precedence (`Priority: 10`), whereas a parser supplementing metadata later as a byproduct of parsing has low precedence (`Priority: 100`).
202
202
2.**Priority-Based Deterministic Pruning**:
203
-
When a cycle is detected across fan-in dependencies, the graph resolver deterministically prunes the fan-in edge with the **lowest priority (highest numerical value)** within the cycle, restoring an acyclic DAG.
204
-
3.**Strict Fail-Fast on Priority Ties**:
205
-
If edges in a cycle share identical priorities and the resolver cannot deterministically pick which edge to prune, it does not guess. Graph resolution fails fast immediately with an error.
206
-
4.**Data Preservation via Multi-Stage Execution (`AllowMultiStageExecution`)**:
207
-
For pure, side-effect-free aggregator tasks (such as in-memory inventory aggregators), annotating them with `AllowMultiStageExecution()` permits the resolver to automatically split and clone them into early and late execution stages. The early stage receives high-priority inputs, while the late stage collects feedback inputs from dependent parsers, safely resolving cycles without dropping data.
203
+
When candidate fan-in edges are evaluated, the graph resolver considers producers in order of priority and deterministically prunes any candidate edge that would form a cycle, yielding a safe, acyclic DAG.
@@ -110,9 +108,7 @@ When using fan-in aggregation, circular dependencies (cycles) can arise under th
110
108
2.**Deterministic Pruning via Priority for a Stable Graph**:
111
109
Arbitrarily cutting edges to break cycles causes execution order and data flow to fluctuate based on task registration order, producing an unreproducible, unstable graph.
112
110
-`coretask.WithTagPriority(priority)` (default: `DefaultTagPriority = 100`, where lower numbers indicate higher precedence) lets producers declare the certainty and priority of their contribution.
113
-
- The graph resolver deterministically prunes the lowest-priority fan-in edge within the cycle, producing an always unique and stable graph. If priorities tie within a cycle and the choice is ambiguous, the resolver does not guess and fails fast with an error.
To avoid losing data when pruning feedback edges, pure side-effect-free aggregator tasks should declare `AllowMultiStageExecution()`. The resolver automatically splits and clones the aggregator into an early stage (passing high-priority inputs to parsers) and a late stage (collecting feedback outputs after parsers complete). If an unlabelled task requires multi-stage execution to resolve a cycle, graph resolution fails fast with an error.
111
+
- The graph resolver deterministically prunes candidate fan-in edges that form cycles, consistently producing a safe, unique, and stable single-stage DAG.
116
112
117
113
For architectural details, see [Concept Guide: 6. Prerequisites of Fan-In Cycles and Graph Stabilization via Priority](../khi-task-system-concept.md#6-prerequisites-of-fan-in-cycles-and-graph-stabilization-via-priority).
0 commit comments