[Feature] Add retry semantics to onNoConsensus transition

[alxsuv]

Problem

onFailure retry N otherwise "X" exists today (NodeBuilderTest.kt:70, wired via RetryBuilder.kt:27 → FailureTransition). Stochastic parallel agents can tie on first attempt and resolve on retry. Extend
the same DSL form to onNoConsensus.

Investigation (do first)

1. onNoConsensus transition type. Determine whether it constructs a FailureTransition (shared HensuState.retryCount → cross-corruption with execution failures) or its own transition with an isolated
   counter. Source: hensu-dsl/.../ParallelNodeBuilder.kt. If alias → must split before adding retry.
2. FailureTransition.evaluate side-effect (FailureTransition.java:16 increments inside evaluate). Decide: fix purity first, or scope-document and move on.

Implementation

• Distinct NoConsensusTransition (or split from FailureTransition) with its own counter (consensusRetryCount on HensuState).
• DSL: onNoConsensus retry 3 otherwise "fallback".
• Increment counter in executor / TransitionPostProcessor, not inside evaluate.

Tests

• Stochastic stub agents tie on attempt 1, break tie on attempt 2 → success.
• Retry budget exhausted → routes to otherwise target.
• Execution failure on a parallel node does NOT consume consensusRetryCount, and vice versa.

Out of scope

• onReject retry N (rejected: deterministic reviewer → infinite loop; same input across retries provides no value).
• Arbitrator/judge node pattern.
• General evaluate() purity refactor across other transition types.

[alxsuv]

Hi @Ilya0527, thanks for opening this discussion. Both points are valid and were identified during investigation. The original task definition had structural problems that required revision before implementation. Here's what changed and what proposed.

What was wrong in the initial task definition

The ticket framed onNoConsensus as a mirror of onFailure retry — same verb, same topology (self-loop on the consensus node). Investigation revealed three problems with that framing:

1. A consensus node is a review/assessment node. Self-looping re-runs the same reviewers on the same input. The work product lives upstream — the producer node is the one that needs to iterate, not the reviewers.

2. retry implies a blind self-loop for transient failures (network timeout, rate limit). Consensus failure is semantic — reviewers disagree on substance. Blind repetition doesn't address their objections. This is the same reason onReject retry N was rejected in the original scope: deterministic reviewer + identical input = infinite loop. A self-looping onNoConsensus retry has the same flaw.

3. The flat retryCount on HensuState is global. This is an existing latent bug: sequential nodes A→B sharing onFailure retry consume a single counter. Splitting it into consensusRetryCount (as initially proposed) fixes cross-corruption between failure and consensus, but — as you correctly identified — does not fix re-entry within workflow loops. A goto cycle or diamond convergence on the same parallel node starts with a partially consumed budget.

What was revised

Verb and topology: revise, not retry

The verb changed from retry (self-loop) to revise (backtrack to producer). DSL:

onNoConsensus revise "work_node" retries 3 otherwise "escalate"

revise backtracks execution to the named producer node. The consensus node's reviewers are not re-invoked directly — they run again naturally when the revised work product flows back through the graph.

Counter keying: Map<String, Integer> by namespace:nodeId

The global int retryCount is replaced with Map<String, Integer> retryCounters keyed by "namespace:nodeId" (e.g., "failure:generate", "consensus:review_panel"). This addresses:

• Cross-corruption — failure and consensus counters are independent, even on the same node.
• Re-entry in loops — counters are reset when the node succeeds (TransitionPostProcessor clears both failure:<nodeId> and consensus:<nodeId> on ResultStatus.SUCCESS). A workflow loop that revisits a node after success starts with a fresh budget. The existing FailureTransition gets the same fix as a side-effect of the refactor.
• Snapshot persistence — the counter map is included in HensuSnapshot, so pause/resume preserves retry state correctly.

Perturbation mechanism: feedback injection, not RNG

This is the most important architectural point. You're right that a blind retry of stochastic agents is only meaningful if something actually changes between attempts. Temperature jitter or seed offset is one mechanism, but it's fragile — it couples correctness to inference configuration, and the stochastic-stub contract becomes "trust that Random advances," which is untestable against production agents.

Design of the system doesn't rely on stochasticity. The perturbation is explicit feedback:

1. When consensus fails, TransitionPostProcessor collects dissenting feedback from the ConsensusResult:

   • Algorithmic strategies (MAJORITY_VOTE, UNANIMOUS, WEIGHTED_VOTE): each branch already produces a recommendation via needsSelfScoring(). Dissenting branch recommendations are collected.
   • JUDGE_DECIDES: the judge's reasoning is used directly.

2. The collected feedback is written to state.context["recommendation"] — the same universal engine variable that RecommendationVariableInjector already manages and that producer nodes already template via {recommendation}. No new injector is needed. The producer prompt already includes {recommendation} as a standard pattern (e.g., "Write a short article about {topic}. {recommendation}"), so the consensus feedback flows into the producer's next invocation through existing template substitution.

3. The recommendation value is naturally overwritten on each node execution by OutputExtractionPostProcessor, so stale feedback does not leak into subsequent unrelated executions.

This means attempt N+1 is structurally different from attempt N — even at temperature 0 — because the input contains reviewer objections that weren't present on the first pass. The contract is testable: assert that recommendation in context contains consensus feedback when the producer node re-executes after a revise backtrack.

The "stochastic stub agents tie on attempt 1, break tie on attempt 2" test from the original spec is reframed: the stub producer agent returns improved output when {recommendation} contains consensus feedback, and the stub reviewers reach consensus on the improved version. This validates the feedback loop end-to-end without depending on RNG.

Summary of proposed implementation

Concern	Resolution
Counter cross-corruption (failure vs consensus)	Namespaced keys: failure:<nodeId>, consensus:<nodeId>
Counter re-entry in workflow loops	Reset on node success in TransitionPostProcessor
Counter persistence across pause/resume	retryCounters map added to HensuSnapshot
Self-loop on review node	Backtrack to producer node (revise verb)
Identical input across attempts	Consensus feedback written to {recommendation} — existing engine variable, already templated by producer nodes
Agent re-seeding / temperature jitter	Not needed — feedback via {recommendation} is the perturbation, not RNG
evaluate() purity	Side-effect (incrementRetryCount) moved to TransitionPostProcessor

Implementation is tracked in this ticket — phases: per-node counters (bugfix) → NoConsensusTransition record → namespaced consensus context keys → DSL wiring (revise verb, ReviseBuilder chain) → consensus feedback via {recommendation} in TransitionPostProcessor → tests.

[alxsuv]

Scope note — the implementing branch landed well beyond this ticket's ask.

The original request was narrow: add retry/backtrack semantics to onNoConsensus. During implementation the scope was deliberately widened to fix a class of
related bugs in one pass, and a second feature plus a docs-sync rode along on the same branch. Recording the delta here so the issue history reflects what
actually shipped.

In scope (what #73 asked for)

• revise semantics on onNoConsensus: consensus failure backtracks to the producer node instead of re-running reviewers.

Expanded under #73 (same concern, fixed together)

• Generalized to a BoundedTransition decorator covering all loop-prone triggers — onRejection, onNoConsensus, and score arms — not just no-consensus.
• Fixed the pre-existing global retry-counter bug: budgets are now namespaced per node and per trigger (approval, consensus, score, failure), so sequential
  nodes no longer share one counter.
• New NoConsensusTransition rule, TransitionTargets/NodeTargets helpers, and a requiredEngineVars() capability method that removed instanceof introspection from
  the enricher/post-processor sites.
• FailureTransition reduced to a pure trigger; retry increment moved out of evaluate() into TransitionPostProcessor.
• DSL revise "producer" retry N otherwise "escalate" chain + bounded/noConsensus serialization with round-trip tests.
• V3__add_retry_counters.sql and JDBC repo wiring for the new counter column.

Out of scope for #73, but in the same commit

• Feedback auto-injection: FeedbackContextInjector appends a ### Previous Feedback prompt section so the rerun node sees why it was sent back; withFeedback
  keyword for forward goto.
• Documentation actualization across AGENTS.md, READMEs, and the architecture/developer guides (transition model, 7-injector enricher order, corrected MCP
  split-pipe terminology).

If clean history matters, this should be split into three commits (bounded-revise / feedback-injection / docs-sync) on the shared branch.