Skip to content

refactor(workflow): move output validation off inline node paths#1047

Merged
wolo-lab merged 4 commits into
v2from
wolo/output-validation-cleanup
Jun 17, 2026
Merged

refactor(workflow): move output validation off inline node paths#1047
wolo-lab merged 4 commits into
v2from
wolo/output-validation-cleanup

Conversation

@wolo-lab

Copy link
Copy Markdown
Contributor

Problem

Output validation for FunctionNode and ToolNode was duplicated. Each node
validated its output inline (inside wrappedFn / runTool) and the
scheduler validated it again scheduler-side via BaseNode.ValidateOutput
(introduced in #970). The same OutputSchema was therefore checked twice on
every activation.
Beyond the redundancy, the two copies are a drift hazard: as the
scheduler-side helper evolves (e.g. #970 added the *genai.Content text
fallback and passthrough for framework control values), the inline copies fall
behind and the two paths diverge. The validation behaviour also stopped being
uniform across node types — custom Node implementations only ever went through
the scheduler, while FunctionNode/ToolNode carried their own logic.
The FunctionTool {"result": X} unwrap convention was buried inline inside
ToolNode.runTool, mixing a tool-specific quirk into the execution path instead
of expressing it as part of the validation contract.

Solution

Make the scheduler the single, authoritative output-validation point and remove
the inline validation from the node run paths:

  • FunctionNode: drop the inline OutputSchema check from all three
    constructors (newFunctionNodeWithResolvedSchemas, NewFunctionNodeFromState,
    and the emitting variant). The schema still flows through BaseNode, so the
    scheduler enforces it via BaseNode.ValidateOutput. wrappedFn is back to
    doing one thing: materialise the typed parameter and call the user function.
  • ToolNode: remove the inline validation from runTool and move the
    FunctionTool {"result": X} unwrap into a dedicated ToolNode.ValidateOutput
    override. The override tries standard validation first, falls back to
    unwrapping {"result": X}, and re-runs standard validation on total failure
    so the caller still sees the original schema-mismatch error. This is the right
    home for the convention because it is tool-specific — making it a general
    default could mask genuine validation errors in other node types.

Behaviour note

Validation errors now surface scheduler-side rather than inside Run. For real
workflows this is unobservable (every activation goes through the scheduler).
Callers that invoke node.Run() directly (mostly tests) now receive the raw
output and must call ValidateOutput themselves. Tests are split accordingly:
Run-level tests assert raw passthrough; validation is asserted via
ValidateOutput (TestFunctionNode_ValidateOutput,
TestToolNode_ValidateOutput covering direct-valid, {"result": X} unwrap,
fail-both-paths, and nil-schema cases).

@wolo-lab
wolo-lab force-pushed the wolo/output-validation-cleanup branch 2 times, most recently from eedb6ee to 3df7c29 Compare June 16, 2026 18:15
Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).

The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.

Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
  TestFunctionNode_ValidateOutput, which asserts Run passes the output
  through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
  output from Run (no inline unwrap); the new
  TestToolNode_ValidateOutput covers the four override cases
  (direct-valid passthrough, {"result": X} unwrap, both-fail original
  error, nil-schema passthrough).

BUG=516382303
BUG=516382092
@wolo-lab
wolo-lab force-pushed the wolo/output-validation-cleanup branch from 3df7c29 to ca8a6f6 Compare June 16, 2026 18:17
@wolo-lab
wolo-lab marked this pull request as ready for review June 16, 2026 18:19
@wolo-lab
wolo-lab requested a review from hanorik June 16, 2026 18:19
@wolo-lab wolo-lab self-assigned this Jun 16, 2026
Comment thread workflow/function_node_test.go Outdated
// is enforced through the node-level ValidateOutput contract (invoked
// scheduler-side), not inline inside Run. Run itself passes the output
// through unchanged; ValidateOutput surfaces the schema mismatch.
func TestFunctionNode_ValidateOutput(t *testing.T) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe to add a success test case as well?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. Adding.

wolo-lab added 3 commits June 17, 2026 14:38
Cover the positive path: a schema-conforming output passes
ValidateOutput and is returned unchanged. Brings the test to parity
with TestToolNode_ValidateOutput, which already asserts a direct-valid
case.
…put table-driven

Separate the two concerns the test conflated: Run-passthrough now lives
in TestFunctionNode_RunDoesNotValidate, while TestFunctionNode_ValidateOutput
becomes table-driven (direct-valid, schema-mismatch, nil-schema). Mirrors
the structure of TestToolNode_Run + TestToolNode_ValidateOutput.
@wolo-lab
wolo-lab merged commit 4c45775 into v2 Jun 17, 2026
3 checks passed
@wolo-lab
wolo-lab deleted the wolo/output-validation-cleanup branch June 17, 2026 14:51
wolo-lab added a commit that referenced this pull request Jun 18, 2026
* refactor(workflow): move output validation off inline node paths

Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).

The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.

Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
  TestFunctionNode_ValidateOutput, which asserts Run passes the output
  through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
  output from Run (no inline unwrap); the new
  TestToolNode_ValidateOutput covers the four override cases
  (direct-valid passthrough, {"result": X} unwrap, both-fail original
  error, nil-schema passthrough).

BUG=516382303
BUG=516382092

* test(workflow): add success case to TestFunctionNode_ValidateOutput

Cover the positive path: a schema-conforming output passes
ValidateOutput and is returned unchanged. Brings the test to parity
with TestToolNode_ValidateOutput, which already asserts a direct-valid
case.

* test(workflow): split FunctionNode validation tests, make ValidateOutput table-driven

Separate the two concerns the test conflated: Run-passthrough now lives
in TestFunctionNode_RunDoesNotValidate, while TestFunctionNode_ValidateOutput
becomes table-driven (direct-valid, schema-mismatch, nil-schema). Mirrors
the structure of TestToolNode_Run + TestToolNode_ValidateOutput.
wolo-lab added a commit that referenced this pull request Jun 22, 2026
* refactor(workflow): move output validation off inline node paths

Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).

The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.

Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
  TestFunctionNode_ValidateOutput, which asserts Run passes the output
  through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
  output from Run (no inline unwrap); the new
  TestToolNode_ValidateOutput covers the four override cases
  (direct-valid passthrough, {"result": X} unwrap, both-fail original
  error, nil-schema passthrough).

BUG=516382303
BUG=516382092

* test(workflow): add success case to TestFunctionNode_ValidateOutput

Cover the positive path: a schema-conforming output passes
ValidateOutput and is returned unchanged. Brings the test to parity
with TestToolNode_ValidateOutput, which already asserts a direct-valid
case.

* test(workflow): split FunctionNode validation tests, make ValidateOutput table-driven

Separate the two concerns the test conflated: Run-passthrough now lives
in TestFunctionNode_RunDoesNotValidate, while TestFunctionNode_ValidateOutput
becomes table-driven (direct-valid, schema-mismatch, nil-schema). Mirrors
the structure of TestToolNode_Run + TestToolNode_ValidateOutput.
wolo-lab added a commit that referenced this pull request Jun 30, 2026
* refactor(workflow): move output validation off inline node paths

Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).

The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.

Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
  TestFunctionNode_ValidateOutput, which asserts Run passes the output
  through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
  output from Run (no inline unwrap); the new
  TestToolNode_ValidateOutput covers the four override cases
  (direct-valid passthrough, {"result": X} unwrap, both-fail original
  error, nil-schema passthrough).

BUG=516382303
BUG=516382092

* test(workflow): add success case to TestFunctionNode_ValidateOutput

Cover the positive path: a schema-conforming output passes
ValidateOutput and is returned unchanged. Brings the test to parity
with TestToolNode_ValidateOutput, which already asserts a direct-valid
case.

* test(workflow): split FunctionNode validation tests, make ValidateOutput table-driven

Separate the two concerns the test conflated: Run-passthrough now lives
in TestFunctionNode_RunDoesNotValidate, while TestFunctionNode_ValidateOutput
becomes table-driven (direct-valid, schema-mismatch, nil-schema). Mirrors
the structure of TestToolNode_Run + TestToolNode_ValidateOutput.
wolo-lab added a commit that referenced this pull request Jun 30, 2026
* refactor(workflow): move output validation off inline node paths

Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).

The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.

Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
  TestFunctionNode_ValidateOutput, which asserts Run passes the output
  through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
  output from Run (no inline unwrap); the new
  TestToolNode_ValidateOutput covers the four override cases
  (direct-valid passthrough, {"result": X} unwrap, both-fail original
  error, nil-schema passthrough).

BUG=516382303
BUG=516382092

* test(workflow): add success case to TestFunctionNode_ValidateOutput

Cover the positive path: a schema-conforming output passes
ValidateOutput and is returned unchanged. Brings the test to parity
with TestToolNode_ValidateOutput, which already asserts a direct-valid
case.

* test(workflow): split FunctionNode validation tests, make ValidateOutput table-driven

Separate the two concerns the test conflated: Run-passthrough now lives
in TestFunctionNode_RunDoesNotValidate, while TestFunctionNode_ValidateOutput
becomes table-driven (direct-valid, schema-mismatch, nil-schema). Mirrors
the structure of TestToolNode_Run + TestToolNode_ValidateOutput.
wolo-lab added a commit that referenced this pull request Jun 30, 2026
* refactor(workflow): move output validation off inline node paths

Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).

The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.

Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
  TestFunctionNode_ValidateOutput, which asserts Run passes the output
  through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
  output from Run (no inline unwrap); the new
  TestToolNode_ValidateOutput covers the four override cases
  (direct-valid passthrough, {"result": X} unwrap, both-fail original
  error, nil-schema passthrough).

BUG=516382303
BUG=516382092

* test(workflow): add success case to TestFunctionNode_ValidateOutput

Cover the positive path: a schema-conforming output passes
ValidateOutput and is returned unchanged. Brings the test to parity
with TestToolNode_ValidateOutput, which already asserts a direct-valid
case.

* test(workflow): split FunctionNode validation tests, make ValidateOutput table-driven

Separate the two concerns the test conflated: Run-passthrough now lives
in TestFunctionNode_RunDoesNotValidate, while TestFunctionNode_ValidateOutput
becomes table-driven (direct-valid, schema-mismatch, nil-schema). Mirrors
the structure of TestToolNode_Run + TestToolNode_ValidateOutput.
wolo-lab added a commit that referenced this pull request Jun 30, 2026
* refactor(workflow): move output validation off inline node paths

Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).

The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.

Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
  TestFunctionNode_ValidateOutput, which asserts Run passes the output
  through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
  output from Run (no inline unwrap); the new
  TestToolNode_ValidateOutput covers the four override cases
  (direct-valid passthrough, {"result": X} unwrap, both-fail original
  error, nil-schema passthrough).

BUG=516382303
BUG=516382092

* test(workflow): add success case to TestFunctionNode_ValidateOutput

Cover the positive path: a schema-conforming output passes
ValidateOutput and is returned unchanged. Brings the test to parity
with TestToolNode_ValidateOutput, which already asserts a direct-valid
case.

* test(workflow): split FunctionNode validation tests, make ValidateOutput table-driven

Separate the two concerns the test conflated: Run-passthrough now lives
in TestFunctionNode_RunDoesNotValidate, while TestFunctionNode_ValidateOutput
becomes table-driven (direct-valid, schema-mismatch, nil-schema). Mirrors
the structure of TestToolNode_Run + TestToolNode_ValidateOutput.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants