Wolo/output validation llmagent#1048
Draft
wolo-lab wants to merge 2 commits into
Draft
Conversation
Make the scheduler the single authoritative point for output
validation. FunctionNode.wrappedFn and ToolNode.runTool no longer
validate the output against their schema inline; the schema still
flows through BaseNode so the scheduler enforces it on every yielded
event with non-nil output. This removes duplicate validation and the
risk of drift as the scheduler logic evolves (symmetric to the input
validation cleanup).
The FunctionTool {"result": X} unwrap fallback is not lost: it moves
to a ToolNode.ValidateOutput override, the correct home for that
tool-specific convention. The override tries standard validation
first, then unwraps a "result" key and validates the unwrapped value,
and finally re-runs standard validation so callers see the original
schema-mismatch error rather than a fallback artifact. nil schema is
a passthrough.
Tests that previously asserted on inline behavior are updated:
- FunctionNode: the inline ValidationError table case is replaced by
TestFunctionNode_ValidateOutput, which asserts Run passes the output
through unchanged and ValidateOutput surfaces the schema mismatch.
- ToolNode: TestToolNode_Run now asserts the raw FunctionTool map
output from Run (no inline unwrap); the new
TestToolNode_ValidateOutput covers the four override cases
(direct-valid passthrough, {"result": X} unwrap, both-fail original
error, nil-schema passthrough).
BUG=516382303
BUG=516382092
…tate Complete the long-standing TODO in maybeSaveOutputToState: when LlmAgent.OutputSchema is set, the final model text is JSON-parsed and validated against the schema, and the validated object is written to state[OutputKey] (and Event.Output) instead of the raw text. With no schema, the raw text is saved as before. Invalid JSON or output that does not match the schema causes the state write to be skipped rather than persisting malformed output. Uses the existing utils.ValidateOutputSchema helper (already used by agenttool) for the genai.Schema parse-and-validate, keeping the behavior consistent across the codebase. Brings adk-go to parity with adk-python's __maybe_save_output_to_state, where state[output_key] holds the validated object. Behavior change: users who set both OutputKey and OutputSchema previously got the raw model text in state; they now get the parsed and validated object. Users who set only OutputKey (the common case) see no change. This is documented as a behavior change; the prior behavior was an explicitly unfinished TODO and had no test coverage for the schema-present path. Tests: llmagent_saveoutput_test.go now covers the three schema paths (valid JSON matching schema saves the parsed object, invalid JSON skips the write, valid JSON not matching the schema skips the write). BUG=516382153
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
2. Or, if no issue exists, describe the change:
If applicable, please follow the issue templates to provide as much detail as
possible.
Problem:
A clear and concise description of what the problem is.
Solution:
A clear and concise description of what you want to happen and why you choose
this solution.
Testing Plan
Please describe the tests that you ran to verify your changes. This is required
for all PRs that are not small documentation or typo fixes.
Unit Tests:
Please include a summary of passed go test results.
Manual End-to-End (E2E) Tests:
Please provide instructions on how to manually test your changes, including any
necessary setup or configuration. Please provide logs or screenshots to help
reviewers better understand the fix.
Checklist
Additional context
Add any other context or screenshots about the feature request here.