Skip to content

Conversation

@gr2m
Copy link
Collaborator

@gr2m gr2m commented Oct 30, 2025

Background

Discovered in #9896 (review)

Summary

As far as I can tell, result has to be set to something that is not null or undefined. This makes the spec more precise and would have surfaced the problem of #9896 (review) immediately

Manual Verification

When the fix from this PR is applied, a type error is thrown in #9896

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

Future Work

Related Issues

@gr2m gr2m marked this pull request as draft October 30, 2025 23:44
@gr2m gr2m requested a review from a team October 30, 2025 23:45
@gr2m gr2m changed the title spec update: toll result .result is unknown, but not nullable fix(spec): LanguageModelV3ToolResult["result"] is unknown, but non-nullable Oct 30, 2025
@lgrammel
Copy link
Collaborator

We also need to remove the providerExecuted flag in this file that was somehow added to the spec. It would always be true i.e. not needed.

@lgrammel
Copy link
Collaborator

Removed the providerExecuted flag in #9936

* Result of the tool call. This is a JSON-serializable object.
*/
result: unknown;
result: NonNullable<unknown>;
Copy link
Collaborator

@lgrammel lgrammel Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we may want to also consider NonNullable<JSONValue> now that JSONValue supports undefined but the implications are unclear

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I. thought so too. I'll do that and see how it goes

Copy link
Collaborator Author

@gr2m gr2m Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't it work with JSONObject | JSONArray?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm it's tricky because it means we cannot use unknown for any (deep) value of the result object

like in line 50 here

export const fileSearchOutputSchema = lazySchema(() =>
zodSchema(
z.object({
queries: z.array(z.string()),
results: z
.array(
z.object({
attributes: z.record(z.string(), z.unknown()),
fileId: z.string(),
filename: z.string(),
score: z.number(),
text: z.string(),
}),
)
.nullable(),
}),
),
);

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the z.unknown() should be z.union([z.string(), z.number(), z.boolean()]) as per OpenAPI spec, will update

gr2m added 3 commits October 31, 2025 07:55
OpenAPI spec is

```json
{
  "anyOf": [
    {
      "additionalProperties": {
        "anyOf": [
          {
            "maxLength": 512,
            "type": "string"
          },
          {
            "type": "number"
          },
          {
            "type": "boolean"
          }
        ]
      },
      "description": "Set of 16 key-value pairs that can be attached to an object. This can be\nuseful for storing additional information about the object in a structured\nformat, and querying for objects via API or the dashboard. Keys are strings\nwith a maximum length of 64 characters. Values are strings with a maximum\nlength of 512 characters, booleans, or numbers.\n",
      "maxProperties": 16,
      "propertyNames": {
        "maxLength": 64,
        "type": "string"
      },
      "type": "object"
    },
    {
      "type": "null"
    }
  ]
}
```

https://github.com/gr2m/ai-provider-monitor/blob/fe2659312a6344501c90942e2bf2baca06f5f9d9/cache/openai/routes/responses/post.json#L2571
@gr2m gr2m changed the title fix(spec): LanguageModelV3ToolResult["result"] is unknown, but non-nullable fix(spec): LanguageModelV3ToolResult["result"] change from unknown to JSONObject | JSONArray Oct 31, 2025
Comment on lines +401 to +404
attributes: z.record(
z.string(),
z.union([z.string(), z.number(), z.boolean()]),
),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as a side note: JSON API / JSON Schema specs do not have an "unknown" type. I haven't checked all providers we support yet, but all the important ones do have a JSON API spec for their APIs.

We still use .unknown() in other places related to provider requests/responses that we could probably all get rid off with some research.

* Result of the tool call. This is a JSON-serializable object.
*/
result: unknown;
result: JSONObject | JSONArray;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will change to NonNullable<JSONValue> after discussing with @lgrammel

@gr2m gr2m changed the title fix(spec): LanguageModelV3ToolResult["result"] change from unknown to JSONObject | JSONArray fix(spec): LanguageModelV3ToolResult["result"] change from unknown to NonNullable<JSONValue> Nov 3, 2025
@gr2m gr2m marked this pull request as ready for review November 3, 2025 19:04
@gr2m gr2m merged commit bb36798 into main Nov 3, 2025
20 checks passed
@gr2m gr2m deleted the spec-v3-tool-result-non-nullable-result branch November 3, 2025 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants