AI chat assistant broken on strict openai-compatible endpoints: "Unrecognized schema" tool-schema conversion failure

[alexcf]

Symptom

With an openai-compatible LLM endpoint configured (llama.cpp / vLLM / LM Studio with structured outputs), the AI chat assistant returns nothing and the browser throws:

Error: JSON schema conversion failed: Unrecognized schema: {"description":"Setting value for the path. Type depends on the path definition."}

A single typeless tool schema aborts the whole tool grammar build, so the assistant produces no response. Categorization/rules/drafts are unaffected (they don't use this tool).

Repro

1. Configure the provider as openai-compatible against any endpoint that builds a constrained-decoding grammar from tool parameters (llama.cpp /v1/chat/completions, vLLM guided decoding, LM Studio structured outputs).
2. Open the assistant chat and send any message.
3. The grammar build fails on the first typeless tool schema → empty response.

Root cause

apps/web/utils/ai/assistant/tools/settings/shared.ts declares the updateAssistantSettings tool's value field as:

value: z.unknown().describe("Setting value for the path. Type depends on the path definition.")

z.unknown() emits a JSON Schema node with no type/form keyword — literally {"description": "..."}. Because the openai-compatible provider is created with supportsStructuredOutputs: true (apps/web/utils/llms/model.ts:272), the tool schema is sent to the endpoint's structured-output path. The strict converter that rejects the typeless node is the LLM server's grammar builder (e.g. llama.cpp json-schema-to-grammar), not an AI-SDK function — the AI SDK passes the tool schema through untouched, and the server error propagates to the browser. Lenient hosted providers (e.g. OpenAI proper) tolerate the typeless node, which is why the bug is endpoint-specific.

Affected configs

Any strict openai-compatible endpoint that constructs a grammar/structured output from the tool JSON Schema — i.e. most self-hosted local-LLM setups.

Proposed fix

Replace z.unknown() with an explicit union of the value shapes the settings paths actually accept (primitives, arrays of primitives, and the existing object schemas). The exact per-path type is still re-validated downstream by parseUpdateAssistantSettingsChanges against the strict settingsChangeSchema discriminated union, so this only needs to be a permissive, fully-typed superset. A naive z.record(...) fix does not work — it emits additionalProperties:false with no properties and silently rejects object values ({title, content} for draft knowledge) at the grammar level.

diff --git a/apps/web/utils/ai/assistant/tools/settings/shared.ts b/apps/web/utils/ai/assistant/tools/settings/shared.ts
@@ export const updateAssistantSettingsInputSchema = z.object({
     .describe("Structured settings changes to apply."),
 });
 
+const settingsPrimitiveValueSchema = z.union([
+  z.string(),
+  z.number(),
+  z.boolean(),
+  z.null(),
+]);
+
+// The value accepted by a settings change depends on the path. Rather than
+// `z.unknown()` (which produces a typeless `{}` JSON Schema node that strict
+// "openai-compatible" endpoints reject when building a constrained-decoding
+// grammar - "Unrecognized schema"), we describe the full set of shapes any path
+// can take. The exact per-path type is still re-validated downstream against
+// `settingsChangeSchema`, so this only needs to be a permissive superset.
+const settingsLlmValueSchema = z.union([
+  settingsPrimitiveValueSchema,
+  z.array(settingsPrimitiveValueSchema),
+  draftKnowledgeUpsertSchema,
+  z.object({
+    title: z
+      .string()
+      .trim()
+      .min(1)
+      .max(200)
+      .describe("Title of the draft knowledge item to delete."),
+  }),
+  scheduledCheckInsConfigSchema,
+]);
+
 export const updateAssistantSettingsLlmChangeSchema = z
   .object({
     path: settingsPathSchema.describe(
       "Writable settings path (use a path returned by getAssistantCapabilities).",
     ),
-    value: z
-      .unknown()
-      .describe(
-        "Setting value for the path. Type depends on the path definition.",
-      ),
+    value: settingsLlmValueSchema.describe(
+      "Setting value for the path: a primitive (string, number, boolean, or null), an array of primitives, or an object whose shape depends on the path.",
+    ),
     mode: z
       .enum(["append", "replace"])
       .nullish()

Validation

Reproduced the real emitted tools[].function.parameters JSON Schema using the pinned SDK versions (ai@6.0.184, zod@4.2.1, @ai-sdk/openai-compatible@2.0.47, @ai-sdk/provider-utils@4.0.27) against a stub server:

• Old schema emits {"description":"Setting value for the path. Type depends on the path definition."} — verbatim the reported error payload, a typeless node.
• New schema emits zero typeless nodes across the whole tool schema (even with the real .refine()/.nullish() chains).
• ajv validation against the emitted schema accepts string / number / boolean / null / array / object.
• Isolated tsc --strict of the union + the consumer's change.value === null access passes; no consumer change required.
• updateAssistantSettings is the only typeless schema in the assistant chat tool set.

Happy to open a PR with this if you'd prefer — wanted to file the bug + fix first.

[alexcf]

Opened a PR with the fix: #2896.