ML Intern: fixed model set with pinned providers, plus a model probe script

ML Intern conversations now run on ML_ASSISTANT_MODELS, a JSON5 list where each
entry pins the inference provider the model was verified on. Creating a mode
conversation swaps any unlisted model (the router alias included) for the set's
default, switching a mode conversation to an unlisted model is refused, and the
pinned provider overrides the user's per-model preference at request time in
both the conversation route and the parked-turn sweeper. With the pill on, the
Models page and the settings sidebar list only the set.

Why: probing GLM-5.3-Flash on the router showed the same model behaving very
differently by provider at 300k-1M tokens. On baseten it intermittently
fabricated, looped, or emitted garbage and then ended the turn with a normal
finish_reason "stop", which chat-ui shows as a finished answer. Together and
novita answered verbatim at 1,034k tokens. Baseten also silently discards a tool
call whose name is not in the tools array. The router's auto choice landed on
baseten, so pinning is the only way to keep the mode off unverified serving.

scripts/probe-model.ts (npm run probe-model) reproduces the check: it sends the
tool loop's request shape with a generated ML Intern history and grades recall
of verbatim log lines, so candidates can be verified before they are listed.
Chart pins from the 2026-09-02 matrix: GLM-5.3-Flash on together, Kimi-K3 on
baseten, GLM-5.3 on novita.

Author: pngwn

.env

@@ -230,6 +230,14 @@ APP_BASE="" # base path of the app, e.g. /chat, left blank as default
 # set). Enforced server-side: submissions reserve their worst case (flavor
 # price x timeout) and settle to actual runtime.
 ML_ASSISTANT_MODE=
+# Models ML Intern conversations may run on, JSON5 array; first entry is the
+# default. Each entry pins the inference provider the model was verified on for
+# 1M-token tool loops (scripts/probe-model.ts) — the user's provider preference
+# and "auto" are ignored for mode conversations. `parameters` merge over the
+# catalog entry's. The router alias cannot be listed. Empty: the mode refuses
+# to start a conversation.
+# e.g. [{"id":"zai-org/GLM-5.3-Flash","provider":"together"},{"id":"moonshotai/Kimi-K3","provider":"together"}]
+ML_ASSISTANT_MODELS=
 ### Body size limit for SvelteKit https://svelte.dev/docs/kit/adapter-node#Environment-variables-BODY_SIZE_LIMIT
 BODY_SIZE_LIMIT=15728640
 PUBLIC_COMMIT_SHA=

.gitignore

@@ -25,4 +25,5 @@ models/*
 /test-results
 /playwright-report
 /blob-report
-/playwright/.cache
+/playwright/.cache
+.probe-results/

chart/env/dev.yaml

@@ -73,6 +73,12 @@ envVars:
   LLM_ROUTER_ENABLE_TOOLS: "true"
   LLM_ROUTER_TOOLS_MODEL: "moonshotai/Kimi-K2.6"
   TRANSCRIPTION_MODEL: "openai/whisper-large-v3-turbo"
+  # ML Intern model set: provider pinned per model from scripts/probe-model.ts runs
+  # (2026-09-02). GLM-5.3-Flash on together recalled 5/5 at 924k tokens; Kimi-K3 on
+  # baseten 6/6 at 1,046k (fireworks-ai is the alternate: 6/6, slower prefill);
+  # GLM-5.3 degrades above ~800k on every provider — novita was the best of them.
+  ML_ASSISTANT_MODELS: >
+    [{"id": "zai-org/GLM-5.3-Flash", "provider": "together"}, {"id": "moonshotai/Kimi-K3", "provider": "baseten"}, {"id": "zai-org/GLM-5.3", "provider": "novita"}]
   MCP_SERVERS: >
     [{"name": "Web Search (Exa)", "url": "https://mcp.exa.ai/mcp?tools=web_search_exa,get_code_context_exa,crawling_exa"}, {"name": "Hugging Face", "url": "https://hf.co/mcp?login"}]
   MCP_TOOL_TIMEOUT_MS: "120000"

chart/env/prod.yaml

@@ -83,6 +83,12 @@ envVars:
   LLM_ROUTER_ENABLE_TOOLS: "true"
   LLM_ROUTER_TOOLS_MODEL: "moonshotai/Kimi-K2.6"
   TRANSCRIPTION_MODEL: "openai/whisper-large-v3-turbo"
+  # ML Intern model set: provider pinned per model from scripts/probe-model.ts runs
+  # (2026-09-02). GLM-5.3-Flash on together recalled 5/5 at 924k tokens; Kimi-K3 on
+  # baseten 6/6 at 1,046k (fireworks-ai is the alternate: 6/6, slower prefill);
+  # GLM-5.3 degrades above ~800k on every provider — novita was the best of them.
+  ML_ASSISTANT_MODELS: >
+    [{"id": "zai-org/GLM-5.3-Flash", "provider": "together"}, {"id": "moonshotai/Kimi-K3", "provider": "baseten"}, {"id": "zai-org/GLM-5.3", "provider": "novita"}]
   MCP_SERVERS: >
     [{"name": "Web Search (Exa)", "url": "https://mcp.exa.ai/mcp?tools=web_search_exa,get_code_context_exa,crawling_exa"}, {"name": "Hugging Face", "url": "https://hf.co/mcp?login"}]
   MCP_TOOL_TIMEOUT_MS: "120000"

docs/source/configuration/overview.md

@@ -84,6 +84,23 @@ ML_ASSISTANT_MODE=true          # Compile in ML Assistant mode (composer header
 Pass it to `npm run build`, or as a `--build-arg` to `docker build`. Setting it on
 an already-built instance has no effect.
 
+## ML Assistant Models
+
+When the build ships ML Assistant mode, its conversations run on a fixed model set
+rather than whatever the composer has selected. Each entry pins the inference
+provider the model was verified on for long tool loops (`npm run probe-model` runs
+that verification); the user's provider preference and `auto` are ignored for mode
+conversations, and the router alias can't be listed. The first entry is the default.
+
+```ini
+ML_ASSISTANT_MODELS=`[
+  {"id": "zai-org/GLM-5.3-Flash", "provider": "together", "parameters": {"max_tokens": 49152}},
+  {"id": "moonshotai/Kimi-K3", "provider": "together"}
+]`
+```
+
+Leave it empty and the mode refuses to start a conversation.
+
 ## User Authentication
 
 Use OpenID Connect for authentication:

package.json

@@ -20,6 +20,7 @@
 		"updateLocalEnv": "vite-node --options.transformMode.ssr='/.*/' scripts/updateLocalEnv.ts",
 		"populate": "vite-node --options.transformMode.ssr='/.*/' scripts/populate.ts",
 		"config": "vite-node --options.transformMode.ssr='/.*/' scripts/config.ts",
+		"probe-model": "vite-node --options.transformMode.ssr='/.*/' scripts/probe-model.ts --",
 		"prepare": "husky"
 	},
 	"devDependencies": {