Replace Agent remote chat with local WebLLM chat#950
Conversation
- Integrate @mlc-ai/web-llm for local browser-based inference - Use Qwen2.5-0.5B-Instruct-q4f16_1-MLC model - Add distinct personalities for all MS Agent characters - Include Windows 98 system help text in LLM context - Fix template literal interpolation in system prompt - Ensure JSON response format for agent animations Co-authored-by: azayrahmad <10110227+azayrahmad@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
- Integrate @mlc-ai/web-llm for local browser-based LLM inference. - Use Qwen2.5-0.5B-Instruct-q4f16_1-MLC for high efficiency. - Define distinct personalities for all MS Agent characters (Clippy, Dot, Genius, etc.). - Inject Windows 98 system help text into the LLM context for grounded assistance. - Fix ReferenceError by correctly exporting and importing getAgentMenuItems in the app registry. - Enforce JSON response format to maintain functional agent animations. - Optimize performance with background model pre-loading on app launch. Co-authored-by: azayrahmad <10110227+azayrahmad@users.noreply.github.com>
This change replaces the legacy remote Vercel-based chat API in the MS Agent application with a fully local inference engine using WebLLM.
Key changes:
@mlc-ai/web-llmas a dependency and created aWebLLMServiceto handle model loading and chat completions.Qwen2.5-0.5B).PR created automatically by Jules for task 10240287449385095634 started by @azayrahmad