Skip to content

Replace Agent remote chat with local WebLLM chat#950

Open
azayrahmad wants to merge 2 commits into
mainfrom
feat/local-agent-chat-10240287449385095634
Open

Replace Agent remote chat with local WebLLM chat#950
azayrahmad wants to merge 2 commits into
mainfrom
feat/local-agent-chat-10240287449385095634

Conversation

@azayrahmad

Copy link
Copy Markdown
Owner

This change replaces the legacy remote Vercel-based chat API in the MS Agent application with a fully local inference engine using WebLLM.

Key changes:

  1. WebLLM Integration: Added @mlc-ai/web-llm as a dependency and created a WebLLMService to handle model loading and chat completions.
  2. Personalities: Defined unique persona descriptions for Clippy, Dot, F1, Genius, Office Logo, Mother Nature, Monkey King, Links, and Rocky.
  3. Prompt Engineering: Enhanced the system prompt with character-specific personalities and a Windows 98 feature summary to improve assistant relevance. The prompt enforces a JSON output format to preserve the agent's ability to trigger animations (e.g., Thinking, Explain, Gesture) alongside text.
  4. Performance: Implemented background initialization of the LLM service upon app launch to reduce wait times during the first interaction.
  5. Stability: Fixed a bug where system prompt variables were escaped, preventing correct interpolation of personalities. Corrected the model ID to a valid MLC prebuilt (Qwen2.5-0.5B).
  6. Cleanup: Reverted out-of-scope changes to E2E test ports to maintain consistency with the existing environment.

PR created automatically by Jules for task 10240287449385095634 started by @azayrahmad

- Integrate @mlc-ai/web-llm for local browser-based inference
- Use Qwen2.5-0.5B-Instruct-q4f16_1-MLC model
- Add distinct personalities for all MS Agent characters
- Include Windows 98 system help text in LLM context
- Fix template literal interpolation in system prompt
- Ensure JSON response format for agent animations

Co-authored-by: azayrahmad <10110227+azayrahmad@users.noreply.github.com>
@google-labs-jules

Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

- Integrate @mlc-ai/web-llm for local browser-based LLM inference.
- Use Qwen2.5-0.5B-Instruct-q4f16_1-MLC for high efficiency.
- Define distinct personalities for all MS Agent characters (Clippy, Dot, Genius, etc.).
- Inject Windows 98 system help text into the LLM context for grounded assistance.
- Fix ReferenceError by correctly exporting and importing getAgentMenuItems in the app registry.
- Enforce JSON response format to maintain functional agent animations.
- Optimize performance with background model pre-loading on app launch.

Co-authored-by: azayrahmad <10110227+azayrahmad@users.noreply.github.com>
Base automatically changed from feat/ms-agent-integration-11875959365845684809 to main March 24, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant