Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions BLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ By leveraging the raw power of the **Snapdragon X Elite** and its high-performan

## 🚀 The Vision: AI at the Edge

Most AI-integrated IDEs rely on heavy cloud APIs. This introduces latency, subscription costs, and—most importantly—privacy concerns. **Nova IDE** flips the script. It uses **MediaPipe GenAI** and **WebGPU** to run Large Language Models (LLMs) locally.
Most AI-integrated IDEs rely on heavy cloud APIs. This introduces latency, subscription costs, and—most importantly—privacy concerns. **Nova IDE** flips the script. It uses **LiteRT LM** and **WebGPU** to run Large Language Models (LLMs) locally.

When you run Nova IDE on a machine powered by the **Snapdragon X Elite**, you're not just running a web app; you're utilizing one of the most efficient NPU/GPU architectures ever designed for portable computing.

Expand All @@ -29,7 +29,7 @@ graph TD
end

subgraph AI_Runtime ["AI Execution Layer"]
MP["MediaPipe GenAI (WASM)"]
MP["LiteRT LM (WASM)"]
WG["WebGPU API"]
end

Expand All @@ -47,8 +47,8 @@ graph TD
MP -->|Streaming Tokens| UI
```

### 1. The Inference Engine (MediaPipe + WebGPU)
At the heart of Nova IDE is the MediaPipe GenAI runtime. Unlike traditional JavaScript which runs on the CPU, Nova IDE uses **WebGPU** to talk directly to the **Qualcomm Adreno GPU**.
### 1. The Inference Engine (LiteRT LM + WebGPU)
At the heart of Nova IDE is the LiteRT LM runtime. Unlike traditional JavaScript which runs on the CPU, Nova IDE uses **WebGPU** to talk directly to the **Qualcomm Adreno GPU**.
- **WebGPU** allows for massively parallel tensor operations required by transformers.
- On the Snapdragon X Elite, the Adreno GPU provides the floating-point performance needed to generate tokens at lightning speed, rivaling cloud-based solutions.

Expand All @@ -74,9 +74,9 @@ The Snapdragon X Elite is a breakthrough for web-based AI. While Intel and AMD h

When you click **"⚡ Load Local AI Model"** in Nova IDE, the following sequence occurs:

1. **WASM Initialization**: The IDE loads the MediaPipe GenAI WebAssembly runtime.
1. **WASM Initialization**: The IDE loads the LiteRT LM WebAssembly runtime.
2. **GPU Adapter Request**: The browser requests a WebGPU adapter. On an X Elite machine, this identifies the **Qualcomm Adreno GPU**.
3. **Model Loading**: The model (in `.task` format) is fetched into a `SharedArrayBuffer`.
3. **Model Loading**: The model (in `.litertlm` format) is fetched into a `SharedArrayBuffer`.
4. **GPU Compilation**: The model's computation graph is compiled into GPU-specific kernels.
5. **Streaming Inference**: When you type a prompt, the tokens are generated on the Adreno GPU and streamed back to the UI in real-time.

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ⚡ Nova IDE

Nova IDE is a lightweight, browser-based coding environment designed for the future of **on-device AI**. It leverages WebGPU and MediaPipe to run high-performance Large Language Models (LLMs) entirely within your browser—no cloud, no APIs, and total privacy.
Nova IDE is a lightweight, browser-based coding environment designed for the future of **on-device AI**. It leverages WebGPU and LiteRT LM to run high-performance Large Language Models (LLMs) entirely within your browser—no cloud, no APIs, and total privacy.

![Nova IDE Screenshot](src/assets/hero.png)

Expand Down Expand Up @@ -60,7 +60,7 @@ Nova IDE is a lightweight, browser-based coding environment designed for the fut

- **Core**: JavaScript (ESM), HTML5, CSS3
- **Editor**: [CodeMirror 6](https://codemirror.net/)
- **AI Engine**: [MediaPipe GenAI](https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference)
- **AI Engine**: [LiteRT LM](https://ai.google.dev/edge/litert-lm/js)
- **Bundler**: [Vite](https://vitejs.dev/)
- **Storage**: IndexedDB (Virtual File System)

Expand Down
51 changes: 19 additions & 32 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
"@codemirror/lang-python": "^6.2.1",
"@codemirror/lint": "^6.9.6",
"@codemirror/theme-one-dark": "^6.1.3",
"@mediapipe/tasks-genai": "^0.10.27",
"@litert-lm/core": "^0.12.1",
"codemirror": "^6.0.2"
}
}
4 changes: 2 additions & 2 deletions packages/web-agent-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A purely in-browser, WebGPU-accelerated, zero-dependency autonomous ReAct (Reason+Act) Agent Framework.

Built for local-first, privacy-respecting AI applications using MediaPipe GenAI and lightweight models like Gemma.
Built for local-first, privacy-respecting AI applications using LiteRT LM and lightweight models like Gemma.

## Features
- **Zero Backend**: Runs entirely in the browser using WebGPU.
Expand Down Expand Up @@ -41,7 +41,7 @@ const myTools = [
// Your LLM object just needs a \`generateRaw(history, onToken)\` method
const aiEngine = {
generateRaw: async (prompt, onToken) => {
// Call MediaPipe or Transformers.js here
// Call LiteRT LM, Transformers.js, or another local model runtime here
// stream tokens to onToken(token)
}
};
Expand Down
4 changes: 2 additions & 2 deletions packages/web-agent-core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
"agent",
"llm",
"webgpu",
"mediapipe",
"litert-lm",
"react",
"autonomous",
"browser",
Expand All @@ -26,4 +26,4 @@
"publishConfig": {
"registry": "https://npm.pkg.github.com"
}
}
}
4 changes: 2 additions & 2 deletions packages/web-agent-core/website/docs/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ sidebar_position: 1

Welcome to **Nova Web Agent Core**!

This package provides a purely in-browser, WebGPU-accelerated, zero-dependency autonomous ReAct (Reason+Act) Agent Framework. It is built for local-first, privacy-respecting AI applications using MediaPipe GenAI and lightweight models like Gemma.
This package provides a purely in-browser, WebGPU-accelerated, zero-dependency autonomous ReAct (Reason+Act) Agent Framework. It is built for local-first, privacy-respecting AI applications using LiteRT LM and lightweight models like Gemma.

## Why this framework?

Expand Down Expand Up @@ -80,7 +80,7 @@ Your LLM wrapper only needs one method: `generateRaw(prompt, onTokenCallback)`.
```javascript
const aiEngine = {
generateRaw: async (prompt, onToken) => {
// Call MediaPipe, Transformers.js, or even a cloud API here
// Call LiteRT LM, Transformers.js, or even a cloud API here
// stream tokens to onToken(token)
}
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ const FeatureList = [
icon: '🧠',
description: (
<>
Powered by the cutting-edge WebGPU backend and MediaPipe, execute your agent's reasoning loops completely client-side without API keys.
Powered by the cutting-edge WebGPU backend and LiteRT LM, execute your agent's reasoning loops completely client-side without API keys.
</>
),
},
Expand Down
Loading