|
1 | | -# web-llm |
| 1 | +Web LLM Starter (React + Vite) |
| 2 | + |
| 3 | +Run small LLMs fully in your browser via WebGPU using WebLLM. |
| 4 | + |
| 5 | +Quickstart |
| 6 | +- Prereqs: Node 18+, Chrome/Edge (WebGPU), good GPU. |
| 7 | +- Install: `npm i` (or `pnpm i`/`yarn`) |
| 8 | +- Dev: `npm run dev` then open the printed URL. |
| 9 | +- Build: `npm run build` and `npm run preview`. |
| 10 | + |
| 11 | +Deploy to GitHub Pages |
| 12 | +- Push this repo to GitHub (default branch `main`). |
| 13 | +- GitHub Actions workflow `.github/workflows/deploy.yml` builds and publishes to Pages. |
| 14 | +- Enable Pages: Repo Settings → Pages → Build and deployment → Source: GitHub Actions. |
| 15 | +- Your site will be at `https://<user>.github.io/<repo>/`. |
| 16 | + |
| 17 | +Notes on hosting |
| 18 | +- Vite `base` is set from `BASE_PATH`. The workflow sets it to `/<repo>/` so assets resolve correctly on Pages. |
| 19 | +- WebGPU does not require COOP/COEP. If you later use WASM multithreading, GitHub Pages cannot set COOP/COEP headers. |
| 20 | + |
| 21 | +Notes |
| 22 | +- First load downloads model artifacts (hundreds of MB). They are cached for subsequent runs. |
| 23 | +- Model menu defaults to tiny options (1–2B params) for smoother UX. Larger models may be slow or fail on low‑VRAM GPUs. |
| 24 | +- If WASM multithreading is required by your browser, consider serving with COOP/COEP headers. |
| 25 | + |
| 26 | +Troubleshooting |
| 27 | +- “WebGPU not supported”: Enable chrome://flags/#enable-unsafe-webgpu or update your browser/OS/GPU drivers. |
| 28 | +- “Failed to compile/initialize”: Try a smaller model, and use the “Clear caches” button to remove stale artifacts. |
| 29 | +- Dev server CORS/isolation: Vite works out of the box for WebGPU. For strict WASM threading you may add COOP/COEP headers via a proxy or plugin. |
| 30 | + |
| 31 | +Project Structure |
| 32 | +- `src/ui/model/useEngine.ts`: Loads models and generates text. |
| 33 | +- `src/ui/model/models.ts`: Model presets. Adjust as needed. |
| 34 | +- `src/ui/chat/*`: Chat state and UI. |
| 35 | +- `src/utils/useIndexedDB.ts`: Minimal IndexedDB helper used to persist chats per model. |
0 commit comments