66[ ![ Platform] ( https://img.shields.io/badge/Platform-Android%20%7C%20Linux%20%7C%20RPi%20%7C%20macOS%20%7C%20Windows-green.svg )] ( )
77[ ![ Status] ( https://img.shields.io/badge/Status-Active-brightgreen.svg )] ( )
88[ ![ Free Forever] ( https://img.shields.io/badge/Free-Forever-brightgreen.svg )] ( )
9- [ ![ Version] ( https://img.shields.io/badge/Version-0.9.0 -blue.svg )] ( )
9+ [ ![ Version] ( https://img.shields.io/badge/Version-0.9.5 -blue.svg )] ( )
1010
1111---
1212
@@ -61,7 +61,7 @@ Then run:
6161llamdrop
6262```
6363
64- That's it. Two commands. No compilation. No configuration. No account needed.
64+ Two commands. No compilation. No configuration. No account needed.
6565
6666---
6767
@@ -71,7 +71,7 @@ That's it. Two commands. No compilation. No configuration. No account needed.
7171- 🔍 ** Full device profiling** — reads RAM, CPU model, core layout (big.LITTLE aware), CPU flags (AVX2/AVX512/NEON), GPU vendor, storage, Android SoC/API level
7272- 🖥️ ** 7-tier classification** — Micro / Low / Low-Mid / Mid / High / Desktop / Workstation — auto-configures everything per tier
7373- 🧠 ** Backend auto-selection** — picks the correct backend for every platform×GPU combination: Termux pkg, CUDA, ROCm, Vulkan, Metal/Ollama, IPEX-LLM, or CPU
74- - ⚡ ** GPU acceleration** — Vulkan for Adreno/Mali/AMD desktop, CUDA for NVIDIA, Metal via Ollama on Apple Silicon — with clear explanations for why GPU is or isn't active
74+ - ⚡ ** GPU acceleration** — Vulkan for Adreno/Mali/AMD desktop, CUDA for NVIDIA, Metal via Ollama on Apple Silicon
7575- 🚫 ** Android GPU safety** — never forces GPU on Android (Mali Vulkan is slower than CPU; Adreno crashes). CPU-only, no guessing.
7676- 👋 ** First-launch Device Profile** — shows detected specs card with tier, backend decision, runtime flags, and model recommendations. Runs once.
7777
@@ -81,48 +81,35 @@ That's it. Two commands. No compilation. No configuration. No account needed.
8181 - 🔎 ** Live HuggingFace search** — search any GGUF model with live RAM estimates
8282- ⬇️ ** Resilient downloader** — auto-resumes on connection drops, retries automatically, verifies via SHA-256 checksum
8383- 🎯 ** Smart quantization** — picks the best Q4/Q2/Q5/IQ variant based on your * live* RAM at download time
84- - 🧩 ** IQ quant support** — IQ3_M and IQ2_M variants for more models — better quality than Q2_K at same RAM. Vulkan auto-disabled for IQ quants (incompatible) .
85- - 📊 ** Benchmark scores** — tokens/second recorded per model (rolling average, last 5 runs), shown in browser as ⚡ X t/s
86- - 🗂️ ** Cancelled download cleanup** — partial files deleted immediately on cancel, never show as valid models
84+ - 🧩 ** IQ quant support** — IQ3_M and IQ2_M variants — better quality than Q2_K at same RAM. Vulkan auto-disabled for IQ quants.
85+ - 📊 ** Benchmark scores** — tokens/second recorded per model (rolling average, last 5 runs)
86+ - 🗂️ ** Cancelled download cleanup** — partial files deleted immediately on cancel
8787
8888### Chat & Inference
89- - 🤖 ** Ollama backend** — auto-detected on Linux/desktop and macOS. Routes inference through Ollama HTTP API when running.
90- - 💬 ** Stable chat** — automatic context trimming prevents out-of-memory crashes. Always preserves your first exchange — only the middle gets trimmed .
89+ - 🤖 ** Ollama backend** — auto-detected on Linux/desktop and macOS, hardware-aware auto-tuning
90+ - 💬 ** Stable chat** — automatic context trimming prevents out-of-memory crashes. First exchange always preserved .
9191- 🦙 ** Live thinking indicator** — animated spinner with non-blocking stdout while the model generates
9292- 🎯 ** Prompt format auto-detect** — correct template per model family (ChatML, Llama3, Gemma, Phi3)
9393- 📂 ** File context** — attach a file to your conversation before chatting
94- - 💾 ** Session save/load/delete** — resume conversations where you left off, with auto-save every 5 exchanges (10 messages)
94+ - 💾 ** Session save/load/delete** — resume conversations where you left off, auto-save every 5 exchanges
9595- 📤 ** Chat export** — ` /export ` saves conversation to Downloads as markdown
96- - 🗂️ ** Conditional mmap** — 15–30% lower peak RAM on internal storage models; external/sdcard keeps ` --no-mmap `
97- - 🧹 ** Clean output pipeline** — llama.cpp banner, duplicate responses, timing stats, and format tags (ChatML boundaries) are all stripped. What you see is only the model's response.
96+ - 🧹 ** Clean output pipeline** — llama.cpp banner, duplicate responses, timing stats, and format tags all stripped
9897
9998### System & UX
100- - ⚠️ ** Live RAM monitor** — colour-coded bar in UI (green/yellow/red), warns if memory gets critical during chat
101- - 🔋 ** Battery monitoring** — shows charge %, per-inference battery drop, warns at configurable low threshold. Distinct icons per charge range.
102- - 📂 ** Phone-wide GGUF scanner** — finds models you already have in Downloads, Documents, etc . Runs in background — UI stays responsive with a live counter .
103- - 🆙 ** Self-update** — ` llamdrop update ` pulls latest version from GitHub (resolves correct install root)
104- - 🩺 ** Doctor** — ` llamdrop doctor ` checks binary, libraries, RAM, storage, network, Python version, Termux permissions, and Ollama status
105- - ⚙️ ** Config file** — override threads, context, temperature, system prompt, auto-save, battery warning threshold at ` ~/.llamdrop/config.json ` . Hot-reloads on external edits.
99+ - ⚠️ ** Live RAM monitor** — colour-coded bar in UI (green/yellow/red)
100+ - 🔋 ** Battery monitoring** — shows charge %, per-inference battery drop, warns at configurable low threshold
101+ - 📂 ** Phone-wide GGUF scanner** — finds models already on your device . Runs in background — UI stays responsive.
102+ - 🆙 ** Self-update** — ` llamdrop update ` pulls latest version from GitHub
103+ - 🩺 ** Doctor** — ` llamdrop doctor ` checks binary, libraries, RAM, storage, network, Python version, Termux permissions, and Ollama status. ` --cleanup ` removes orphaned partial downloads.
104+ - ⚙️ ** Config file** — override threads, context, temperature, system prompt, auto-save, battery threshold at ` ~/.llamdrop/config.json ` . Hot-reloads on external edits.
106105- 🌐 ** Multi-language UI** — English, Hindi, Spanish, Portuguese, Arabic
107- - 🖥️ ** Curses TUI** — keyboard-navigable menu with live RAM bar, battery line, llama.cpp + GPU status, and update notices
108- - ⚡ ** Fast startup** — hardware detection runs exactly once at launch. Startup is noticeably faster on low-end devices.
106+ - 🖥️ ** Curses TUI** — keyboard-navigable menu with live RAM bar, battery line, llama.cpp + GPU status
107+ - ⚡ ** Fast startup** — hardware detection runs exactly once at launch
109108
110109---
111110
112111## Model Catalog
113112
114- llamdrop uses a ** two-layer model system** :
115-
116- ### Layer 1 — Verified Catalog (` models.json ` )
117- A community-maintained list of models ** confirmed to work** on low-RAM devices.
118- Every entry has been tested, has known RAM requirements, and is safe to download.
119- No login or account required.
120-
121- ### Layer 2 — Live HuggingFace Search
122- Search any model on HuggingFace directly from llamdrop.
123- The tool estimates RAM requirements from file size and quantization type.
124- Clearly marked as ** unverified** — for experienced users who want to explore beyond the catalog.
125-
126113** Current verified catalog (41 models across 6 tiers):**
127114
128115| Tier | Available RAM | Example Models |
@@ -145,17 +132,13 @@ The browser automatically hides models outside your device's tier — you only s
145132llamdrop # Launch UI
146133llamdrop update # Update to latest version
147134llamdrop doctor # Check install health
135+ llamdrop doctor --cleanup # Remove orphaned partial downloads
148136llamdrop version # Show version
149137```
150138
151- ** Chat commands:**
152- ```
153- /help — show commands
154- /export — save conversation as markdown
155- /clear — clear history
156- /ram — show current RAM usage
157- /quit — exit chat
158- ```
139+ ** Chat commands:** ` /help ` ` /export ` ` /clear ` ` /ram ` ` /quit `
140+
141+ For full usage guides, see the [ Wiki] ( https://github.com/DeVenLucaz/llamdrop/wiki ) .
159142
160143---
161144
@@ -197,13 +180,16 @@ llamdrop/
197180│ ├── i18n.py # Multi-language UI strings (EN/HI/ES/PT/AR)
198181│ ├── updater.py # Self-update + background catalog updater
199182│ ├── benchmarks.py # Tokens/sec benchmark storage (rolling average, 5 runs)
200- │ ├── doctor.py # Install health checker + Ollama check
183+ │ ├── doctor.py # Install health checker + partial download cleanup
201184│ ├── config.py # User config file with mtime-aware hot-reload
202185│ ├── battery.py # Battery monitoring during inference
203186│ ├── filecontext.py # File attachment for chat context
204187│ └── backends/
205188│ ├── __init__.py # Backends package
206189│ └── ollama.py # Ollama HTTP backend (auto-detected)
190+ ├── tests/
191+ │ ├── test_specs.py # Hardware tiering + device profiling tests
192+ │ └── test_prompts.py # Prompt builder tests (ChatML, Llama3, etc.)
207193└── docs/
208194 ├── CONTRIBUTING.md # How to contribute
209195 └── DEVICES.md # Community device compatibility list
@@ -213,52 +199,21 @@ llamdrop/
213199
214200## Roadmap
215201
216- ### v0.7 — Done
217- - [x] Chip-aware threads — 30+ chips mapped to actual big core count
218- - [x] Fixed context thresholds — 2048–8192 tokens based on device class
219- - [x] Device class detection — ultra_low / low / mid / high / desktop
220- - [x] First-launch welcome screen — detected specs + model recommendations
221- - [x] Ollama backend — auto-detected on Linux/desktop, HTTP API routing
222- - [x] IQ quant support — IQ3_M/IQ2_M, Vulkan auto-disabled
223- - [x] Conditional mmap — 15–30% RAM saving on internal storage models
224- - [x] Clean inference extraction — ` _run_inference() ` / ` _dispatch_inference() `
225- - [x] 25 models in catalog
226-
227- ### v0.8.5 — Done
228- - [x] Full ` DeviceProfile ` dataclass — single source of truth for all device decisions
229- - [x] 7-tier classification (Micro / Low / Low-Mid / Mid / High / Desktop / Workstation)
230- - [x] 80+ SoC chip translation table
231- - [x] GPU-aware install — CUDA / ROCm / Vulkan / CPU binary auto-selected at install time
232- - [x] Native Windows PowerShell installer (` install.ps1 ` )
233- - [x] macOS Homebrew + Ollama install path
234- - [x] IQ quant Vulkan guard — IQ2/IQ3/IQ4 force ` gpu-layers=0 `
235- - [x] SHA-256 binary verification in installer
236- - [x] Config hot-reload on external edits (mtime tracking)
237- - [x] Battery icon levels (🪫 / 🔴 / 🟡 / 🔋)
238- - [x] Arabic language added
239- - [x] Icon-based menu index — adding/removing items no longer shifts handlers
240- - [x] RAM estimate overhead raised to 1.4× for KV cache accuracy
241- - [x] Multiple critical bug fixes (browser content swap, blocking stdout, config cache, auto-save counter, menu index offsets, Vulkan false positives)
242-
243202### v0.9.5 — Current
244- - [x] Device-aware model browser — shows only models that fit your hardware tier and RAM
245- - [x] Catalog expanded from 25 to 41 models across 6 tiers (micro through desktop/workstation)
246- - [x] Chat output pipeline fully cleaned — no llama.cpp banner bleed, no duplicate responses, no leaked timing stats or format tags
247- - [x] Cancelled downloads cleaned up immediately — partial files never appear as valid models
248- - [x] Background GGUF scanner — My Models screen no longer freezes, live counter while scanning
249- - [x] Smarter context trimming — first exchange always preserved, middle trimmed instead of tail
250- - [x] RAM reads consolidated to one shared source across all modules
251- - [x] Hardware detection runs once at startup instead of three separate times
252- - [x] Incremental prompt buffer — full prompt no longer rebuilt from scratch every turn
253- - [x] Noise filter moved to stderr only — model stdout no longer silently corrupted
254- - [x] Auto-save constant named and documented (` _AUTOSAVE_EVERY_TURNS = 10 ` )
203+ - [x] RAM monitoring unified — single source of truth across all modules
204+ - [x] Inference blocking fixed — non-blocking stdout, smooth "Thinking..." animation
205+ - [x] Ollama first-class — hardware-aware auto-tuning, same as llama.cpp path
206+ - [x] Formal test suite added — ` tests/ ` with 6 passing tests
207+ - [x] ` doctor --cleanup ` — detects and removes orphaned partial downloads
208+ - [x] GitHub URL migration — all references updated to DeVenLucaz/llamdrop
255209
256210### v1.0 — Planned
257211- [ ] Web-based model catalog (GitHub Pages)
258212- [ ] Community device profile submissions
259213- [ ] ` /doc ` command — document chat with chunking (no vector DB needed)
260214- [ ] llamdrop server mode — run on phone, access from browser on WiFi
261215- [ ] Streaming tokens via Ollama backend
216+ - [ ] Multiple file context — attach more than one file to a conversation
262217
263218---
264219
@@ -280,7 +235,6 @@ See [CONTRIBUTING.md](docs/CONTRIBUTING.md) for full details.
280235
281236** GNU General Public License v3.0** — see [ LICENSE] ( LICENSE )
282237
283- In plain language:
284238- ✅ Free to use forever
285239- ✅ Free to modify and share
286240- ❌ Cannot be sold
0 commit comments