Skip to content

Commit 33c855e

Browse files
authored
Update README version and formatting
Updated version number in README and improved formatting.
1 parent bacb783 commit 33c855e

1 file changed

Lines changed: 33 additions & 79 deletions

File tree

README.md

Lines changed: 33 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
[![Platform](https://img.shields.io/badge/Platform-Android%20%7C%20Linux%20%7C%20RPi%20%7C%20macOS%20%7C%20Windows-green.svg)]()
77
[![Status](https://img.shields.io/badge/Status-Active-brightgreen.svg)]()
88
[![Free Forever](https://img.shields.io/badge/Free-Forever-brightgreen.svg)]()
9-
[![Version](https://img.shields.io/badge/Version-0.9.0-blue.svg)]()
9+
[![Version](https://img.shields.io/badge/Version-0.9.5-blue.svg)]()
1010

1111
---
1212

@@ -61,7 +61,7 @@ Then run:
6161
llamdrop
6262
```
6363

64-
That's it. Two commands. No compilation. No configuration. No account needed.
64+
Two commands. No compilation. No configuration. No account needed.
6565

6666
---
6767

@@ -71,7 +71,7 @@ That's it. Two commands. No compilation. No configuration. No account needed.
7171
- 🔍 **Full device profiling** — reads RAM, CPU model, core layout (big.LITTLE aware), CPU flags (AVX2/AVX512/NEON), GPU vendor, storage, Android SoC/API level
7272
- 🖥️ **7-tier classification** — Micro / Low / Low-Mid / Mid / High / Desktop / Workstation — auto-configures everything per tier
7373
- 🧠 **Backend auto-selection** — picks the correct backend for every platform×GPU combination: Termux pkg, CUDA, ROCm, Vulkan, Metal/Ollama, IPEX-LLM, or CPU
74-
-**GPU acceleration** — Vulkan for Adreno/Mali/AMD desktop, CUDA for NVIDIA, Metal via Ollama on Apple Silicon — with clear explanations for why GPU is or isn't active
74+
-**GPU acceleration** — Vulkan for Adreno/Mali/AMD desktop, CUDA for NVIDIA, Metal via Ollama on Apple Silicon
7575
- 🚫 **Android GPU safety** — never forces GPU on Android (Mali Vulkan is slower than CPU; Adreno crashes). CPU-only, no guessing.
7676
- 👋 **First-launch Device Profile** — shows detected specs card with tier, backend decision, runtime flags, and model recommendations. Runs once.
7777

@@ -81,48 +81,35 @@ That's it. Two commands. No compilation. No configuration. No account needed.
8181
- 🔎 **Live HuggingFace search** — search any GGUF model with live RAM estimates
8282
- ⬇️ **Resilient downloader** — auto-resumes on connection drops, retries automatically, verifies via SHA-256 checksum
8383
- 🎯 **Smart quantization** — picks the best Q4/Q2/Q5/IQ variant based on your *live* RAM at download time
84-
- 🧩 **IQ quant support** — IQ3_M and IQ2_M variants for more models — better quality than Q2_K at same RAM. Vulkan auto-disabled for IQ quants (incompatible).
85-
- 📊 **Benchmark scores** — tokens/second recorded per model (rolling average, last 5 runs), shown in browser as ⚡ X t/s
86-
- 🗂️ **Cancelled download cleanup** — partial files deleted immediately on cancel, never show as valid models
84+
- 🧩 **IQ quant support** — IQ3_M and IQ2_M variants — better quality than Q2_K at same RAM. Vulkan auto-disabled for IQ quants.
85+
- 📊 **Benchmark scores** — tokens/second recorded per model (rolling average, last 5 runs)
86+
- 🗂️ **Cancelled download cleanup** — partial files deleted immediately on cancel
8787

8888
### Chat & Inference
89-
- 🤖 **Ollama backend** — auto-detected on Linux/desktop and macOS. Routes inference through Ollama HTTP API when running.
90-
- 💬 **Stable chat** — automatic context trimming prevents out-of-memory crashes. Always preserves your first exchange — only the middle gets trimmed.
89+
- 🤖 **Ollama backend** — auto-detected on Linux/desktop and macOS, hardware-aware auto-tuning
90+
- 💬 **Stable chat** — automatic context trimming prevents out-of-memory crashes. First exchange always preserved.
9191
- 🦙 **Live thinking indicator** — animated spinner with non-blocking stdout while the model generates
9292
- 🎯 **Prompt format auto-detect** — correct template per model family (ChatML, Llama3, Gemma, Phi3)
9393
- 📂 **File context** — attach a file to your conversation before chatting
94-
- 💾 **Session save/load/delete** — resume conversations where you left off, with auto-save every 5 exchanges (10 messages)
94+
- 💾 **Session save/load/delete** — resume conversations where you left off, auto-save every 5 exchanges
9595
- 📤 **Chat export**`/export` saves conversation to Downloads as markdown
96-
- 🗂️ **Conditional mmap** — 15–30% lower peak RAM on internal storage models; external/sdcard keeps `--no-mmap`
97-
- 🧹 **Clean output pipeline** — llama.cpp banner, duplicate responses, timing stats, and format tags (ChatML boundaries) are all stripped. What you see is only the model's response.
96+
- 🧹 **Clean output pipeline** — llama.cpp banner, duplicate responses, timing stats, and format tags all stripped
9897

9998
### System & UX
100-
- ⚠️ **Live RAM monitor** — colour-coded bar in UI (green/yellow/red), warns if memory gets critical during chat
101-
- 🔋 **Battery monitoring** — shows charge %, per-inference battery drop, warns at configurable low threshold. Distinct icons per charge range.
102-
- 📂 **Phone-wide GGUF scanner** — finds models you already have in Downloads, Documents, etc. Runs in background — UI stays responsive with a live counter.
103-
- 🆙 **Self-update**`llamdrop update` pulls latest version from GitHub (resolves correct install root)
104-
- 🩺 **Doctor**`llamdrop doctor` checks binary, libraries, RAM, storage, network, Python version, Termux permissions, and Ollama status
105-
- ⚙️ **Config file** — override threads, context, temperature, system prompt, auto-save, battery warning threshold at `~/.llamdrop/config.json`. Hot-reloads on external edits.
99+
- ⚠️ **Live RAM monitor** — colour-coded bar in UI (green/yellow/red)
100+
- 🔋 **Battery monitoring** — shows charge %, per-inference battery drop, warns at configurable low threshold
101+
- 📂 **Phone-wide GGUF scanner** — finds models already on your device. Runs in background — UI stays responsive.
102+
- 🆙 **Self-update**`llamdrop update` pulls latest version from GitHub
103+
- 🩺 **Doctor**`llamdrop doctor` checks binary, libraries, RAM, storage, network, Python version, Termux permissions, and Ollama status. `--cleanup` removes orphaned partial downloads.
104+
- ⚙️ **Config file** — override threads, context, temperature, system prompt, auto-save, battery threshold at `~/.llamdrop/config.json`. Hot-reloads on external edits.
106105
- 🌐 **Multi-language UI** — English, Hindi, Spanish, Portuguese, Arabic
107-
- 🖥️ **Curses TUI** — keyboard-navigable menu with live RAM bar, battery line, llama.cpp + GPU status, and update notices
108-
-**Fast startup** — hardware detection runs exactly once at launch. Startup is noticeably faster on low-end devices.
106+
- 🖥️ **Curses TUI** — keyboard-navigable menu with live RAM bar, battery line, llama.cpp + GPU status
107+
-**Fast startup** — hardware detection runs exactly once at launch
109108

110109
---
111110

112111
## Model Catalog
113112

114-
llamdrop uses a **two-layer model system**:
115-
116-
### Layer 1 — Verified Catalog (`models.json`)
117-
A community-maintained list of models **confirmed to work** on low-RAM devices.
118-
Every entry has been tested, has known RAM requirements, and is safe to download.
119-
No login or account required.
120-
121-
### Layer 2 — Live HuggingFace Search
122-
Search any model on HuggingFace directly from llamdrop.
123-
The tool estimates RAM requirements from file size and quantization type.
124-
Clearly marked as **unverified** — for experienced users who want to explore beyond the catalog.
125-
126113
**Current verified catalog (41 models across 6 tiers):**
127114

128115
| Tier | Available RAM | Example Models |
@@ -145,17 +132,13 @@ The browser automatically hides models outside your device's tier — you only s
145132
llamdrop # Launch UI
146133
llamdrop update # Update to latest version
147134
llamdrop doctor # Check install health
135+
llamdrop doctor --cleanup # Remove orphaned partial downloads
148136
llamdrop version # Show version
149137
```
150138

151-
**Chat commands:**
152-
```
153-
/help — show commands
154-
/export — save conversation as markdown
155-
/clear — clear history
156-
/ram — show current RAM usage
157-
/quit — exit chat
158-
```
139+
**Chat commands:** `/help` `/export` `/clear` `/ram` `/quit`
140+
141+
For full usage guides, see the [Wiki](https://github.com/DeVenLucaz/llamdrop/wiki).
159142

160143
---
161144

@@ -197,13 +180,16 @@ llamdrop/
197180
│ ├── i18n.py # Multi-language UI strings (EN/HI/ES/PT/AR)
198181
│ ├── updater.py # Self-update + background catalog updater
199182
│ ├── benchmarks.py # Tokens/sec benchmark storage (rolling average, 5 runs)
200-
│ ├── doctor.py # Install health checker + Ollama check
183+
│ ├── doctor.py # Install health checker + partial download cleanup
201184
│ ├── config.py # User config file with mtime-aware hot-reload
202185
│ ├── battery.py # Battery monitoring during inference
203186
│ ├── filecontext.py # File attachment for chat context
204187
│ └── backends/
205188
│ ├── __init__.py # Backends package
206189
│ └── ollama.py # Ollama HTTP backend (auto-detected)
190+
├── tests/
191+
│ ├── test_specs.py # Hardware tiering + device profiling tests
192+
│ └── test_prompts.py # Prompt builder tests (ChatML, Llama3, etc.)
207193
└── docs/
208194
├── CONTRIBUTING.md # How to contribute
209195
└── DEVICES.md # Community device compatibility list
@@ -213,52 +199,21 @@ llamdrop/
213199

214200
## Roadmap
215201

216-
### v0.7 — Done
217-
- [x] Chip-aware threads — 30+ chips mapped to actual big core count
218-
- [x] Fixed context thresholds — 2048–8192 tokens based on device class
219-
- [x] Device class detection — ultra_low / low / mid / high / desktop
220-
- [x] First-launch welcome screen — detected specs + model recommendations
221-
- [x] Ollama backend — auto-detected on Linux/desktop, HTTP API routing
222-
- [x] IQ quant support — IQ3_M/IQ2_M, Vulkan auto-disabled
223-
- [x] Conditional mmap — 15–30% RAM saving on internal storage models
224-
- [x] Clean inference extraction — `_run_inference()` / `_dispatch_inference()`
225-
- [x] 25 models in catalog
226-
227-
### v0.8.5 — Done
228-
- [x] Full `DeviceProfile` dataclass — single source of truth for all device decisions
229-
- [x] 7-tier classification (Micro / Low / Low-Mid / Mid / High / Desktop / Workstation)
230-
- [x] 80+ SoC chip translation table
231-
- [x] GPU-aware install — CUDA / ROCm / Vulkan / CPU binary auto-selected at install time
232-
- [x] Native Windows PowerShell installer (`install.ps1`)
233-
- [x] macOS Homebrew + Ollama install path
234-
- [x] IQ quant Vulkan guard — IQ2/IQ3/IQ4 force `gpu-layers=0`
235-
- [x] SHA-256 binary verification in installer
236-
- [x] Config hot-reload on external edits (mtime tracking)
237-
- [x] Battery icon levels (🪫 / 🔴 / 🟡 / 🔋)
238-
- [x] Arabic language added
239-
- [x] Icon-based menu index — adding/removing items no longer shifts handlers
240-
- [x] RAM estimate overhead raised to 1.4× for KV cache accuracy
241-
- [x] Multiple critical bug fixes (browser content swap, blocking stdout, config cache, auto-save counter, menu index offsets, Vulkan false positives)
242-
243202
### v0.9.5 — Current
244-
- [x] Device-aware model browser — shows only models that fit your hardware tier and RAM
245-
- [x] Catalog expanded from 25 to 41 models across 6 tiers (micro through desktop/workstation)
246-
- [x] Chat output pipeline fully cleaned — no llama.cpp banner bleed, no duplicate responses, no leaked timing stats or format tags
247-
- [x] Cancelled downloads cleaned up immediately — partial files never appear as valid models
248-
- [x] Background GGUF scanner — My Models screen no longer freezes, live counter while scanning
249-
- [x] Smarter context trimming — first exchange always preserved, middle trimmed instead of tail
250-
- [x] RAM reads consolidated to one shared source across all modules
251-
- [x] Hardware detection runs once at startup instead of three separate times
252-
- [x] Incremental prompt buffer — full prompt no longer rebuilt from scratch every turn
253-
- [x] Noise filter moved to stderr only — model stdout no longer silently corrupted
254-
- [x] Auto-save constant named and documented (`_AUTOSAVE_EVERY_TURNS = 10`)
203+
- [x] RAM monitoring unified — single source of truth across all modules
204+
- [x] Inference blocking fixed — non-blocking stdout, smooth "Thinking..." animation
205+
- [x] Ollama first-class — hardware-aware auto-tuning, same as llama.cpp path
206+
- [x] Formal test suite added — `tests/` with 6 passing tests
207+
- [x] `doctor --cleanup` — detects and removes orphaned partial downloads
208+
- [x] GitHub URL migration — all references updated to DeVenLucaz/llamdrop
255209

256210
### v1.0 — Planned
257211
- [ ] Web-based model catalog (GitHub Pages)
258212
- [ ] Community device profile submissions
259213
- [ ] `/doc` command — document chat with chunking (no vector DB needed)
260214
- [ ] llamdrop server mode — run on phone, access from browser on WiFi
261215
- [ ] Streaming tokens via Ollama backend
216+
- [ ] Multiple file context — attach more than one file to a conversation
262217

263218
---
264219

@@ -280,7 +235,6 @@ See [CONTRIBUTING.md](docs/CONTRIBUTING.md) for full details.
280235

281236
**GNU General Public License v3.0** — see [LICENSE](LICENSE)
282237

283-
In plain language:
284238
- ✅ Free to use forever
285239
- ✅ Free to modify and share
286240
- ❌ Cannot be sold

0 commit comments

Comments
 (0)