An expanded, feature-rich Bun-native CLI assistant and toolsuite with local & cloud multi-provider LLM support (Ollama, Google Gemini, Anthropic Claude, xAI Grok, Mistral AI, OpenAI ChatGPT), 100% offline local voice processing (STT Speech-to-Text & TTS Text-to-Speech), pure JavaScript Markdown/PDF conversion tools, persistent memory management, Kanban task board, custom character profiles, chat session management, agentic web tools, Telegram & WhatsApp Bridges, OpenWebUI-style Dashboard, and the Ollama CLI Autonomous Coding Engine!
-
π€ Multi-Provider LLM Engine:
- Ollama (Local & Offline):
llama3,mistral,qwen2.5,phi3,llama3.1,deepseek-r1 - Google Gemini:
gemini-2.5-flash,gemini-2.5-pro,gemini-2.0-flash,gemini-1.5-pro,gemini-1.5-flash - Anthropic Claude:
claude-3-7-sonnet-20250219,claude-3-5-sonnet-20241022,claude-3-5-haiku-20241022,claude-3-opus-20240229 - xAI Grok:
grok-2-latest,grok-2-1212,grok-2-vision-1212,grok-beta - Mistral AI:
mistral-large-latest,mistral-medium-latest,mistral-small-latest,codestral-latest,ministral-8b-latest,ministral-3b-latest - OpenAI ChatGPT:
gpt-4o,gpt-4o-mini,gpt-4-turbo,gpt-3.5-turbo,o1,o3-mini - Dynamic provider auto-routing, seamless key management, and real-time streaming responses.
- One shared model setting across CLI chat, web voice, Telegram, WhatsApp, learning, BabyAGI, and coding tools. On a fresh install, the first model returned by Ollama is selected automatically; if Ollama is unavailable, the documented first local model (
llama3) is used until changed. - Ephemeral one-shot Ollama inference (default ON): every local-Ollama turn shells out to the bun-ollama CLI (
bun run <ollama-lite>/src/index.ts run <model> --messages-json '<msgs>'with the model read from settings and full role-labeled history preserved), answers once, then unloads the model so RAM is freed immediately β no persistentllama-serverbetween turns. Toggle on any surface with@ephemeral [on|off|status](or@set-setting ollama_ephemeral 0for the persistentserver_urldaemon). One-shot CLI:bun run ask "Why is the sky blue?"(usessettings.model;--model <name>overrides).
- Ollama (Local & Offline):
-
π 100% Local STT & TTS Voice Processing (Offline):
- Speech-To-Text (STT): Local Whisper ONNX model (
Xenova/whisper-tiny.en) transcribes voice attachments and audio files locally. - Text-To-Speech (TTS): Local MMS-TTS ONNX model (
Xenova/mms-tts-eng) synthesizes text answers into speech audio files (.wav/.ogg). - Automatic local voice note responses for Telegram and WhatsApp voice messages!
- Speech-To-Text (STT): Local Whisper ONNX model (
-
π Dagowan Real-Time WebSocket Voice Bridge & Web UI (
src/voice_socket.js):- Bun-native WebSocket server listening on
ws://127.0.0.1:8767/wsand serving Web UI athttp://127.0.0.1:8767. - Interactive Dagowan dark-mode interface with a glowing reactive voice orb & live audio spectrum visualizer.
- Real-time local Whisper STT transcription, LLM query processing, and local MMS-TTS audio playback in the browser.
- Supports "Hold to Speak" and "Continuous Duplex Voice" conversation modes.
- Security panel (π header button): PIN + OTP locks/unlocks on the shared
webscope β@security-status/@otp-statusstatus checks,@unlock <pin>/@lock,@set-security-pin/@remove-security-pin,@otp-set/@otp-login <code>/@otp-lock/@otp-remove <code>, unlock TTL (security_unlock_minutes, also editable in Engine Settings). All lock/unlock/info messages are generated dynamically from the livesecurity_unlock_minutesvalue (e.g.@help,@security-status,@otp-status, locked replies) β no hard-coded durations. Header lock badge (π/π) updates from server replies; status is auto-queried on connect. - Commands browser (π header button): searchable list of all supported
@commands (security, OTP, models, memory, tasks, coding, MCP, search, cast, voice, β¦) with one-click βΆ Send, plus@help(the manual itself is lock-gated like any control command). The@unlockentry shows the livesecurity_unlock_minutesvalue and refreshes whenever settings sync. - Full
@command execution: the Web bridge dispatches the same tool commands as Telegram/WhatsApp β@make,@create-file,@youtube-play/@play,@spotify-play,@blog,@fact-check,@web-search/@web-fetch/@web-status,@browser-*,@cast-*,@voice-reply, PDF tools, backup/restore,@remember, tasks,@stats,@models,@sessions(plus natural blog publish intents) β so the Web UI and the Chrome side panel actually run tools instead of chatting about them. Command replies are text-only (no TTS) and lock-gated like everywhere else. - Mobile-responsive layout (
voice_interface.htmlmedia queries at 640px/380px: stacked header, scaled orb, viewport-height transcript, full-width 44px touch controls, 16px input to prevent iOS auto-zoom) β the same full-duplex UI on phones and desktops. - Follow-up prompts never stall: turns run concurrently with turn-scoped interruption (an interrupt only cancels turns already in flight, never the next prompt), a superseded turn skips its TTS synthesis instead of blocking the next answer, and any turn failure delivers a visible error reply instead of leaving the orb stuck in "thinking". When Browser TTS is selected, the server skips its own synthesis entirely β the reply text arrives immediately and speech is rendered client-side.
- Launch via CLI command
@web(or@voice-socket,@dagowan) or launch flag./starter.sh --web(orbun run web).
- Bun-native WebSocket server listening on
-
π§© Chrome Extension Companion (
chrome-extension/):- Unpacked MV3 sidebar widget (v1.1.1) talking to the same local bridge (
ws://127.0.0.1:8767/ws, configurable) β chat, lock/unlock, OTP, and quick@commands without opening the voice tab. The Web bridge dispatches the full Telegram/WhatsApp command set (@youtube-play,@spotify-play,@blog,@fact-check,@web-search,@web-fetch,@web-status,@browser-*,@cast-*,@make,@create-file,@remember, tasks,@stats,@models,@sessions, β¦), so panel commands actually execute. Inline@webin a sentence triggers a live search with results injected into the answer. Header lock badge (π/π) tracks PIN/OTP state from server replies. Shares thewebscope, so its lock state matches the Web UI. Toolbar click opens the side panel (popup fallback on older Chrome). - Right-click any selected text β Ask Dagowan about selection (context menu stashes it; the side panel opens and auto-sends).
- Install:
./starter.sh --web, thenchrome://extensionsβ Developer mode β Load unpacked βchrome-extension/. Seechrome-extension/README.md.
- Unpacked MV3 sidebar widget (v1.1.1) talking to the same local bridge (
-
π Ollama CLI Autonomous Coding Engine (
ollama_cli/&src/ollama_coding.js):- Integrated high-performance local AI coding agent toolset built with Bun and Ollama.
@make <prompt>: Autonomously executes multi-step project generation workflows using file tools (read_file,write_file,list_directory,search_files) and host shell execution (run_command) in the configured project path.@create-file <file_path> [content|prompt]: Fast direct file creation or LLM file generation with automatic recursive parent directory creation.@opencli <task>: API-driven OpenAI coding agent β runs a real tool loop (read_file,write_file,list_directory,search_files,run_command) against the OpenAI API, jailed to the project path. NeedsOPENAI_API_KEY(@set-api-key openai <key>); remote surfaces (Telegram/WhatsApp) are owner-gated viaopencli_allowed_chats. CLI/Web UI count as local. Every tool call is audit-logged tologs/opencli.log.@opencode <task>: externalopencodeCLI runner β needs@set-project-path <dir>first (no default) and a model via@opencode-model <provider/model>or opencode's ownopencode.json(no hardcoded default). Owner-gated remotely viaopencode_allowed_chats; audit-logged tologs/opencode.log. First-time setup on any surface:@opencode-setup(checks binary/auth/path/model/permissions with fix commands, ends with an end-to-end test probe) +@opencode-setup-auto-allow [full|safe|review](writes hands-free file permissions into<projectDir>/opencode.json). Sends aRUNNING: "<task>"start message immediately (project/model included) and aFINISHED successfully (<s>)/FAILEDmessage with timing when opencode exits β on CLI, Telegram, and Web UI as two messages; on WhatsApp/chat(single-reply) as one enrichedFINISHEDmessage with the start logged server-side.@change-project-path <path>(or@project-path): Configure and persist custom project output folders (stored inapp_settings.jsonand~/.ollama_cli.json).- Standalone CLI support (
ollama_cli/) with-p, --project-pathflag and interactive REPL commands. - Native support across CLI sessions, Telegram Bot, and WhatsApp Bridge.
-
π API-first playback + blog drafts + fact-check + Chromium fallback (
src/play_api.js,src/blog_api.js,src/factcheck_api.js,src/browser.js):@fact-check <claim> [--limit N] [--source auto|papers|gemini](or@factcheck,@verify): Searches Semantic Scholar for papers (title, abstract, year, citation count, DOI, open-access PDF) to verify a claim. Anonymous access works (~100 requests / 5 min); a free API key (SEMANTIC_SCHOLAR_API_KEY, private 1 req/sec) raises limits. WithGEMINI_API_KEYset, a Gemini verdict (SUPPORTED / REFUTED / MIXED / UNVERIFIABLE) is added on top of the abstracts, or standalone via--source gemini/FACTCHECK_SOURCE.@youtube-play <query|URL>(or@play): Official YouTube Data API v3 by default (YOUTUBE_API_KEY), hands the top video to your system default browser (--no-showskips the handoff). Playwright/Chromium automation is the fallback.@spotify-play <query|URL>(or@spotify): Official Spotify Web API by default (SPOTIFY_CLIENT_ID/SPOTIFY_CLIENT_SECRET), hands the top track/episode to your default browser / Spotify app. Chromium automation is the fallback.@blog <topic>(or@wordpress,@blogger,@post): The active LLM writes a complete post (350-550 words HTML; starter template only if the model is unreachable), then the official WordPress REST / Blogger v3 APIs create a real draft (WORDPRESS_SITE_URL/USERNAME/APP_PASSWORD,BLOGGER_BLOG_ID/ACCESS_TOKEN, or@blogger-authOAuth) and hand the editor to your default browser. Chromium editor automation is the fallback. Natural callout also works:@Parsly, write a blog post about <topic> on @wordpress(oron @blogger).@browser-search/@browser-search-spotify/@browser-open/@browser-read/@browser-shot/@browser-status/@browser-close: Full page interaction toolkit, identical on CLI, Telegram, WhatsApp, and Web UI.- Same tools exposed to the Ollama CLI agent loop (
browser_open,browser_search_youtube,browser_play_youtube,browser_search_spotify,browser_play_spotify,browser_draft_blog,fact_check_papers,browser_read,browser_screenshot). - Safety rails: domain allowlist, auto headless fallback on display-less machines (
BROWSER_HEADLESSoverride), audit log inlogs/browser.log.
-
π Live web search (
src/websearch.js+src/mcp_web_server.js, zero new dependencies):- Mention
@live-webanywhere in a sentence (latest BTC price @live-web) and the turn runs a real-time search whose results are injected as system context β so every provider (local Ollama + OpenAI/Gemini/Claude/Grok/Mistral cloud models) answers with current facts and[1],[2]citations. Works on CLI, Telegram, WhatsApp, Web UI, and the Chrome side panel; works while PIN/OTP-locked (mid-sentence@live-webis chat, not a control command). (@webstays the voice-bridge toggle.) - Mention
@live-geminianywhere in a sentence (explain black holes @live-gemini) for Google AI Mode: Gemini APIgoogle_searchgrounding whenGEMINI_API_KEYis set (AI answer + cited sources), else best-effort keyless Google scrape (google.com/search?udm=50), else DuckDuckGo fallback. Same inline/explicit/session rules as@live-web. - Engines (
autodefault): Brave Search API β Tavily β keyless DuckDuckGo fallback (Instant Answer abstract + HTML results merged, deduped). Explicit@live-web <query>/@live-gemini <query>/@web-search <query> [--provider auto|duckduckgo|brave|tavily|google-ai] [--limit N],@web-fetch <url>(page β plain text),@web-status. - MCP pair: built-in
webserver (web__web_search/web__web_fetch) auto-registered next tolocal; keys via@set-api-key brave|tavily|gemini <key>(orBRAVE_API_KEY/TAVILY_API_KEY/GEMINI_API_KEY). Settings:websearch_enabled(default on),websearch_provider,websearch_max_results(1β10, default 5),websearch_timeout_ms,gemini_search_model(defaultgemini-2.5-flash). Unit-tested (src/websearch.test.js, mocked fetch β no network).
- Mention
-
πͺ‘ Needle On-Device Tool Calling (
src/needle.js+src/needle_bridge.py):- Cactus Needle (8-29 MB automation model,
pip install cactus-needle) for grammar-constrained tool routing, typed extraction, and local embeddings β fully offline after the one-time weight download. @needle <request>: routes to a local tool (remember_memory,register_task,save_note,set_proactive_timer) with a calibrated confidence score; executes only at/aboveneedle_min_confidence(default 0.7), shows below-threshold calls for confirmation, and refuses off-topic input with an empty call list.- Automatic fast-path (
needle_enabled=1, off by default): plain chat turns on CLI / Telegram / WhatsApp / voice that Needle confidently owns are executed locally without waking the main LLM β saving a full LLM roundtrip per local-action turn. Cheap keyword pre-filter β warm daemon route (~4-9 s CPU, ~23 s one-time cold start) β confidence gate β shared executor; refusals and low-confidence turns fall through to the normal LLM flow untouched. The daemon idles out afterneedle_daemon_idle_ms(default 5 min, frees ~150 MB). @needle-extract <JSON schema> <text>,@needle-embed <text>,@needle-tools,@needle-status: typed extraction without a large LLM, embedding debugging, catalog and install/model/threshold/daemon status.- Learning triage: background LLM extraction now skips trivial <3-word turns (one saved LLM call per chitchat turn; the synchronous heuristic pass still runs).
- Self-healing engine fetch: works around upstream wheel-version skew (e.g. package 3.0.5 vs published 3.0.0/3.0.1 engines) by seeding the newest compatible wheel.
- Setup:
pip install cactus-needle(already covered byinstall.sh/install_ubuntu.sh/Dockerfile, which also pre-seed the engine + weights best-effort), then in-app:@needle-status # verify install / model / daemon state @needle remember that I prefer tea # one-shot on-device routing @set-setting needle_enabled 1 # automatic fast-path on every surface - Settings:
needle_enabled,needle_min_confidence,needle_model_path(custom.cact),needle_python,needle_daemon_idle_ms.
- Cactus Needle (8-29 MB automation model,
-
π MCP (Model Context Protocol) Servers (
src/mcp.js, zero new dependencies):- Speaks MCP stdio (newline-delimited JSON-RPC:
initializeβtools/listβtools/call) against any Claude-formatmcp_servers.json({ "mcpServers": { "<name>": { "command": ..., "args": [...], "env": {...}, "disabled": true } } }). - Works on first run: no config file yet β
@mcp-status(or any MCP use) auto-createsmcp_servers.jsonwith the built-inlocalserver (src/mcp_local_server.jsvia Bun β offlineping/read_file/list_directory, paths jailed to the working directory) plus the built-inwebserver (src/mcp_web_server.jsvia Bun β liveweb_search/web_fetch: Brave/Tavily with keys, keyless DuckDuckGo fallback). Existing configs are migrated (missingwebentry added, custom entries untouched). Enabled by default (mcp_enabled: true);npx-based examples inmcp_servers.example.jsonstay disabled. - Agent integration:
@make/@create-filecoding agent merges MCP tools (<server>__<tool>) with built-ins each turn and routes calls through the manager (ollama_cli/src/agent.ts);@mcp-call <server__tool> [json_args]calls any tool manually from CLI / Telegram / WhatsApp. @mcp-status/@mcp-list/@mcp-tools [server]/@mcp-reconnect [server]/@mcp-enable|disable <server>plus settingsmcp_enabled,mcp_config_path,mcp_timeout_ms(default 15 s for coldnpxstarts). Connections lazy-spawn, handshake once, idle-cleaned on@mcp-reconnectand process exit. Unit-tested (src/mcp.test.js, incl. a real stdio roundtrip β no network).
- Speaks MCP stdio (newline-delimited JSON-RPC:
-
πΊ TV Cast, Castor-style (
src/cast/, Bun-only, no new dependencies):- Casts direct video URLs, HLS/DASH playlists, and web pages with embedded video to DLNA/UPnP smart TVs and Chromecast / Android TV β from CLI, Telegram, WhatsApp, and Web UI with one shared parser.
@cast-scanlists DLNA renderers (SSDP multicast) + Chromecasts (mDNS_googlecast+ DIAL SSDP) merged by IP;@cast <url>auto-routes (direct file β as-is, page β headless-Chromium stream extraction, YouTube URL β native YouTube app on Chromecast via DIAL);@cast-url/@cast-player/@cast-pause/@cast-play/@cast-status/@cast-stopcover the rest.- The TV pulls from a local
Bun.serve()relay: byte-proxy withRangesupport for MP4/MKV, per-request ffmpeg MPEG-TS pipe for HLS/MPD on DLNA, native playlist passthrough on Chromecast.ffprobedecides direct vs transcode; no DRM (same limit as Castor). @set-cast-device <name|ip|description-url>selects/pins the target (IP pin uses unicast, so it works across VLANs/Termux where multicast is blocked);@set-setting cast_device_type <auto|dlna|chromecast>forces a protocol.- Needs any recent
ffmpeg/ffprobeonPATH(only basic encode/probe flags are used) and Chromium (bunx playwright install chromium) for page extraction; machine and TV must share a LAN (no guest-WiFi isolation). Unit-tested (src/cast/cast.test.js,src/cast/chromecast.test.js, no network needed).
-
π Pure JavaScript PDF & Markdown Tools:
- Convert Markdown notes in date ranges to PDF files using pure JS (
pdfkit+marked). - Extract text from PDF files back into Markdown notes (
pdf-parse).
- Convert Markdown notes in date ranges to PDF files using pure JS (
-
π Security PIN + π OTP 2FA + π YubiKey Hardware 2FA (
src/security.js+src/otp.js+src/yubikey.js):@set-security-pin <6+ digits>sets a PIN that locks all@control commands on CLI, Telegram, WhatsApp, and Web UI until@unlock <pin>(per-chat unlocks on Telegram/WhatsApp, expiring aftersecurity_unlock_minutes, default 10;@lockre-locks,@security-statusinspects,@remove-security-pinclears). Every lock/unlock/status/info message is rendered dynamically from the livesecurity_unlock_minutessetting (src/security.js:formatUnlockDuration()/unlockDurationInfo();@helpviabuildCommandsHelp(settings)insrc/commands.js, wired into all four surfaces) β changing the setting immediately changes the wording everywhere, including the Web commands browser.@otp-setenrolls TOTP 2FA (Google Authenticator / Authy / 1Password β dependency-free RFC 6238, SHA-1, 30s, 6 digits). While enrolled, the OTP layer sits before@unlock: every surface needs@otp-login <code>first (per-chat logins on Telegram/WhatsApp, same TTL;@otp-lockdrops,@otp-statusinspects,@otp-remove <code>clears and requires both OTP auth and PIN unlock).@yubi-setenrolls a YubiKey hardware key (Yubico OTP β 44-char modhex code on tap: 12-char public id + 32-char encrypted payload). While enrolled, a single@yubi-login <code>is enough to unlock a surface on its own β no PIN/OTP required (per-chat logins on Telegram/WhatsApp, same TTL;@yubi-lockdrops,@yubi-statusinspects,@yubi-remove <code>clears). YubiKey codes are stored only as the 12-char public id (yubikey_idinapp_settings.json, manageable only via@yubi-*).β οΈ YubiKey functionality is in testing phase only.- Regular chatting β including
@Namepersona mentions β always works without the PIN/OTP/YubiKey. The PIN is stored only as a salted SHA-256 hash and the OTP secret (otp_secretinapp_settings.json, manageable only via@otp-*) is never displayed; auth state is in-memory, so a restart always re-locks. Unit-tested (src/security.test.js,src/otp.test.js,src/yubikey.test.js).
-
βοΈ Telegram Bot Integration (src/telegram.js+src/persona_bridge.js):- Long-polling Telegram bot runner.
- On-demand persona polling: on boot only the ACTIVE persona polls Telegram. Other personas wake on explicit request (
@characterswitch,@wake <name>,@Name ...mention routing in CLI/WhatsApp, any Telegram/WhatsApp activity) and auto-sleep afterpersona_idle_timeout_minutes(default 15, active persona never sleeps). Idle sweep runs everypersona_polling_sweep_seconds(default 60). Disable withpersona_polling_on_demand: falseinapp_settings.jsonto poll all personas at once (legacy behavior). - Single-poller guarantees: one token is never polled by two loops at once β concurrent wakes are serialized, double-
start()is rejected,stop()aborts the in-flightgetUpdates,@characterswitches migrate the shared fallback-token runner instead of duplicating it, and duplicate pollers stop themselves on Telegram's 409 conflict instead of retrying. Rate limits (429) are honored viaretry_afterwith exponential backoff, and deliveredupdate_ids are deduped so overlaps never double-reply. - Processes incoming text and voice messages using active character profiles, memory context, and workspace notes.
- Sends text replies and native Telegram voice note responses (
sendVoice).
-
π± WhatsApp HTTP Bridge (
src/whatsapp.js):- Baileys Web Socket bridge with pairing code & QR code web viewer (
http://127.0.0.1:8766/qr, port honorsWHATSAPP_STATUS_PORT). - No duplicate work: only live (
notify) deliveries are answered β history replays on reconnect are ignored β and redelivered stanzas are deduped, so one message never triggers concurrent LLM calls or double replies. - Serialized replies per chat: concurrent deliveries for the same JID are queued (in-order replies, no interleaved session saves); different chats still process in parallel.
- Automatic local Whisper transcription for incoming WhatsApp voice notes and native voice note replies (
ptt: true). - Group
@character-namecallouts: in group chats each persona responds only when explicitly mentioned (e.g.@Parsly what do you think?). Mention several characters and each one replies in turn (@Whiskers and @Parsly discuss this). Matching is case-insensitive and_/-/spaces are equivalent (@Obi Wan==@Obi_Wan==@ObiWan); multi-word names also answer to their first word (@Dungeonβ Dungeon master). 1:1 DMs with an@character-namemention reply as the mentioned persona (otherwise as the active character), and@bot commands (@help,@wake,@play,@blog,@fact-check, β¦ β the full server command list) always bypass the gate. Disable withwhatsapp_mention_only_groups: falseinapp_settings.json(orWHATSAPP_MENTION_ONLY_GROUPS=0); related toggleswhatsapp_strip_mention/whatsapp_mention_prefix(envWHATSAPP_STRIP_MENTION/WHATSAPP_MENTION_PREFIX). - Examples: DM
@Whiskers what do you think?β answered by Whiskers; DMhelloβ answered by the active character; group@Whiskers and @Parsly discuss thisβ one reply per tagged persona. - Self-healing socket process: the Baileys bridge auto-restarts with backoff after crashes (no more silent WhatsApp death);
stop()escalates SIGTERM β SIGKILL so stale sockets never compete with the next launch. Optional/chatauth viaWHATSAPP_BRIDGE_TOKEN(set on both processes; unset keeps loopback trust) plus a 2 MB payload cap.
- Baileys Web Socket bridge with pairing code & QR code web viewer (
-
β± Proactive Chat Timers & Background Check-Ins:
- Configurable proactive chat idle timers in minutes (
@set-proactive-timer <minutes>). - Automatic background activity tracking per chat session (CLI, Telegram, WhatsApp).
- Sends background check-ins when user has been inactive for the set idle duration.
- Status and check interval configuration (
@proactive-status,@set-proactive-check <seconds>).
- Configurable proactive chat idle timers in minutes (
-
π§ Incremental Learning & Persistent Memory Storage (Eliza Cognitive Layer):
- Dual-Tier Storage Architecture: Backed by high-performance SQLite (
data/bot.db) with automatic two-way synchronization tomemory.json. - Per-Character Private Memory Pools: Every character persona owns its own memories via
memories.character_id(src/memory.js,src/db/schema.js). Retrieval, learning extraction,@remember/@correct/@forget/@update-memory, and CLI/Telegram/WhatsApp/voice context injection are all scoped to the active character (plus legacysharedentries).@memories/@memories-allshow an[owner]tag per entry, and@statsbreaks counts down by character β one bot can no longer see another bot's private pool. - Multilanguage Memory Layer (no fixed language list): Every memory is auto-tagged with its turn language (
memories.language,src/language.js) β any ISO 639 code the model supports gets its own pool and per-language file (memory.json+memory.lt.json,memory.en.json,memory.ja.json, β¦) with zero configuration. Each turn auto-detects its language (Unicode-script routing for Japanese/Korean/Chinese/Arabic/Hebrew/Thai/Hindi/Greek/β¦, stopword tables for Latin-script languages, Ukrainian split from Russian) and recalls from its own pool + shared first. High-value cross-language hits surface through an offline keyword bridge (English pivots, importance-gated, score-penalised, labelled[via en memory β translated association]), so pools stay isolated yet associated β@memoriesshows a[lang]tag per entry,@memory-langs/@statsbreak counts down by language, and the model always replies in the user's active language. - Per-Character Chat History: Sessions already store
character_id(chat_sessions.json,src/sessions.js); cross-session past-context lookup (retrieveSessionContext) is now filtered by character, and session save/rename/delete enforce ownership. - Structured Typed Memories: Tracks memories with explicit types (
fact,preference,goal,project,concept,relationship,instruction,correction,observation), confidence levels, importance scores, tags, and lifecycle statuses (active,superseded,forgotten). - Conflict Resolution & Automatic Superseding: Automatically detects contradictions and topic updates, marking superseded memories when facts, preferences, or tools change.
- Versioned Structured Knowledge Base (
src/knowledge.js): Stores modular concepts, definitions, and procedures with semantic versioning (v1,v2, ...), change reasons, and audit trails. - TF-IDF & Token Relevance Search: Advanced ranking with inverse document frequency, importance/confidence boosts, correction priority weighting, and title match boosts.
- Neuromorphic Cognitive Memory (plasticity layer): Additive activation-based re-ranking over TF-IDF retrieval (
src/neuromorphic_memory.js,src/neuromorphic_command.js). Every memory carries an activation level that spikes on retrieval, decays over time (forgetting curve), and is boosted by Hebbian "fire together, wire together" links between memories co-retrieved in the same turn. Activation also weights importance, confidence, and type salience (corrections/projects/goals boosted), so frequently-used memories rise to the top while stale ones fade. Background consolidation promotes repeatedly-activated traces (short-term β long-term) after each turn; pruning marks decayed, idle memories as forgotten. Per-memory traces (activation,last_spike_at,decay_rate) and Hebbian synapses (weight,last_coactivated) live in additivememory_traces/memory_synapsestables indata/bot.dbβ no migration required. Enabled by default; control via@neuromorphic-memory/@nc-mem(status | toggle | consolidate | prune | spikes | help) on CLI, Telegram, WhatsApp, Voice Web, and Dashboard.@statsreports trace/synapse counts and average activation. Works across all surfaces transparently β retrieval results are re-scored without changing existing@remember/@memories/@forgetbehavior. - Background Continuous Learning Pipeline (
src/learning.js):- Instant Deterministic Heuristics: Captures explicit corrections ("Actually, I changed it..."), projects ("I am building..."), preferences ("I prefer..."), and goals in real-time.
- Asynchronous Deep LLM Extraction: Deep post-turn extraction analyzing conversations to discover durable facts, structured knowledge, and explicit corrections without latency overhead. LLM replies are parsed tolerantly (
safeParseLlmJson): strips ```json fences, uses balanced-brace extraction, and auto-repairs1.β `1.0`, `.8` β `0.8`, and trailing commas β so malformed numbers no longer drop the turn as `extraction_error`. - Candidate Rule Proposals & Response Quality Evaluator: Monitors conversational strategies and proposes deterministic patterns.
- Dual-Tier Storage Architecture: Backed by high-performance SQLite (
-
π Kanban Task Board:
- Task statuses: Processing (
p), Ongoing (o), Finished (f). - Automatic task numbering and background reminders (
@set-task-reminder).
- Task statuses: Processing (
-
πΌ Integrated BabyAGI Autonomous Planning:
- A Bun-native classic BabyAGI loop with task execution, task creation, prioritization, and retrieval of prior task results from SQLite-backed vector memory.
- Uses the assistant's selected provider and model, so it works with local Ollama or configured OpenAI, Gemini, Claude, Grok, and Mistral models.
- Persists its function registry, execution history, and task context in
data/babyagi.sqlite.
-
β‘ Memoisation for Expensive Pure Tasks (
src/utils/memoize.js):- Generic
memoize(sync) +memoizeAsync(dedupes in-flight promises, drops rejections) withJSON.stringifykeying, bounded LRU eviction, optional TTL, and.clear()/.delete()/.stats. - Applied to pure hot paths only:
tokenize(memory.js/knowledge.js), provider string helpers (providers.js), per-turn regex gates (needle.js:isNeedleFastPathCandidate/isTrivialTurn), task parsers (tasks.js),parseDate(pdf_utils.js). - LLM / network / DB / filesystem writes are deliberately never memoized.
- Generic
-
π§΅ Threading for CPU-Bound Batch Work (
node:worker_threads):- DB/file reads stay on the main thread (
bun:sqlitehandles are not transferable); only scoring/rendering fans out. src/utils/concurrency.js:availableParallelism()/chunkArray()/mapWithConcurrency()/getOptimalWorkerCount()(CPU-bounded pool sizing).src/utils/retrieval_worker.js+src/utils/parallel_retrieval.js: TF-IDF/word-overlap scoring for memories/knowledge/sessions runs in workers above thresholds (100/100/300 docs), sync fallback below or on worker failure.MemoryRetriever.searchParallel,KnowledgeRetriever.searchParallel,retrieveSessionContextParallel.src/chat_context.js:retrieveChatContextParallel()resolves memories + knowledge + past sessions concurrently viaPromise.all; wired intoindex.jschat loop with sync fallback.src/utils/pdf_worker.js+src/utils/pdf_parallel.js: per-note MarkdownβPDF renders andpdf-parseextracts run in parallel workers (convertNotesToPdf(..., { useWorkers, concurrency }),convertPdfToMd(..., { useWorkers }),convertPdfsToMdBatch()); merged batch PDF stays single-threaded.- Deliberately not threaded: memoized microsecond string helpers, single LLM
unifiedChatcalls, SQLite writes, session JSON saves.
- DB/file reads stay on the main thread (
-
π Character Profiles & System Prompts:
- Custom character personas in
characters.json. - Configurable system prompt templates and per-character LLM models.
- Persona templates control tone and emotion (warmth, humor, expressiveness) plus per-character
temperatureβ this is what makes a bot feel emotional, not the Needle tool-router (which only picks local tools and never writes replies). - Each persona gets a private memory pool and private chat sessions (shared/legacy memories stay visible to all; ownership is shown as
[owner]in@memories).
- Custom character personas in
-
π Simulation Mode (simple separate chats, same persona):
- Start with
@simulation <name> [optional note], end with@simulation-end(or@simulation-stop,@simulation-passed,@simulation-pass,@simulation end|stop|passed). - Each simulation is just normal chat with the active persona and owns its own archive file in
data/simulations/<name>.json(chats + memories stay separate from the global memory DB). - While idle, relevant past-simulation entries are recalled automatically when the prompt matches (
@simulation-search <query>for manual search). Full transcript review via@simulation-review <name>. Works on CLI, Telegram, WhatsApp, and web voice.
- Start with
-
πͺ Daily Self-Reflections (
src/reflections.js, separate bank indata/reflections.json):- Run
@insight [optional focus]and the active LLM writes today's reflection (3-6 bullets: what mattered, what it learned about the user, what to remember tomorrow, one thing to improve) β one entry per character per day (re-running the same day refreshes it). Falls back to a deterministic template when the LLM is unreachable, so the command never fails silently. - Manage with
@insights [N](or@insight-list),@insight-search <query>,@insight-show <id|YYYY-MM-DD>,@insight-delete <id|YYYY-MM-DD>,@insight-status. Per-character scoped like memories; unit-tested insrc/reflections.test.js. - Later turns recall relevant reflections by association and inject them as
[Relevant Self-Reflections]alongside memories/knowledge/sessions (src/chat_context.js+ all chat pipelines), deepening existing knowledge over time. - Adaptive auto-insight (
src/auto_insight.js, ON by default): after every completed exchange on CLI / Telegram / WhatsApp / voice / Dashboard, a background gate counts substantive turns (β₯24 chars, not trivial chitchat, not@commands). Every N turns (default 8) + cooldown (default 30 min, max 6/day) it writes a 1-2 bullet micro self-observation (source:auto-insight) into the same bank β never blocks the reply. Control with@insight-auto [on|off|status|every <n>|cooldown <mins>]; settingsinsight_auto_enabled / insight_auto_every_n / insight_auto_cooldown_minutes / insight_auto_min_chars / insight_auto_max_per_day. Unit-tested insrc/auto_insight.test.js. - Lock-gated like any control command: needs
@otp-login(+@unlock <pin>when a PIN is set) on CLI, Telegram, WhatsApp, Web UI/voice, Dashboard, and Chrome side panel (via the shared Web bridge); regular chatting always works while locked.
- Run
-
πΎ Backup & Restore State:
- Export/import full application state, settings, memories, tasks, sessions, and simulations (
@export-settings,@import-settings).
- Export/import full application state, settings, memories, tasks, sessions, and simulations (
Recent improvements to make the assistant noticeably smarter without changing providers:
- Hybrid retrieval + MMR diversity (
src/memory.js,src/chat_context.js): trigram semantic similarity, recency/importance weighting, and maximal-marginal-relevance dedup across memories, knowledge, and past sessions. - Stricter learning extraction (
src/learning.js): schema validation, confidence/importance clamping, and dedup/conflict handling for LLM-extracted memories and knowledge. - Context budgeting + summaries (
src/context_budget.js): compact summaries of older turns keep long conversations inside the context window. - Clever chat-load sorting (
src/context_budget.js,src/nsga.js): deduplicates repeated turns, prioritises high-value/recent context via stable priority sort, and applies non-dominated (NSGA-II) multi-objective sorting to trim prompts to a token budget before each API call β reducing repeated processing of information the model has already seen. - Needle-aware model routing (
src/model_router.js): per-turn task classification (chat/coding/reasoning/factual/creative/local action) to pick a better model without changing the shared default. - Personalization profile (
src/personalization.js): durable preferences, instructions, goals, and corrections are injected as a stable user-profile block.
The repository includes universal self-healing installation scripts and runtime patches (patch_onnx.js) designed to run on the majority of devices, shells, and operating systems out of the box (Full support for Zsh and Bash on macOS, Ubuntu, Debian, Arch Linux, Fedora, Android Termux / PRoot-Distro, VPS, and Cloud VMs).
Run the universal installer (install.sh), which automatically detects your shell (Zsh or Bash), package manager, sets up system tools, installs Node.js & Bun, configures shell profiles (~/.zshrc, ~/.zprofile, ~/.bashrc), installs dependencies (root bun install + isolated npm install --prefix whatsapp_bridge for the Baileys bridge + Python packages incl. cactus-needle for on-device @needle routing, with best-effort engine/weights pre-seed), and applies runtime patches:
# Set executable permissions
chmod +x install.sh starter.sh
# Run installer (supports direct execution, Zsh, or Bash)
./install.sh
# or explicitly using Zsh:
zsh install.sh
# or using Bash:
bash install.shTip for Zsh Users (macOS / Linux): After installation, reload your shell profile to ensure Bun and Node are available in your current terminal:
source ~/.zshrc # or: exec zsh
- macOS (Apple Silicon M-series & Intel): Native Zsh & Bash support, automatic Homebrew path detection (
/opt/homebrew/bin&/usr/local/bin), and audio/system tool setup. - Ubuntu & Debian (Desktop / Server / VPS - 20.04, 22.04, 24.04 LTS): Automatically detects
sudoprivileges, installslibvips-dev,ffmpeg,espeak, Python tools, Node.js 20, and Bun. (A dedicated./install_ubuntu.shorzsh install_ubuntu.shis also available). - Android (PRoot-Distro Ubuntu / Debian / Termux): Installs APT packages, Node.js 20, Bun runtime, avoids native gyp compilation hurdles (
--ignore-scripts), and applies ONNX WASM & Sharp fallbacks. - Arch Linux / Manjaro: Detects
pacmanand installs required build and runtime dependencies. - Fedora / RHEL / CentOS: Detects
dnfand configures necessary dependencies. - Windows 10/11 (native): Every
.shscript has a tested.battwin (see below) β no WSL required.
Each shell script ships with a Windows batch equivalent (same name, .bat extension). The .sh files are untouched and still used on Unix; the .bat files are verified working on Windows (Bun 1.4 + Node 24 + Python 3.13).
install.bat :: full setup: winget deps (Git, Node 20, Python, ffmpeg),
:: Bun check, pip packages, bun install, WhatsApp bridge deps,
:: ONNX/sharp patches, Playwright Chromium
starter.bat :: launch the app (same flags: --web, --whatsapp)
update.bat :: fetch + pull --rebase origin main (refuses mid-rebase/merge,
:: detached HEAD; prints rebase recovery steps on conflict)
git.bat :: stage + timestamped commit + pull --rebase + push
:: (same guards + conflict-marker / JSON validation)
dockerize.bat :: docker build/run/up/down/logs/shell/clean (needs Docker Desktop)
install_ubuntu.bat :: WSL wrapper that runs install_ubuntu.sh inside Ubuntu
docker-entrypoint.bat :: Windows-side equivalent of docker-entrypoint.shinstall.bat
starter.bat --webNotes:
install.batinstalls missing tools viawingetwhen available, otherwise prints manual install links. A fresh terminal picks upffmpegonPATH(the installer also wires it for the current session).- The WhatsApp bridge's
git+sshdependency is refused by newernpm(EALLOWGIT), so the scripts fall back tobun install --cwd whatsapp_bridge(this createswhatsapp_bridge/bun.lock, untracked). git.batis named like the tool itself: inside this folder plaingitwould re-invoke the script, so it (andupdate.bat) callgit.exeexplicitly β do the same in your own commands here.
node_modules/ is gitignored, so hand-edits there vanish on every fresh install. All runtime patches live as code in patch_onnx.js (single source of truth) and are re-applied automatically on every install path (postinstall, install.sh / install_ubuntu.sh, starter.sh, Docker build + entrypoint). Re-runs are idempotent and never stack duplicates.
What it patches and why:
@xenova/transformersdist +onnxruntime-nodebackend β silence verbose ONNX logging, prefer the WASM backend under Bun.@xenova/transformersimage utils / onnx backend β optionalsharpimport andonnxruntime-webbackend so Termux / PRoot / ARM machines without native binaries still run.sharpβ fallback stub when the native libvips binary is missing.pdf-parse,@napi-rs/canvasβ suppress canvas polyfill warnings and tolerate missing native bindings under Bun.
bun run patch # (re-)apply patches
bun run patch:check # read-only verify; exits 1 if a patch is missing
# (e.g. after a dependency upgrade changed upstream sources)To add a patch, append a patchFile(...) entry with exact search β replace strings; if replace contains search, set skipIfPresent so re-runs stay idempotent. The WhatsApp bridge (whatsapp_bridge/, baileys/qrcode only) needs none of these patches.
Start the assistant via the startup wrapper:
./starter.sh
# or if using Zsh:
zsh starter.shOn Windows (cmd.exe) use the batch twin β same flags:
starter.bat
starter.bat --web- Interactive CLI Chat:
./starter.sh # or zsh starter.sh # or bun start
- Real-Time Voice Web UI (Dagowan WebSocket Interface):
Then open
./starter.sh --web # or with Zsh: zsh starter.sh --web # or: bun run web
http://127.0.0.1:8767in your browser. The web bridge supports typed prompts, browser-recorded audio, local/OpenAI transcription, and speech replies.bun testincludes a no-network integration check for all three paths. - OpenWebUI-Style Dashboard (
src/dashboard.js+dashboard.html):Then open./starter.sh --dashboard # single-dash -dashboard works too # or: bun run dashboard # custom port: ./starter.sh --dashboard-port 8899 # or DASHBOARD_PORT=8899
http://127.0.0.1:8768in your browser. Sidebar chat list, model + persona pickers, quick@tools, settings panel, and a REST API (/api/health,/api/chat,/api/sessions,/api/characters,/api/memories,/api/tasks,/api/stats,/api/settings,/api/models,/api/commands). Chat turns reuse the normal memory + knowledge + session-context LLM flow. Toggle from inside the app with@dashboard [port]/@dashboard off(@dashalias). Header buttons open the enabled popups: π Avatar (/avatar), π€ AI Friend (/aifriend), π€π¬ Bud Pro (/aifrienddash); the voice toggle fans AI replies out to the avatar / AI Friend popup for lipsync speech. Shares thewebPIN/OTP lock scope with the voice UI (@lock/@unlock, mutating REST endpoints return423while locked). Env:DASHBOARD_ENABLED=1to autostart,DASHBOARD_PORTfor the port (settings:dashboard_enabled,dashboard_portinapp_settings.json). Unit-tested (src/dashboard.test.js, no network). - Avatar lipsync popup (
avatar.html, π pumpkin theme):Then open./starter.sh --avatar # implies --dashboard; --no-avatar always winshttp://127.0.0.1:8768/avatar(or the Avatar button in the dashboard header) and press π Enable audio. The popup is the only audio player β dashboard AI replies are fanned out to it (BroadcastChannel + storage fallback) and spoken with sample-accurate lipsync via server TTS (POST /api/tts, local MMS-TTS or OpenAI), with automatic browser-voice fallback when server synthesis is slow or unreachable. Slow local-TTS turns no longer drop the connection (serveridleTimeoutraised β previously surfaced asnet::ERR_EMPTY_RESPONSE). - AI Friend avatar popup (
aifriend.html, π€ AI Bud Pro character):Then open./starter.sh --aifriend # implies --dashboard; --no-aifriend always wins # or: AIFRIEND_ENABLED=1 bun run dashboard
http://127.0.0.1:8768/aifriend(or the π€ AI Friend button in the dashboard header) and press π Enable audio. Vanilla-JS port ofai_bud_pro/src/LiveCharacter3D.tsx(Three.js cat-eye 3D character + speech bubble, idle/listening/thinking/speaking states, mouth driven by the AI Bud Pro animation curve + live server-TTS audio volume). Like/avatar, it is the only audio player β dashboard AI replies fan out to it over bothdagowan-avataranddagowan-aifriendchannels (+ localStorage fallback), so--avatarand--aifriendpopups are interchangeable. Status atGET /api/aifriend-status. - AI Friend dashboard (
aifrienddash.html, π€π¬ AI Bud Pro chat):Then open./starter.sh --aifrienddash # implies --dashboard; --no-aifrienddash always wins # or: AIFRIENDDASH_ENABLED=1 bun run dashboard
http://127.0.0.1:8768/aifrienddash(or the π€π¬ Bud Pro button in the dashboard header). Vanilla-JS port ofai_bud_pro/src/main.tsxwired to the dagowan dashboard REST API (/api/chat,/api/sessions,/api/characters,/api/memories,/api/models,/api/settings) instead of Ollama directly: sessions, rename/delete, search across conversations + memories, persona/model pickers, memory manager, theme/font prefs, export, browser speech recognition, optional read-aloud with mini 3D avatar, and a β pop-out to/aifriend. Status atGET /api/aifrienddash-status. Combine flags to serve everything at once:./starter.sh --dashboard --avatar --aifriend --aifrienddash. - With WhatsApp Bridge (opt-in; stays off unless a phone is linked or it is enabled):
Configure once with
./starter.sh --whatsapp # or with Zsh: zsh starter.sh --whatsapp@set-whatsapp-phone <number>then@whatsapp-enabled on; link via the QR page or@whatsapp-pair <number>(numeric code β required on Termux, which cannot scan its own screen). - Run Unit & Tool Tests:
bun test - Run BabyAGI from the terminal:
bun run babyagi -- "Create a practical launch plan for an offline voice assistant" --iterations 4 - One-shot ephemeral answer (model from settings, RAM freed afterwards):
bun run ask "Why is the sky blue?" bun run ask --model smollm2:135m "Why is the sky blue?"
Ship the whole toolsuite as a portable container with Dockerfile + docker-compose.yml and drive it via dockerize.sh. The image bundles the Bun runtime, Node.js (WhatsApp Baileys bridge), Python helpers, ffmpeg/espeak, and all JS dependencies with the ONNX patches pre-applied. Running Portainer? Use portainer-stack.yml and the step-by-step PORTAINER.md guide (named volumes only, no host bind mounts).
| Inside container | Host URL | Purpose |
|---|---|---|
8765 |
http://127.0.0.1:8765 |
WhatsApp HTTP bridge (/chat, /health) |
8766 |
http://127.0.0.1:8766 |
WhatsApp QR / pairing view (/qr) |
8767 |
http://127.0.0.1:8767 |
Dagowan voice Web UI + WebSocket (/ws) |
8768 |
http://127.0.0.1:8768 |
OpenWebUI-style dashboard (/ + /api/*, opt-in via DASHBOARD_ENABLED=1 / --dashboard) |
- Docker 24+ with the Compose v2 plugin (
docker compose version). - An Ollama daemon either on the host (
ollama serve, default) or via the bundled compose profile. Cloud models work too (API keys only, no Ollama needed).
cp .env.example .env
# then edit .env: OLLAMA_HOST, OPENAI_API_KEY / GEMINI_API_KEY / ...chmod +x dockerize.sh
./dockerize.sh build
./dockerize.sh run # interactive CLI (+ --web; adds --whatsapp when WHATSAPP_ENABLED=1 / WHATSAPP_PHONE is set), Ctrl+C to stop
./dockerize.sh run --detach # background mode
./dockerize.sh logs -f # follow logs
docker attach dagowan # re-attach to the interactive CLI./dockerize.sh up # app only, uses host Ollama at host.docker.internal:11434
./dockerize.sh up --ollama # also starts bundled `ollama/ollama` daemon
# first time with bundled Ollama, pull a model once:
docker exec -it dagowan-ollama ollama pull llama3
./dockerize.sh down- Host Ollama (default): keep
OLLAMA_HOST=http://host.docker.internal:11434and runollama serveon the host. On native Linux add--add-host=host.docker.internal:host-gateway(already set indockerize.sh/ composeextra_hosts). - Bundled Ollama:
./dockerize.sh up --ollamawithOLLAMA_HOST=http://ollama:11434. - ollama-lite (bun-ollama) backend: any Ollama-compatible server on
settings.server_url(defaulthttp://127.0.0.1:11434) works, includingollama-lite serve/ollama-lite daemon(they share one port + PID file, so run only one). Use the canonical model namellama3.2:1b(ollama:llama3.2:1bis accepted as an alias;llama:3.2:1bwith the extra colon is invalid). Note: local CPU inference on small hosts can take 60β130s+ per turn β if replies come back empty, make sure ollama-lite is updated past theidleTimeout: 255server fix and restarted. - Cloud-only (no Ollama): set any provider key and switch models in-app (
@set-api-key gemini ...,@model gemini-2.5-flash).
Bind mounts keep host copies of ./notes, ./pdfs, ./projects, ./backups, ./logs, ./tmp; named volumes keep data/bot.db (dagowan-data), WhatsApp auth (dagowan-wa-auth), and root *.json state (dagowan-config, synced by the entrypoint so app_settings.json, characters.json, memory.json, chat_sessions.json, tasks.json survive rebuilds). When a host *.json file exists next to dockerize.sh, run bind-mounts it directly for live editing. Use @export-settings (lands in ./backups) for portable backups.
WhatsApp is opt-in: the container starts --web by default and adds --whatsapp only when WHATSAPP_ENABLED=1 or WHATSAPP_PHONE is set. Pairing UX is controlled by WHATSAPP_QR_MODE=auto|qr|pairing|off (auto = numeric code on Termux, QR elsewhere; Termux cannot scan a QR with its own camera, so use @whatsapp-pair <phone> / POST /pair).
./dockerize.sh shell # shell inside the running container
./dockerize.sh stop / ./dockerize.sh start / ./dockerize.sh rm
./dockerize.sh clean # remove container + image + data volumes (host bind dirs untouched)Deploy with portainer-stack.yml (named volumes only β no host bind
mounts, so it works from the Portainer UI) following PORTAINER.md:
repository stack for auto-build, or build + push once and paste the stack
with build: removed. WhatsApp stays opt-in there too
(WHATSAPP_ENABLED / WHATSAPP_PHONE, WHATSAPP_QR_MODE).
Note: servers bind
0.0.0.0inside the container (viaBIND_HOST) so published ports are reachable; on bare metal they still answer on127.0.0.1as before. Ports/URLs can be overridden withWHATSAPP_PORT,WHATSAPP_STATUS_PORT,VOICE_PORT/WEB_PORT,OLLAMA_HOST/OLLAMA_URL.
You can set provider API keys via environment variables, a .env file, or using @ commands inside the application.
Create a .env file in the root directory:
OPENAI_API_KEY=sk-proj-...
GEMINI_API_KEY=AIzaSy...
CLAUDE_API_KEY=sk-ant-...
GROK_API_KEY=xai-...
MISTRAL_API_KEY=...
SEMANTIC_SCHOLAR_API_KEY=... # optional, free; raises @fact-check rate limits
FACTCHECK_SOURCE=auto # @fact-check engine: auto|papers|gemini (Gemini verdict needs GEMINI_API_KEY)
BRAVE_API_KEY=... # optional; upgrades live web search engine (auto-selected when set)
TAVILY_API_KEY=... # optional; upgrades live web search engine (auto-selected when set)
GEMINI_SEARCH_MODEL=gemini-2.5-flash # @live-gemini Google AI Mode grounding model (needs GEMINI_API_KEY)
WEBSEARCH_PROVIDER=auto # live web search engine: auto|duckduckgo|brave|tavily|google-ai (DuckDuckGo needs no key)
WEBSEARCH_MAX_RESULTS=5 # live web search results per search (1-10)
FACTCHECK_GEMINI_MODEL=gemini-2.5-flashInside the running application:
@set-api-key openai sk-proj-...
@set-api-key gemini AIzaSy...
@set-api-key claude sk-ant-...
@set-api-key grok xai-...
@set-api-key mistral ...
@set-api-key semanticscholar ...
@set-api-key brave ... # optional; upgrades live web search engine
@set-api-key tavily ... # optional; upgrades live web search engine
@web(or@voice-socket,@voice-server,@dagowan) β Launch/toggle real-time duplex voice web interface (http://127.0.0.1:8767). Voice-only; live web search uses@live-web/@web-search(see π Live Web Search below).@dashboard [port]/@dashboard off(or@dash) β Launch/toggle OpenWebUI-style chat dashboard (http://127.0.0.1:8768; custom port via@dashboard 8899,DASHBOARD_PORT, or--dashboard-port N).@voice <audio_file_path>β Transcribe audio file locally (Whisper STT), process question with LLM, and synthesize response.@tts <text>β Synthesize text locally into a speech.wavfile using local MMS-TTS model.@voice-reply [on|off](or@voicereply) β Toggle automatic local voice note response generation for answers.
- Mention
@live-webanywhere in a sentence (latest BTC price @live-web) β the turn runs a real-time search and injects the results into the model's context, on CLI, Telegram, WhatsApp, Web UI, and the Chrome side panel. - Mention
@live-geminianywhere (explain black holes @live-gemini) β Google AI Mode search (Geminigoogle_searchgrounding withGEMINI_API_KEY, else keyless Google scrape, else DuckDuckGo fallback), same context injection as@live-web. @live-web <query>/@live-gemini <query>/@web-search <query> [--provider auto|duckduckgo|brave|tavily|google-ai] [--limit N](or@websearch,@search-web) β Explicit search; results print as chat and persist to the session. (@webstays voice-only.)@web-fetch <url>(or@webfetch) β Fetch a page as plain text.@web-statusβ Show engine configuration (provider chain, key status).- Engines: Brave β Tavily (with
@set-api-key brave|tavily <key>) β keyless DuckDuckGo; Google AI Mode via@live-gemini/--provider google-ai(needs@set-api-key gemini <key>for grounded answers, else keyless scrape + DDG fallback). MCP pair:web__web_search/web__web_fetch. Settings:websearch_enabled,websearch_provider,websearch_max_results,websearch_timeout_ms,gemini_search_model.
@modelsβ List all available models across local Ollama, OpenAI, Gemini, Claude, Grok, and Mistral.@model [model-name]β View or switch the single shared model (e.g.,@model gemini-2.5-flash,@model claude-3-5-sonnet-20241022,@model grok-2-latest,@model mistral-large-latest,@model gpt-4o,@model llama3). The change is applied to CLI chat, web voice, Telegram, WhatsApp, learning, BabyAGI, personas, and coding tools.@ephemeral [on|off|status](or@one-shot) β Toggle one-shot Ollama inference via the bun-ollama CLI (default ON: every Ollama turn loads the model, answers once, then unloads it so RAM is freed; OFF uses the persistentserver_urldaemon with the model resident). Same command on CLI, Telegram, WhatsApp, web voice, and dashboard. Settings:ollama_ephemeral,ollama_lite_dir(auto-detected: setting βOLLAMA_LITE_DIRβollama-liteonPATHβ sibling checkout),ollama_ephemeral_timeout_ms(default 180000),ollama_ephemeral_strip. Note: the@makecoding agent loop always needs the persistent endpoint for its multi-step tool calls.@set-api-key <provider> <api_key>(or@set-key <provider> <key>) β Set API key foropenai,gemini,claude,grok,mistral,youtube,spotify,wordpress(<site_url> <username> <app_password>),blogger(<blog_id> <access_token>),semanticscholar(optional free key, raises@fact-checkrate limits),brave/tavily(optional, upgrade live web search β DuckDuckGo works keyless).@api-keys(or@keys) β View status and source of configured provider API keys, plus the@play/@spotify,@blog,@fact-check, and live web search engine status.
@change-project-path <path>(or@project-path) β Set and persist project/code output directory for@makeand@create-file(saved inapp_settings.jsonand~/.ollama_cli.json).@make <project_description>β Execute Ollama CLI agentic toolset (read/write files, list dir, run shell commands, grep search, local browser) to build complete projects in the configured project path.@create-file <file_path> [content|prompt]β Create or generate a file using Ollama CLI toolset in the project path (auto-creates missing parent directories).@opencli <task_description>β API-driven OpenAI coding agent: real tool loop (read/write/list/search/run, defaultgpt-4o-mini) jailed to the project path. Safety rails: path jail (absolute/..escapes refused),run_commandforced to project cwd with secrets scrubbed from env + destructive patterns refused, iteration cap (opencli_max_iterations, default 12), audit log (logs/opencli.log). Telegram/WhatsApp are owner-gated (@set-setting opencli_allowed_chats ["<chat_id_or_jid>"]); CLI/Web UI are local. Disable everywhere with@set-setting opencli_enabled 0.@opencode <task_description>β Run the externalopencodeCLI (opencode run --format json <task>, scoped to the project dir via spawn cwd plusPWD=<projectDir>β the CLI resolves its session directory fromPWD, not the process cwd, so without the override files land in the bot's own directory; no path positional, seeopencode run --help). Requires@set-project-path <dir>first (no silent default); model is never defaulted β set it with@opencode-model <provider/model>or configure it in opencode itself (<projectDir>/opencode.json/~/.config/opencode/opencode.json). Opt-in--autoapprovals via@set-setting opencode_auto 1, timeout viaopencode_timeout_ms(default 180000 ms), binary override viaopencode_bin/OPENCODE_BIN, audit log (logs/opencode.log). Telegram/WhatsApp are owner-gated (@set-setting opencode_allowed_chats ["<chat_id_or_jid>"]); CLI/Web UI are local. Disable everywhere with@set-setting opencode_enabled 0.@opencode-model [<model>]β Show (no args) or set the@opencodemodel: user setting first, else the model from the project/globalopencode.json, else opencode's own default.@opencode-setup [status|path <dir>|model <m>|model clear|models|auth|bin <path>|auto on|off|auto-allow [full|safe|review|status|clear]|test [task]]β Guided opencode setup on any surface: checks binary/auth/project-path/model with fix commands (@opencode-setup path ./projects/my-app,@opencode-setup model ollama/qwen3:8b, β¦) and finishes with an end-to-end@opencode-setup testprobe.@opencode-setup-auto-allow [full|safe|review|status|clear](alias for@opencode-setup auto-allow β¦) β Write auto-allow file permissions into<projectDir>/opencode.jsonso opencode can create/add files and review code without prompting (full = file ops + bash allow; safe = file ops allow, bash asks; review = read-only).
@babyagi <objective> [--iterations N]β Run the integrated classic BabyAGI loop. It executes the next task, creates relevant follow-up tasks, reprioritizes the queue, and uses prior results as context.Nis 1β20 (default 3).@babyagi-statusβ Display registered BabyAGI agents plus persisted execution-log and vector-memory counts.bun run babyagi -- "<objective>" --iterations Nβ Run the same loop non-interactively.
@mcp-statusβ Show MCP enabled state, configured servers, and connection state (auto-createsmcp_servers.jsonwith the built-in offlinelocal+ live-searchwebservers on first run).@mcp-listβ List configured MCP servers frommcp_servers.json.@mcp-tools [server]β List tools from connected MCP servers (agents address them as<server>__<tool>, e.g.local__read_file).@mcp-call <server__tool> [json_args]β Call one MCP tool directly, e.g.@mcp-call local__ping,@mcp-call local__read_file {"file_path":"package.json"}.@mcp-reconnect [server]β Drop cached connections (lazy re-spawn on next use).@mcp-enable <server>/@mcp-disable <server>β Toggle a server inmcp_servers.json.- Add any MCP stdio server to
mcp_servers.json(seemcp_servers.example.json); the@makecoding agent picks its tools up automatically. Settings:mcp_enabled,mcp_config_path,mcp_timeout_ms.
@convert-pdf [from_date] [to_date](or@pdf 2026-07 2026-08) β Batch convert Markdown notes to PDF files inpdfs/.@pdf-to-md <filename.pdf>(or@convert-md notes.pdf) β Extract PDF text into Markdown note files innotes/.@fetch-note-about <topic>(or@write-about <topic>) β Search notes for topic and synthesize response.
@register-task <description>(or@task <desc>) β Register a new task into Processing status (p).@get-tasks(or@tasks) β List active (unfinished) tasks.@all-tasksβ List all tasks including finished.@move-task-<id>-(p|o|f)(or@move-task <id> <p|o|f>) β Move task status (p: processing, o: ongoing, f: finished).@delete-task-<id>(or@delete-task <id>) β Delete task by ID.@set-task-reminder <minutes>β Enable automatic background task reminders.@stop-task-reminderβ Disable task reminders.@task-reminder-statusβ Show current task reminder configuration.
@set-proactive-timer <minutes>(or@proactive-timer <mins>) β Set proactive chat idle timer in minutes (and enable proactive chat).@proactive-chat [on|off](or@proactive [on|off]) β Enable, disable, or toggle proactive chat check-ins.@proactive-status(or@proactive-timer-status) β View current proactive chat timer & check-in settings.@stop-proactive-timer(or@stop-proactive) β Disable proactive chat check-ins.@set-proactive-check <seconds>(or@proactive-check <secs>) β Set background check frequency in seconds (default 60s).
@remember <text>β Save a persistent memory for the ACTIVE character only (each bot has a private pool + shared legacy entries; turn language auto-detected).@memories [query](or@memory,@list-memories) β List recent active persistent memories (15) for the ACTIVE character (shows[owner][lang]per entry) or search with types and confidence scores (query language auto-detected: same-language pool + shared first, other languages via bridge).@memories-all [query](or@memory-all,@list-memories-all) β List ALL active persistent memories for the ACTIVE character, or search across all.@memory-langsβ Show per-language memory pool counts.@forget <id|index>(or@delete-memory) β Deactivate/forget a specific memory by ID or index (only within the active character's visible pool).@update-memory <id|index> <new text>β Update memory content (only within the active character's visible pool).@neuromorphic-memory / @nc-mem [status|toggle|consolidate|prune|spikes|help]β Neuromorphic Cognitive Memory control plane: plasticity (activation/decay/Hebbian) re-ranking of memory retrieval (enabled by default; works on CLI, Telegram, WhatsApp, Voice Web, Dashboard).@correct <statement>β Submit an explicit user correction for the ACTIVE character (superseding conflicting prior beliefs in its pool).@knowledge [query](or@list-knowledge) β List or search knowledge base concepts & entries.@teach <title: content>β Explicitly teach the bot a structured concept or definition.@knowledge-history <id>β View version history of a knowledge concept.@delete-knowledge <id>β Deprecate/remove a knowledge entry.@stats(or@learning-stats) β Display database, memory, knowledge, and learning statistics (includes a per-character memory breakdown + neuromorphic trace/synapse counts when plasticity is enabled).@learning-logs [limit]β View recent incremental learning events log.@set-learning [on|off](or@auto-learning) β Toggle automatic background LLM learning extraction.@rules [status]β Inspect candidate and approved pattern rules.@approve-rule <id>β Approve a candidate rule.@insight [optional focus]β Write today's daily self-reflection with the active LLM intodata/reflections.json(one per character per day; lock-gated; works on CLI, Telegram, WhatsApp, Web, Dashboard).@insights [N](or@insight-list) β List recent self-reflections for the ACTIVE character.@insight-search <query>β Search the reflection bank by association.@insight-show <id|YYYY-MM-DD>β Show one reflection entry.@insight-delete <id|YYYY-MM-DD>β Delete one reflection entry.@insight-statusβ Show reflection count and latest entry for the active character.@insight-auto [on|off|status|every <n>|cooldown <mins>]β Background micro self-memory: every N substantive turns + cooldown, stored assource:auto-insight(enabled by default; same lock-gating / surfaces as@insight).
@character [name|id]β View or switch active character persona profile (wakes its Telegram polling on demand).@charactersβ List available character profiles.@polling-status(or@polling,@pollers) β Show per-persona Telegram polling state (active always polls; idle personas auto-sleep afterpersona_idle_timeout_minutes).@wake <name|id>β Start Telegram polling for a persona on demand.@sleep <name|id>β Stop polling for an idle persona (active persona cannot sleep).@set-polling-idle <minutes>β Set idle auto-sleep timeout for non-active personas (0disables auto-sleep; saved toapp_settings.jsonaspersona_idle_timeout_minutes).@create-character <Name> | <Prompt>β Create a custom character persona using the shared model.@set-persona-prompt <prompt>β Set system prompt template for active character.@set-telegram-token <token>β Save Telegram bot token to character profile.@whatsapp-pair <phone_number>β Generate WhatsApp 8-digit phone pairing code.@set-whatsapp-phone <phone_number>(or@whatsapp-number) β Set default WhatsApp phone number.@reset-whatsapp-authβ Clear stale WhatsApp auth keys to reconnect.
@simulation <name> [optional note]β Start/resume a simulation; normal chat with the active persona, saved to its own archive indata/simulations/.@simulation-end(or@simulation-stop,@simulation-passed,@simulation-pass,@simulation end|stop|passed) β Archive & end the active simulation.@simulation(bare) β Show active simulation status.@simulations(or@simulation-list) β List simulation archives.@simulation-search <query>β Search across simulation archives by association.@simulation-show <name>β Show a simulation archive summary.@simulation-review <name>(or@simulation review <name>) β Full review of ALL conversational data in a saved simulation (stats + topics + every turn).- While a simulation is active, chats and
@remember/@correct/@memoriesstay inside the simulation file; past archives are recalled automatically when the prompt matches.
@sessionsβ List saved conversation sessions for the ACTIVE character.@session <id>β Switch to session by ID (only within the active character's sessions).@new-sessionβ Start a fresh chat session for the ACTIVE character.@delete-session <id>β Delete session by ID (only within the active character's sessions).@rename-session <title>β Rename active session.- Past-conversation context injected into prompts is filtered by character, so bots never see each other's chat history.
@export-settings [filename](or@backup) β Export application state, memories, knowledge, tasks, and sessions tobackups/.@import-settings [filename](or@restore) β Restore application state from a backup file.@list-backupsβ List available backup files.
@set-security-pin <6+ digits> [current-pin](or@set-pin) β Set or change the PIN (digits only, min 6). First-time setup needs no PIN; changing it needs@unlockor the current PIN passed inline. Setting a PIN unlocks the current surface.@unlock <pin>β Unlock control commands on this surface for the configured duration (security_unlock_minutes, default 10,@set-setting security_unlock_minutes <1-1440>while unlocked; the@helpmanual,@security-status,@otp-status, and all lock replies always print the live value). When OTP is enrolled,@otp-loginis required before@unlock.@lockβ Lock immediately (PIN layer; OTP auth is dropped separately with@otp-lock).@security-status(or@security,@pin-status) β Show whether a PIN/OTP is set and whether this surface is unlocked.@remove-security-pin [pin](or@clear-security-pin) β Remove PIN protection (needs unlock or the current PIN).@otp-set [account]β Enroll a new authenticator secret (Google Authenticator / Authy / 1Password β standard TOTP, no extra install). Needs@unlockfirst when a PIN is set; rotating an existing secret needs@otp-login+@unlock. Prints the secret +otpauth://URL to add in your app, then verify with@otp-login.@otp-login <6-digit code>β Authenticate this surface with the current authenticator code (CLI / Telegram / WhatsApp / Web log in separately; per-chat on Telegram/WhatsApp; same TTL as the PIN).@otp-lockβ Drop OTP authentication on this surface.@otp-status(or@otp) β Show whether OTP is enrolled and whether this surface is authenticated.@otp-remove <code>β Remove OTP protection; requires both OTP authentication (code may be passed inline) and@unlockwhen a PIN is set.- While a PIN or OTP is set, every
@control command on CLI, Telegram, WhatsApp, and Web UI stays locked until the required logins succeed β including@help/@commands/@get-commands(the command manual is only shown after@otp-login, plus@unlockwhen a PIN is set; natural blog publish intents are locked too). Regular chatting β including@Namepersona mentions β always works without the PIN/OTP. The PIN is stored only as a salted SHA-256 hash (security_pin_hash/security_pin_saltinapp_settings.json); the OTP secret (otp_secret, manageable only via@otp-*, never via@set-settingor the Web settings event) is never displayed. Auth state is in-memory and expires, so a restart always re-locks. Lost PIN recovery: stop the bot and delete those two keys fromapp_settings.jsonlocally; lost OTP recovery: stop the bot and deleteotp_secretfromapp_settings.jsonlocally (then re-enroll with@otp-set).
@yubi-set [44-char code|12-char id]β Stage a YubiKey enrollment (tap the key and pass the 44-char code, or its 12-char public id; needs@unlockfirst when a PIN is set; rotating needs@yubi-login+@unlock). Nothing is stored until you confirm with@yubi-set confirm(or@yubi-confirm); cancel with@yubi-set cancel. Confirmation auto-authenticates this surface. Bind a PIN with@yubi-set pin <6+ digits>(or pass it inline) and confirm to enable automode (auto-generate 44-char codes + auto-login on every surface).β οΈ YubiKey functionality is in testing phase only β use at your own risk.@yubi-login <44-char code>β Authenticate this surface with a fresh YubiKey code (CLI / Telegram / WhatsApp / Web log in separately; lasts the unlock TTL).@yubi-lockβ Drop YubiKey authentication on this surface.@yubi-status(or@yubi) β Show enrollment/auth state.@yubi-remove <code>β Remove YubiKey protection (needs@yubi-loginauth; PIN unlock too when a PIN is set; code may be passed inline). While a YubiKey is enrolled, authenticating with it alone unlocks control commands β no PIN/OTP required.- YubiKey setup (configuration):
- Install the
yubdependency (already inpackage.json):bun install(adds theyubpackage for optional Yubico cloud verification). - Enroll: Touch or insert your YubiKey (depending on what your key supports) to generate a 44-character code, then run
@yubi-set <code>(or pass only the 12-character public id). Review the pending public id and confirm with@yubi-set confirm(or@yubi-confirm) to store it; cancel with@yubi-set cancel. For a demo/offline id use@yubi-setwith no args (auto-generates a 44-character code). - Verify login: Tap again for a fresh code and run
@yubi-login <code>on each surface you use (CLI, Telegram, WhatsApp, Web). Each surface logs in separately and stays unlocked forsecurity_unlock_minutes. - Daily use: Tap the key and send
@yubi-login <code>whenever a surface is locked. Other factors (PIN/OTP) are not required once a YubiKey is enrolled and authenticated. - Lock/drop:
@yubi-lockto require a fresh tap;@yubi-statusto inspect;@yubi-remove <code>to unenroll (pass the code inline if not already authenticated). - Storage: Only the 12-character public id (
yubikey_id) plus metadata (yubikey_set_at,yubikey_label,yubikey_last_otpfor replay protection), optional per-key PIN (hashed,yubikey_pin_*), automode flag (yubikey_automode), and any pending enrollment state (yubikey_pending_id,yubikey_pending_label,yubikey_pending_at,yubikey_pending_pin_*) is stored inapp_settings.jsonunder theyubikey_*keys β managed only via@yubi-*, never via@set-setting. The device secret never leaves the key. Auth state is in-memory and expires, so a restart always re-locks. Lost-key recovery: stop the bot and delete theyubikey_*keys fromapp_settings.json, then re-enroll. When Yubico cloud credentials (yubico_client_id+yubico_api_key) are configured, OTP verification is delegated to theyublibrary instead of a hand-rolled client. - Automode (optional): Bind a PIN to the enrolled key with
@yubi-set pin <6+ digits>(or pass it inline during staging:@yubi-set <code> <pin>), then@yubi-set confirm. Automode auto-generates 44-char codes and auto-logs in every surface;@yubi-lockforces a fresh login,@yubi-login <code>clears a manual lock.
- Install the
β οΈ Manual YubiKey data clearing (if commands fail or for emergency recovery):- Stop the bot (
exit/quitor Ctrl+C). - Open
app_settings.jsonin a text editor. - Delete or clear these keys:
yubikey_id,yubikey_set_at,yubikey_label,yubikey_last_otp,yubikey_pending_id,yubikey_pending_label,yubikey_pending_at,yubikey_pin_hash,yubikey_pin_salt,yubikey_pin_set_at,yubikey_pending_pin_hash,yubikey_pending_pin_salt,yubikey_automode(set tofalseor remove). - Save the file and restart the bot. The YubiKey enrollment is now fully cleared.
- (Optional) If you configured Yubico cloud credentials, also clear
yubico_client_idandyubico_api_key.
- Stop the bot (
@fact-check <claim> [--limit N] [--source auto|papers|gemini] [--model <gemini-model>] [--no-verdict](or@factcheck,@verify) β Search Semantic Scholar (src/factcheck_api.js) for papers (title, abstract, year, citation count, DOI, open-access PDF) to verify a claim. Anonymous access works; setSEMANTIC_SCHOLAR_API_KEY(or@set-api-key semanticscholar <key>) to raise rate limits. WithGEMINI_API_KEY(or@set-api-key gemini <key>) a Gemini verdict is added on top (--source geminifor verdict only,--no-verdictfor papers only). Tune with@set-setting factcheck_max_results <1-10>/@set-setting factcheck_source <auto|papers|gemini>. Also available to the coding agent asfact_check_papers.@needle <request>β On-device routing via cactus-needle (src/needle.js+src/needle_bridge.py): picks a local tool (remember / task / note / timer) with grammar-guaranteed args + confidence; executes only at/aboveneedle_min_confidence(default 0.7,@set-setting needle_min_confidence <0-1>), otherwise shows the proposal for confirmation. Needspip install cactus-needle(weights download once, then offline); custom model via@set-setting needle_model_path <file.cact>.@needle-extract <JSON schema> <text>β Typed field extraction without a large LLM (refuses rather than inventing fields).@needle-embed <text>/@needle-tools/@needle-statusβ Local embedding debug, on-device catalog, install/model/threshold status.@youtube-play <query|YouTube URL> [--no-show](or@play) β API-first playback (YouTube Data API,YOUTUBE_API_KEY): resolves the top video via API and hands it to your system default browser. Chromium automation is the fallback when no key is set or the API fails. Force engine withPLAY_SOURCE=auto|api|browser(@set-setting play_source <mode>).@spotify-play <query|Spotify URL> [--no-show](or@spotify) β API-first playback (Spotify Web API,SPOTIFY_CLIENT_ID/SPOTIFY_CLIENT_SECRET): resolves the top track/episode via API and hands it to your default browser / Spotify app. Chromium automation is the fallback.@blog <topic> [--on wordpress|blogger] [--title <title>] [--no-show](or@wordpress,@blogger,@post) β API-first blog draft (WordPress REST API,WORDPRESS_SITE_URL/WORDPRESS_USERNAME/WORDPRESS_APP_PASSWORD; Blogger API v3,BLOGGER_BLOG_ID/BLOGGER_ACCESS_TOKEN): the active LLM writes a complete post (350-550 words HTML; starter template only if the model is unreachable), creates a real draft post and hands the editor to your system default browser. Chromium editor automation is the fallback when no tokens are set or the API fails. Force engine withBLOG_SOURCE=auto|api|browser(@set-setting blog_source <mode>). Natural callout also works in chat/bridges:@Parsly, write a blog post about <topic> on @wordpress(oron @blogger).@browser-search <query>β YouTube search in the local browser window, top results without playing.@browser-search-spotify <query>β Spotify search in the local browser window, top results without playing.@browser-open <url>β Navigate the local browser window (allowlisted domains only).@browser-readβ Read text of the current page.@browser-shot [name]β Screenshot the current page intotmp/.@browser-status/@browser-closeβ Inspect / shut down the browser.@search-youtube-for <query>β Search YouTube in default host browser (legacy fire-and-forget open).@search-google-for <query>β Search Google in default browser.@search-giphy-for <query>β Search Giphy in default browser.@open-browserβ Open system web browser.
@youtube-play/@play and @spotify-play/@spotify resolve via the official APIs by default (src/play_api.js) and hand the URL to your system default browser β no Chromium needed. The previous Playwright/Chromium automation (src/browser.js) is now the fallback when API keys are missing or the API request fails (force with PLAY_SOURCE=auto|api|browser or play_source in app_settings.json).
- YouTube: needs a YouTube Data API v3 key (
YOUTUBE_API_KEYenv or@set-api-key youtube <key>; enable "YouTube Data API v3" in Google Cloud Console). Search callsyoutube/v3/search(part=snippet,type=video, top result). Direct watch/shorts/youtu.be URLs skip the API (title via oEmbed) and just hand off. - Spotify: needs app credentials (
SPOTIFY_CLIENT_ID/SPOTIFY_CLIENT_SECRETenv or@set-api-key spotify <id> <secret>; from developer.spotify.com dashboard). Uses client-credentials token +api.spotify.com/v1/search(track,episode,album,playlist,show,artist), preferring track/episode matches. Directopen.spotify.com/.../spotify:...URLs skip the API and just hand off. - Fallback: the assistant drives a real, visible local browser window and hands videos off to your system default browser (
src/browser.js, lazy-loaded Playwright): - One shared browser across CLI
@commands, Telegram / WhatsApp bridges, and the Ollama CLI agent loop (browser_open,browser_search_youtube,browser_play_youtube,browser_search_spotify,browser_play_spotify,browser_draft_blog,browser_read,browser_screenshotinollama_cli/src/tools.ts), so local models can chain search β click β read like the coding tools chain file ops. All surfaces share one@play/@youtube-playparser (parsePlayInput), one@spotify/@spotify-playparser (parseSpotifyPlayInput), one@blog/@wordpress/@blogger/@postparser (parseBlogInput), and reply format. - YouTube flow:
@youtube-play lofi hip hopopensresults?search_query=β¦in the visible window, clicks the firstytd-video-renderercard, waits for the watch page and asserts the<video>element is actually playing β then hands the watch URL to your system default browser (so it plays where you look) and pauses the controlled copy to avoid double audio. Pass--no-showto skip the handoff. - Spotify flow:
@spotify-play <song|podcast>opensopen.spotify.com/search/β¦, prefers a track/episode match over same-name albums/playlists (with a reload retry when the logged-out render only yields a playlist), clicks the matching result card, clicks the first visible play button (retried once) and verifies playback via the header button flipping to "Pause" / a live media element β then hands the Spotify URL to your default browser / app where you're logged in for full playback and pauses the controlled copy. The handoff is verified (exit-checked,$BROWSERhonored first, thenxdg-open/open/fallbacks), so if the host browser can't be reached you'll get aDefault-browser handoff failed: β¦line instead of silence β the URL stays visible in the controlled Chromium window in that case. After a verified handoff the controlled window auto-closes (--no-showkeeps it open); the reply notesControlled window closed.Unlike YouTube, the opened Spotify page never autoplays on load, so the reply reportsState: playingvsloaded (not playing yetβ¦)plus a login hint when Spotify shows its signup wall. Full-track playback in the controlled window requires a Spotify login indata/browser-profile/; direct URLs (open.spotify.com/...orspotify:...URIs) skip the search. - Blog flow:
@blog <topic> [--on wordpress|blogger](aliases@wordpress,@blogger,@post) writes a complete post with the active LLM (starter template only if the model is unreachable), then creates the draft via the official APIs by default (src/blog_api.js) and hands the editor URL to your system default browser β no Chromium needed.- WordPress: needs site + user + Application Password (
WORDPRESS_SITE_URL/WORDPRESS_USERNAME/WORDPRESS_APP_PASSWORDenv or@set-api-key wordpress <site_url> <username> <app_password>; generate the password in WP Admin β Users β Profile β Application Passwords). Self-hosted sites create a realdraftviaPOST {site}/wp-json/wp/v2/posts(authentication). WordPress.com hosted sites don't expose/wp-jsonat the site domain (404), so the code falls back to XML-RPCwp.newPostat{site}/xmlrpc.phpwith the same credentials. Either way hands off thewp-admin/post.php?post=<id>&action=editURL. - Blogger: needs blog id + OAuth2 access token (
BLOGGER_BLOG_ID/BLOGGER_ACCESS_TOKENenv or@set-api-key blogger <blog_id> <access_token>; Blogger writes require OAuth β a plain API key is read-only and cannot create drafts). Easiest path: drop yourclient_secret_*.apps.googleusercontent.com.jsonin the project folder and run@blogger-auth(CLI) β see "Blogger OAuth" below. Creates a draft viaPOST blogger/v3/blogs/{blogId}/posts?isDraft=true. - Fallback: with no tokens (or on API failure) the assistant opens the WP
wp-admin/post-new.php(orwordpress.com/post) / Blogger posts page in the visible local window, pre-fills title + starter body best-effort, and hands the editor URL to your default browser where you log in and save/publish β same verified-handoff + auto-close pattern as@play. Blogger uses thedraft.blogger.comhost, opening the blog's posts page (/blog/posts/<blogId>, "New post" is one click away) whenblogger_blog_idis set (@set-api-key blogger <blog_id> <access_token>saves it even without a token). Force engine withBLOG_SOURCE=auto|api|browser(@set-setting blog_source <mode>). Only drafts are ever created β nothing auto-publishes. If yourapp_settings.jsonpredates this feature and the browser reportsURL not allowed by browser allowlist, addwordpress.com,blogger.com,blogspot.comtobrowser_allowed_domains(orBROWSER_ALLOWED_DOMAINS). - Callouts:
@Parsly, write a blog post about <topic> on @wordpress(oron @blogger) is detected in CLI/Telegram/WhatsApp/Web UI (detectBlogIntent) and drafts directly, bypassing the group mention gate like@play; the Ollama agent loop gets the same capability via thebrowser_draft_blogtool (which also accepts fulltitle/content).
- WordPress: needs site + user + Application Password (
The Blogger API needs a user OAuth token β no client-credentials flow exists. Setup (one time):
- Google Cloud β APIs & Services β Credentials β create an OAuth client ID (any type; enable the Blogger API v3 for the project) and download its
client_secret_*.apps.googleusercontent.com.jsoninto the project folder (gitignored, never committed). - In the same credential, add this exact Authorized redirect URI:
http://127.0.0.1:8791/callback(fixed loopback port; override withBLOGGER_AUTH_PORTonly if you also update the registration). Copy it character-for-character: Google compares URIs exactly, solocalhostvs127.0.0.1, a trailing slash, or a wrong port all fail withError 400: redirect_uri_mismatch. After saving, wait a minute or two for the change to propagate. (If you already registered thelocalhostvariant instead, setBLOGGER_AUTH_HOST=localhost/@set-setting blogger_auth_host localhostand retry β the flow will use it.) - Run
@blogger-auth [optional path to the json]β approve in your browser. Access + refresh tokens are stored inapp_settings.json(blogger_access_token,blogger_refresh_token,blogger_token_expiry) and refreshed automatically (including a one-retry refresh on HTTP 401 mid-draft).
- Display-less fallback: on servers / Docker (no
DISPLAY/WAYLAND_DISPLAY, macOS/Windows always count as having one) it automatically runs headless; a failed visible launch also retries headless. Force either mode withBROWSER_HEADLESS=0/1orbrowser_headless: false/true/"auto"inapp_settings.json. - Safety rails: domain allowlist (
app_settings.json: browser_allowed_domainsorBROWSER_ALLOWED_DOMAINS), 30s navigation timeouts (BROWSER_NAV_TIMEOUT_MS), idle auto-close after 5 min of no browser action (browser_idle_close_ms/BROWSER_IDLE_CLOSE_MS,0disables), persistent profile indata/browser-profile/, and every action appended tologs/browser.log. - Setup:
playwrightis already inpackage.json;install.sh/install_ubuntu.shfetch the Chromium engine (~170MB) automatically, and the Docker image bakes it in at build time. On a manual setup, download it once withbunx playwright install chromium. Without the engine, every browser command fails with the exact install hint instead of crashing.
@cast-scan(or@scan-tv) β List TVs/renderers on your LAN (DLNA + Chromecast).@cast <url>β Auto: direct video URL casts as-is, page URL is extracted first.@cast <youtube-url>β Plays in the TV's native YouTube app (needs a Chromecast/Android TV target; honourst=start time).@cast-url <url>β Cast a direct stream/video URL (.mp4,.mkv,.m3u8, β¦).@cast-player <page-url>β Extract the video from a web page, then cast.@cast-pause/@cast-playβ Pause / resume playback on the TV.@cast-statusβ Show current cast session + TV state.@cast-stop(or@stop-cast) β Stop playback and shut down the relay.@set-cast-device <name|ip|description-url>β Select/pin the target TV (IP pin uses unicast, works where multicast is blocked).@set-setting cast_device_type <auto|dlna|chromecast>β Force a cast protocol;@set-setting cast_max_height 1080caps transcode height.- Same commands on CLI, Telegram, WhatsApp, and Web UI (group-safe). Machine and TV must share a LAN; YouTube URLs need a Chromecast target (DLNA can't play them); no DRM. In Docker, discovery needs
network_mode: hoston Linux (commented option indocker-compose.yml); bridge mode only works with a pinned IP over the published relay port.
@get-commands(or@commands,@help) β Display full command manual (requires@otp-login/@unlockfirst while locked).@clearβ Clear active conversation context window.exit/quitβ Save state and exit application.
.
βββ index.js # Main CLI entry point with interactive REPL & streaming
βββ package.json # Bun dependencies & scripts
βββ starter.sh # Smart startup wrapper script (self-healing)
βββ install.sh # Universal multi-platform installer script
βββ install_ubuntu.sh # Dedicated Ubuntu Desktop/Server/VPS installer script
βββ Dockerfile # Portable Bun + Node + Python + ffmpeg runtime image
βββ docker-compose.yml # App service (+ optional bundled `ollama` profile) with volumes & ports
βββ dockerize.sh # Build/run/up/down/logs/shell/clean helper for Docker & Compose
βββ docker-entrypoint.sh # Container entrypoint: syncs env into settings, patches, starts CLI
βββ portainer-stack.yml # Portainer stack (named volumes only; see PORTAINER.md)
βββ PORTAINER.md # Portainer setup guide (build/push, deploy, volumes, troubleshooting)
βββ .env.example # Template for OLLAMA_HOST, API keys, ports, bridge + mention toggles
βββ .dockerignore # Keeps secrets, DBs, logs and auth out of the image
βββ patch_onnx.js # Idempotent node_modules patches (log-silencing, sharp/canvas fallbacks; `bun run patch:check` verifies)
βββ app_settings.json # Active application configuration
βββ memory.json # Memory persistence store (auto-synced with SQLite; global pool with per-row language)
βββ memory.<lang>.json # Per-language memory pools, e.g. memory.en.json / memory.lt.json (auto-synced, auto-created per ISO 639 code)
βββ tasks.json # Kanban tasks persistence store
βββ characters.json # Character profiles & system prompts
βββ babyagi.ts # BabyAGI CLI entry (`bun run babyagi -- "<objective>"`)
βββ md_to_pdf.py / pdf_to_md.py # Python noteβPDF helpers
βββ chat_sessions.json # Chat sessions store
βββ data/
β βββ bot.db # SQLite database for persistent memories, knowledge, and learning logs
βββ whatsapp_qr.html # Web view for WhatsApp QR code & pairing
βββ voice_interface.html # Dagowan real-time voice web UI (security panel + commands browser)
βββ dashboard.html # OpenWebUI-style dashboard chat (model/persona pickers, sessions, settings panel, avatar/AI Friend fan-out)
βββ avatar.html # Lipsync avatar popup served at /avatar (π pumpkin theme, server TTS + browser-voice fallback)
βββ aifriend.html # AI Friend avatar popup served at /aifriend (vanilla-JS port of ai_bud_pro LiveCharacter3D.tsx)
βββ aifrienddash.html # AI Friend dashboard served at /aifrienddash (vanilla-JS port of ai_bud_pro main.tsx β dashboard REST API)
βββ chrome-extension/ # MV3 companion sidebar widget v1.1.1 (chat + locks/unlocks + Ask-Dagowan context menu)
β βββ manifest.json # MV3 manifest (side panel + popup fallback + service worker, 127.0.0.1:8767 host perms)
β βββ sidepanel.html / sidepanel.js # Sidebar widget: full-height WS client (text_prompt β llm_complete), lock badge, PIN/OTP inputs
β βββ popup.html / popup.js # Same client as a popup fallback (pre-sidePanel Chrome)
β βββ background.js # Toolbar click opens side panel; "Ask Dagowan" context menu β stash β panel auto-send
β βββ content.js # Page selection helper
β βββ README.md # Extension install + usage
βββ backups/ # Exported JSON backups
βββ projects/ # Ollama CLI agent output directory
βββ notes/ # Workspace Markdown notes directory
βββ pdfs/ # Output directory for generated PDF files
βββ whatsapp_bridge/ # Node.js Baileys web socket bridge for WhatsApp
β βββ index.js # Baileys socket + group mention pre-filter + status server
β βββ mentions.js # CJS mirror of src/whatsapp_mentions.js (bridge pre-filter)
β βββ package.json # Isolated bridge deps (installed via npm --prefix)
β βββ package-lock.json # Pinned bridge dependency tree
βββ ollama_cli/ # High-performance local AI agentic coding CLI toolset
βββ src/
βββ db/
β βββ schema.js # SQLite database schema (memories incl. character_id + language, knowledge, logs, rules)
β βββ database.js # SQLite connection manager, query helpers & character_id/language migration
βββ language.js # Open-ended multilanguage layer: detection (scripts + stopwords, any ISO 639 code), translation bridge, per-language files
βββ language.test.js # Multilanguage layer tests (pools, scoping, bridge, scripts)
βββ memory.js # Character- + language-scoped MemoryStore with TF-IDF retrieval, bridge recall, superseding, & JSON sync
βββ knowledge.js # Versioned KnowledgeStore & KnowledgeRetriever
βββ learning.js # Incremental Learning Pipeline (heuristic & LLM extraction)
βββ config.js # Settings manager, export & import settings
βββ tasks.js # Kanban task board & reminder manager
βββ proactive.js # Proactive chat timers & idle check-in manager
βββ characters.js # Character profile manager & system prompt templates
βββ logger.js # Prompt/error logging to logs/
βββ model_selection.js # Single shared-model resolution across all features
βββ agentic.js # Legacy web action dispatcher (YouTube/Google/Giphy/Spotify open & search)
βββ sessions.js # Chat sessions manager & character-scoped cross-session context retrieval
βββ workspace.js # Workspace note file reader & relevance search
βββ chat_context.js # Parallel chat-context assembly (memories+knowledge+sessions via Promise.all + workers)
βββ pdf_utils.js # Pure JavaScript Markdown-to-PDF & PDF-to-Markdown tools (parallel workers for batch)
βββ providers.js # Unified Multi-Provider LLM engine (Ollama, Gemini, Claude, Grok, Mistral, OpenAI)
βββ ollama_ephemeral.js # Ephemeral one-shot Ollama inference via bun-ollama CLI (@ephemeral/@one-shot, default ON; RAM freed per turn)
βββ ollama_ephemeral.test.js # Ephemeral one-shot unit tests (discovery, sanitizing, toggle parsing; no network/model)
βββ one_shot.js # One-shot CLI (`bun run ask "<prompt>" [--model <name>]`, model from settings)
βββ audio.js # 100% Local STT (Whisper) & Local TTS (MMS-TTS) audio models
βββ voice_socket.js # Real-Time WebSocket Voice Bridge & Server
βββ dashboard.js # OpenWebUI-style dashboard server (/avatar, /aifriend, /aifrienddash, /api/* REST + TTS)
βββ dashboard.test.js # Dashboard flag/port/chat/lock unit tests (no network)
βββ telegram.js # Telegram bot runner integration with voice attachment support
βββ whatsapp.js # WhatsApp HTTP bridge server integration (group gate + DM mention router)
βββ whatsapp_mentions.js # Shared @character-name matching/strip helpers (server side)
βββ persona_bridge.js # On-demand Telegram polling bridge (active-only boot, @wake/@sleep, idle auto-sleep sweeper)
βββ persona_bridge.test.js # Persona polling bridge unit tests (mocked runners, no network)
βββ ollama_coding.js # Ollama CLI agent & toolset integration bridge
βββ opencli.js # API-driven OpenAI coding agent (@opencli): jailed tool loop + owner gating + audit log
βββ opencli.test.js # OpenCLI jail/gating/guard unit tests (no network)
βββ opencode.js # External opencode CLI runner (@opencode/@opencode-model/@opencode-setup/@opencode-setup-auto-allow): explicit project path + user/opencode model + guided setup + auto-allow permissions + audit log
βββ opencode.test.js # opencode config/gating/path/model/parse/setup unit tests (no network)
βββ browser.js # Agentic local-browser service (visible window, YouTube/Spotify search/click/play, blog editors, read, shot)
βββ browser.test.js # Browser allowlist/URL-helper unit tests (no live browser needed)
βββ play_api.js # Official-API-first playback (YouTube Data API, Spotify Web API) + Chromium fallback
βββ play_api.test.js # Playback API unit tests (mocked fetch, no network/Chromium)
βββ blog_api.js # Official-API-first blog drafts (WordPress REST + XML-RPC fallback, Blogger v3) + Chromium editor fallback
βββ blog_api.test.js # Blog draft API/parser unit tests (mocked fetch, no network/Chromium)
βββ factcheck_api.js # Fact-checking (@fact-check): Semantic Scholar papers + Gemini verdict (mocked-fetch tests)
βββ factcheck_api.test.js # Fact-check API/parser unit tests (mocked fetch, no network)
βββ needle.js # On-device routing/extraction/embeddings via cactus-needle (spawns needle_bridge.py)
βββ needle_bridge.py # Stdin/stdout Python bridge to cactus-needle (status/complete/extract/embed)
βββ needle_daemon.py # Persistent stdio daemon: warm turns skip ~10 s startup (supervised by needle.js)
βββ needle.test.js # Needle parser/formatter/bridge unit tests (mocked spawn, no model/network)
βββ blogger_auth.js # Google OAuth loopback flow for Blogger API (`@blogger-auth`)
βββ blogger_auth.test.js # Blogger OAuth unit tests (mocked, no network)
βββ cast/ # Bun-only TV cast (Castor-style): DLNA + Chromecast + DIAL YouTube
β βββ caster.js # Orchestrator (scan β extract β probe β relay β cast) + shared @cast parser
β βββ dlna.js # SSDP discovery + SOAP AVTransport (SetAVTransportURI/Play/Stop/Pause)
β βββ castv2.js # CastV2 client (node:tls + minimal protobuf) + YouTube URL helpers
β βββ mdns.js # mDNS _googlecast discovery (compression-aware DNS parser)
β βββ dial.js # DIAL launches (native YouTube app) over plain HTTP
β βββ serve.js # Bun.serve LAN relay: Range proxy + ffmpeg transcode pipe
β βββ probe.js # ffprobe inspect β direct vs transcode decision
β βββ extract.js # Headless Chromium page β stream sniffing + ranking
β βββ cast.test.js # DLNA/extract/probe/parser unit tests (no network)
β βββ chromecast.test.js # CastV2/mDNS/DIAL/YouTube unit tests (synthetic packets)
βββ babyagi_integration.ts # BabyAGI framework bridge (provider/model wiring)
βββ babyagi/ # Bun-native BabyAGI loop (core, packs, db, types)
βββ incremental_learning.test.js # Memory/knowledge/learning pipeline tests
βββ ollama_coding.test.js # Coding-agent file/project tool tests
βββ voice_socket.test.js # Voice bridge integration tests: turns, interrupts, @ command dispatch (no network)
βββ utils/
β βββ memoize.js # Generic memoize/memoizeAsync (LRU + TTL) for pure hot paths
β βββ memoize.test.js # Memoisation unit tests (sync/async + project integrations)
β βββ concurrency.js # CPU-bounded mapWithConcurrency/chunking/worker-count helpers
β βββ retrieval_worker.js # worker_threads entry: memory/knowledge/session chunk scoring
β βββ parallel_retrieval.js # Threshold-gated parallel scoring with sync fallback
β βββ pdf_worker.js # worker_threads entry: single-file md-to-pdf / pdf-to-md
β βββ pdf_parallel.js # Bounded parallel PDF render/extract helpers
β βββ threading.test.js # Threading unit tests (worker equivalence + fallbacks)
βββ commands.js # @ commands dispatcher & help manual
This occurs when the active model is set to local Ollama (http://127.0.0.1:11434), but the Ollama daemon is not running:
- Start Ollama daemon:
ollama serve & - Download model:
ollama pull llama3
- Or switch to Cloud LLMs:
@set-api-key gemini YOUR_KEY @model gemini-2.5-flash
Fixed: the CLI simulation path omitted the current turn from the model payload, so the model answered stale history and alternate turns came back empty. Update to the latest version. If stuck, end the simulation with @simulation-end and start a new turn.
In PRoot-Distro, if external API calls fail, check /etc/resolv.conf:
echo "nameserver 8.8.8.8" > /etc/resolv.confThis public repository ships without any credentials:
app_settings.jsoncontains the full current settings schema with all secrets empty (*_api_key,*_secret,telegram_token,wordpress_*,blogger_*,spotify_*,youtube_api_key,brave/tavilykeys) and no security PIN, OTP, or YubiKey enrolled (security_pin_hash/otp_secret/yubikey_idare empty), so the project is not locked β all@control commands work immediately after setup.characters.jsonships with demo personas and no Telegram tokens.- Machine-specific runtime files are gitignored and never committed:
.env,mcp_servers.json,client_secret_*.json,data/*.db,logs/,tmp/,node_modules/, WhatsAppauth/. - Configure your own keys after install via
.env(see.env.example) or in-app@set-api-keycommands β see "π 3. Configure API Keys" above.
Known pre-existing issue:
bun testreports 1 failure insrc/blogger_auth.test.js("loadOAuthClient ... rejects junk"). It fails identically upstream and does not affect runtime behaviour.
MIT License Β© Andrius Pratusis
Run bun run mcp (or bun run index.js --mcp) to expose Dagowan as an MCP server (stdio + HTTP/SSE) for any MCP-capable assistant.
- HTTP Streamable endpoint:
http://0.0.0.0:${MCP_PORT:-8790}/mcp - Legacy SSE endpoint:
http://0.0.0.0:${MCP_PORT:-8790}/sse - Health:
http://0.0.0.0:${MCP_PORT:-8790}/health - Set
MCP_PORTto change the port. - Set
MCP_TOKENorsettings.mcp_server_tokento requireAuthorization: Bearer <token>. - Use
--mcp-stdio-onlyfor pure stdio mode (local desktop assistants).
The server above binds to your machine, so a cloud-based assistant can only reach it if you expose it publicly and secure it with a token:
- Start Dagowan in MCP (HTTP) mode:
MCP_PORT=8790 MCP_TOKEN=some-secret bun run index.js --mcp
- Expose it to the internet (ChatGPT runs in the cloud, so
localhostwon't work):You'll get a publiccloudflared tunnel --url http://localhost:8790 # or: ngrok http 8790https://...URL. - Add that URL as an MCP connector in your assistant, pointing at
https://<tunnel>/mcp, and use the sameMCP_TOKENas the bearer token. - The assistant will discover the tools (
tools/list) and call them (tools/call) β e.g.dagowan_chatto talk to the assistant.
Caveats:
- ChatGPT web MCP/connector support depends on your plan/version. If it isn't available, use a desktop MCP client instead.
- Local desktop clients (Claude Desktop, Cursor, Windsurf, β¦) don't need a tunnel β use pure stdio mode:
{ "mcpServers": { "dagowan": { "command": "bun", "args": ["run", "/path/to/dagowan/index.js", "--mcp-stdio-only"] } } } - In MCP mode only the curated tool set is exposed (
ping,dagowan_chat,list_sessions,get_memories,remember,list_tasks,add_task,web_search,web_fetch). The full assistant is reachable throughdagowan_chat(which runs the normal LLM flow with@commands, memory, sessions, tasks, web search, etc.). Other surfaces (Telegram/WhatsApp/voice/web UI) are not started in MCP mode.