Everything is a signal
LeapFlow is a signal-driven, self-evolving agent framework. Virtual signals (IM apps, OS events, web) and physical signals (robot arms, depth cameras, scientific instruments, IoT) flow through one pipeline into observation → learning → decision-control loops. Knowledge compounds across episodes: skills, device trust, and environment understanding accumulate — the agent gets smarter the more you use it.
- v0.6.0 — Benchmark qualification suite (36 scenarios, 12 external integrations), governed hardware degradation coordinator, failure fingerprints, CUA window-target contract, adaptive context retention.
- v0.5.0 — LeapRobot: governed robot control, local/remote VLA inference, real-time servo, teleoperation + episode export; hc.v1 descriptors, robot-arm & ROS 2 transports.
- v0.4.0 — Engine decomposition (6 delegates, 5 sub-packages), BM25 progressive tool disclosure, PCD prefix-commitment, cache-hit telemetry, skill curator, scheduler productization.
Previous releases
- v0.3.0 — Event-sourced self-evolution pipeline,
leap evolveCLI, LeapBoard evolution & causal views. - v0.2.0 — Teacher/student world-model self-evolution, Hardware Context Protocol (hc.v0), 8 generic hardware tools, LeapBoard hardware lens, device preview pipeline.
- v0.1.1 — DeepSeek Harness plugin compatibility, restricted Node.js bridge, component-aware verdicts.
- v0.1.0 — First-class plugin subsystem: adaptive evolution, marketplace (Ed25519 signing), sandbox isolation, Gateway adapter registry.
- v0.0.9 — Real-time signal pipeline, TUI thinking display, cross-session history, long-task convergence hardening.
- v0.0.8 — Cross-platform Windows support (DaemonTransport), journey test layer with cassette-backed CI.
- v0.0.7 — 1M-class context windows, self-calibrating token estimator, concurrent-TUI session isolation.
1. Install
pip install leapflowFrom source
git clone https://github.com/modelscope/leapflow.git && cd leapflow
uv sync --all-extras --no-extra leapspace2. Configure LLM — launch the TUI and set your provider (key goes to the local secret vault, never plaintext):
leap/config llm set --base-url https://dashscope.aliyuncs.com/compatible-mode/v1 \
--model qwen3.8-flash --api-key sk-...
Any OpenAI-compatible endpoint works — DashScope, OpenAI, DeepSeek, Anthropic, etc.
Shell / CI
leap config llm set \
--base-url https://api.openai.com/v1 \
--model gpt-6.1-sol \
--ask-api-key3. Go
leap # interactive TUI
leap "summarize this repo" # one-shot
leap --mock-host # no native execution backendHave hardware? See Physical AI.
Locate → read → edit → verify code, out of the box:
| Tool | What it does |
|---|---|
repo_map |
Project orientation — languages, structure, entry points |
code_search |
Regex search (ripgrep) |
edit_file |
Anchored search-replace or unified diff — rejects ambiguous edits |
code_intel |
Document symbols (Python AST) |
git_query / git_write |
Structured git operations |
test_run / lint_check |
Auto-detect and run test / lint suites |
web_fetch |
Fetch pages and JSON APIs — read-only, cached, retried |
terminal_session |
Persistent shell sessions (opt-in) |
leap teach "open Safari and bookmark this page"Record your actions → distill into parameterized skills → earn autonomy through success:
DRAFT → CANDIDATE → VERIFIED → PRODUCTION.
/board # session analysis (default)
/board finance # finance lens
/board templates add ./custom.yaml
Real-time monitoring dashboard with Server-Driven UI. Bring your own YAML template — one file, zero code.
❯ connect to Telegram
Turn any IM channel (Feishu, DingTalk, Telegram, Slack, …) into a signal boundary. Inbound events are classified, filtered, and routed; each chat gets an isolated session. Setup is conversational — no config files to edit.
LeapFlow bridges LLM intelligence and real-world hardware through a three-layer protocol stack and a layered control hierarchy:
LHP Fabric cross-node data plane (JSON Lines)
──────────────── inventory · reads · writes · halts · permits · fleet
HCP per-device contract
──────────────── 6-method transport: open / close / read / write / probe / halt
Southbound transport plugins
──────────────── ROS2 · Piper · Feetech · RealSense · model service · MCP · media …
System-2 (LLM) 0.2–1 Hz task planning → node skill (plan → approve → run)
↓
System-1.5 (skills) 5–30 Hz perceive · grasp · policy.run (VLA action chunks)
↓
System-1 (servo) 100–1000 Hz PID / impedance — never crosses the fabric
↓
actuators escalation flows back up ↑
The host sends goals; the node decomposes them locally. A skill is planned on the node, approved once by its plan digest, then executed next to the device with a run-scoped permit — so a 1 kHz servo loop and a 30 Hz policy stream stay safe regardless of network latency.
Protocol once, devices for free — new device = one YAML on the node; new protocol = one transport plugin; new node = one fleet entry.
Hardware tools
| Group | Tools | Risk |
|---|---|---|
| Inspect | hw_list · hw_describe · hw_read · hw_status |
read-only |
| Act | hw_configure · hw_actuate · hw_dispense · hw_batch_actuate |
approval-gated |
| Safety | hw_estop |
no approval — node-local; holds in place, reports stop_effect / stop_verified |
| Control | hw_control_start · hw_control_stop · hw_control_status |
leased deadman |
| Fleet | hw_fleet_status · hw_fleet_discover · hw_fleet_halt_all |
multi-node |
| Skills | hw_skill_list · hw_skill_run · hw_skill_status · hw_skill_stop |
plan-digest approval, node-local |
| Data | hw_skill_label |
labels a recorded episode's outcome |
Governance
- Admission before visibility — unverified or halt-less devices are demoted to read-only.
- Signed execution permits — Ed25519, single-use, nonce-protected, TTL-limited; node verifies and still re-runs local safety.
- Halt never waits — node-local, bypasses approval and network.
- Progressive trust — autonomy earned per device/action on verified success, lost on failure.
Bundled physical-AI stack
- Robot arms (
leapflow.robot) — LeRobot-derived: serial bus (Feetech), teleop, camera capture, episode export, policy registry (vla_local/position_control). - Node skills — calibration (
calib.*), perception (perceive.locate), grasp proposals (grasp.propose), model-written motion programs (motion.sequence), and learned policies (policy.run) run on the node behind the same gate chain; with an episodes directory configured, every actuating run is recorded as a training episode. - Models as devices — a policy or perception server (
model_servicetransport, HTTP) appears as an HCP device; inference never moves anything — only a skill's gated motion writes to actuators. - Depth cameras — RealSense integration; frames as governed binary channels.
- ROS2 — sidecar transport over Unix socket; topics/services/actions map into HCP.
Deployment modes
| Mode | Description |
|---|---|
| Host only | Edge node has no LeapFlow runtime. |
| Host + node | Host agent + edge leap hw serve (recommended). |
| Node only | Edge node operates independently. |
Host + node setup:
# Edge node
pip install leapflow
openssl rand -hex 24 > ~/.leapflow/.lhp-token && chmod 600 ~/.leapflow/.lhp-token
LEAPFLOW_LHP_TOKEN="$(cat ~/.leapflow/.lhp-token)" \
leap hw serve --node-id edge01 --host 0.0.0.0 --port 8767 \
--log-file ~/.leapflow/logs/lhp-node.log # size-rotated; config hardware.node_log_*
# Host
export EDGE_LHP_TOKEN='paste-the-token-here'
export LEAPFLOW_FLEET_NODES='[{"node_id":"edge01","endpoint":"tcp://<ip>:8767","token_env":"EDGE_LHP_TOKEN"}]'
leap daemon restart && leap hw listStop the edge node (run on the node, with the same user and profile):
leap hw stop # stop the sole local node and its workers
leap hw stop --node-id edge01 # select a node if several are running
leap hw stop --timeout 60 --json # wait longer; structured outcomestop uses the owning systemd user unit when present, preventing its restart
policy from immediately relaunching the node; otherwise it sends SIGTERM to the
identified serve process. It waits for the node and observed workers to exit.
Repeated stops return not_running; ambiguous targets are refused. Discovery is
local and profile-scoped, and the host daemon/TUI and unrelated services remain
running. On Linux, legacy nodes without lifecycle records can also be discovered;
on macOS, the node must have been started by a version that writes these records.
A timeout returns failure without sending SIGKILL. A systemd unit can independently
force termination at its own TimeoutStopSec; an abnormal unit result is reported
as unclean, not as a successful graceful stop.
Stopping the service is not hardware power-off or a parking operation:
- Graceful shutdown cancels runs/leases, attempts declared actuator halts, and releases camera streams, previews, and device connections.
- Piper halt attempts a powered hold-in-place; closing CAN sends no disable, reset, parking move, or gripper-open command. An enabled arm may remain energized and a gripper may retain its last command. Actual posture, holding torque, and object retention after disconnect are not verified; other hardware follows its driver's and firmware's shutdown behavior.
- Node-side monitoring/deadman and host remote control/estop through this node are unavailable after exit. Keep the hardware attended and follow its physical safety procedures. Host fleet health changes on its configured heartbeat schedule.
The command prints these consequences and distinguishes cleanup completion from
physical verification (hardware_state_verified: false). For a supervised node,
resume with systemctl --user start <owning-unit>; otherwise reuse the original
leap hw serve ... command. Stopping a unit does not disable its boot activation.
Device onboarding
leap hw init my_sensor # generate hc.v1 template
leap hw validate my_sensor # parse + check
leap hw probe my_sensor # connectivity test
leap hw verify my_sensor --as operator # digest-bound verification
leap hw permit-key # Ed25519 keypair for permits Virtual Physical
IM · OS · clipboard · web arms · cameras · instruments · IoT
│ │
└──────────────┬─────────────────────┘
▼
┌─────────────────────┐
│ Unified Pipeline │
│ normalize → gate → │
│ dedup → memory → │
│ causal fusion │
└────────┬────────────┘
▼
┌─────────────────────┐
│ OODA Engine │
│ Observe → Orient → │
│ Decide → Act │
└─────────────────────┘
Three nested loops on the pipeline:
| Loop | Scope | Example |
|---|---|---|
| Decision | per turn | Depth adapts to task difficulty; bounded frames compose |
| Learning | per outcome | World-model verdicts, skill distillation, trust accrual |
| Evolution | per environment delta | Typed capability requirements → resolution before acquisition |
Every capability arrives behind a typing.Protocol (@runtime_checkable). One lifecycle applies to all: reversible registration, fiber state machine, hot-reload, trust ledger.
| Protocol | Role |
|---|---|
ToolPlugin |
Expose tools to the LLM |
GatewayAdapterPlugin |
IM platform adapter |
LLMProviderPlugin |
LLM inference backend |
SignalSource / ActiveSignalSource |
Signal sources |
HardwareTransport |
Southbound device protocol (entry point) |
InferenceStrategy |
VLA / policy inference (entry point) |
CVProcessor |
Computer-vision processor |
| Command | Description |
|---|---|
/status |
Runtime and backend status |
/tool |
Inspect available tools |
/config |
View or update settings (tab-completion) |
/model |
Show or switch the active model |
/usage |
Context and token usage |
/board |
Open the monitoring dashboard |
/gateway |
Platform connection status |
/orient |
Layered orientation and re-entries |
| Action | Behavior |
|---|---|
Safe shell (ls, git status) |
Auto-execute |
Dangerous shell (sudo, rm -r) |
Approval prompt — [a]lways for session |
Destructive (rm -rf /, mkfs) |
Always blocked |
| File writes (safe types) | Auto-approve |
| Hardware actuation | Approval-gated; hw_estop always allowed |
leap config list # all settings
leap config set memory.working_max_tokens 12000Inside the TUI: /config list, /config set … — hot-reloads the active session.
Common keys
| Key | Default | Description |
|---|---|---|
llm.model |
— | Primary model |
llm.base_url |
— | OpenAI-compatible endpoint |
llm.context_length |
1000000 |
Runtime context budget shown in the TUI status bar |
memory.working_max_tokens |
12000 |
Working memory budget |
tools.terminal_session_enabled |
false |
Persistent terminal sessions |
dashboard.enabled |
true |
LeapBoard |
evolution.enabled |
false |
Self-evolution |
Self-Evolution
Disabled by default. When enabled, LeapFlow observes environment deltas and evolves its plugin composition under governance:
leap config set evolution.enabled truePipeline: observation → adaptation verdict → capability gap → resolution-first check → proposal → LLM code generation → multi-stage validation → dual approval → install at DRAFT trust → progressive accrual.
Even when disabled, the world model still produces verdicts and distils knowledge. See docs/world_model_plugin_harness_evolution.md.
Plugin Self-Management
The agent manages its own plugins at runtime: plugin_list, plugin_generate, plugin_install, plugin_reload, plugin_disable, plugin_remove, plugin_rollback.
All mutations require HIGH-risk approval. Trust: DRAFT → CANDIDATE → VERIFIED → PRODUCTION. Untrusted plugins run sandboxed (subprocess JSON-RPC). Marketplace plugins are Ed25519-signature verified.
Adaptive-Depth Execution
The agent loop adapts depth to task difficulty. Hard tasks earn wider budgets; recursive subagents run isolated OODA frames.
Config: agent.subagent_full_loop, agent.reentry_enabled, agent.calibration_enabled. Inspect with /orient. Design doc: docs/design/adaptive_depth_ooda.md.
Workflow Copilot (Preview)
Multi-tier next-action predictor: L0 hash match (<1ms) → L1 n-gram (<5ms) → L2 embedding (<50ms) → L3 LLM+RAG (200–2000ms). Display overlay pending; predictions are computed and cached internally.
Native Execution Backend (macOS)
For desktop perception — screen capture, accessibility tree, input injection:
brew install trycua/tap/cua-driver && leap host doctorWithout this, use --mock-host.
Multi-Profile
LEAPFLOW_PROFILE=work leapEach profile owns its own daemon, config, credentials, and databases under ~/.leapflow/profiles/<name>/.
Custom Gateway Platform
Drop a YAML manifest in ~/.leapflow/profiles/<profile>/gateway/manifests/, implement the adapter Protocol (connect, send, disconnect), and say "connect to my_platform" in the TUI.
CLI Reference
| Command | Description |
|---|---|
leap |
Interactive TUI |
leap "question" |
One-shot chat |
leap teach [goal] |
Record and distill a skill |
leap run [prompt] |
Execute a matched skill |
leap skills list|show|export|import |
Skill library |
leap evolve |
Run the learning boundary |
leap board |
LeapBoard dashboard |
leap board status|stop [--all] |
List / stop LeapBoard web servers (incl. orphans) |
leap hw list|serve|verify|estop|… |
Hardware / LHP node |
leap daemon start|stop|restart|status |
Manage leapd |
leap config list|set|show |
Configuration |
leap host doctor |
Execution backend check |
Flags: --mock-host, --thinking
make setup && make test && make lintMore
make test-unit # mock layer, parallel
make test-e2e # real journeys
make test-impact BASE=origin/main
make test-live # real provider (needs API key)src/leapflow/
├── cli/ ├── engine/ ├── hardware/
├── robot/ ├── tools/ ├── plugins/
├── llm/ ├── memory/ ├── perception/
├── signal_fusion/ ├── causal/ ├── world_model/
├── evolution/ ├── learning/ ├── skills/
├── copilot/ ├── monitor/ ├── dashboard/
├── gateway/ ├── platform/ ├── security/
├── daemon/ ├── scheduler/ ├── domain/
├── storage/ ├── recording/ └── analysis/
| Symptom | Fix |
|---|---|
cua-driver not found |
brew install trycua/tap/cua-driver |
MCP connection failed |
leap host doctor |
LLM API key is empty |
/config llm set --api-key … |
| Accessibility denied | System Settings → Privacy & Security → grant |
Device writable: 0 |
leap hw verify — unverified devices are read-only |
permit_required |
leap hw permit-key and configure issuer keys |
Apache 2.0 — see LICENSE.
ModelScope · Star · Issues · Discussions
LeapFlow — Learning and Evolving from Actual Practice.