Merge pull request #95 from skynetcmd/feature/langchain-checkpointer

LangChain checkpointer + LCEL surfaces (release 2026.7.15.1)

Author: skynetcmd

CHANGELOG.md

@@ -25,6 +25,51 @@ the policy is forward-going only.
 
 ---
 
+## [2026.7.15.1] — 2026-07-15 — LangChain checkpointer + LCEL surfaces
+
+### Added
+
+- **LangGraph checkpointer (`M3Saver`)** — a `BaseCheckpointSaver` backed by
+  m3's local engine DB, so a LangGraph run can pause at a human-in-the-loop
+  `interrupt()`, survive a process restart, and resume/time-travel. Stored in
+  dedicated tables that deliberately bypass the embedder/contradiction pipeline
+  (a checkpoint is machine state, not knowledge). Sync + async surfaces.
+- **LCEL-native memory components** — `MemoryWrite` (a pass-through `Runnable`
+  that persists each piped value, so it composes at the tail of a chain),
+  `MemoryRetrieve` (callable retrieval for the head), and the `with_m3_memory`
+  decorator for zero-boilerplate capture of a callable's input/output.
+- **`M3Retriever.explain(query)`** — the real retrieval signal (blended
+  relevance score, per-memory confidence, type, bitemporal validity) for
+  attaching to a LangSmith run or logging. Reports what m3 computes; never
+  fabricates per-component sub-scores.
+- **`M3_DEFAULT_USER_ID`** — an env fallback so single-user LangChain apps can
+  drop `user_id=` from every call. Resolution stays `explicit > constructor
+  default > env > raise`: there is still no anonymous mode, so tenancy isolation
+  is unchanged.
+- **New examples** under `examples/langchain-agent/`:
+  `graph_checkpointer.py`, `agent_with_memory_and_persistence.py`, and
+  `lcel_memory.py`; plus `bin/mem0_scan.py`, an AST scanner that reports mem0
+  usage and rewrites imports for migration.
+
+### Fixed
+
+- **LangChain integration was not shipped in built wheels.** `m3_memory/
+  integrations/` lacked an `__init__.py` and `setup.py` excluded the whole
+  integrations tree, so `from m3_memory.langchain import …` worked only from a
+  repo checkout. The integration now ships as importable code (verified against
+  an isolated release build). The `m3_memory.langchain` shim also now raises an
+  actionable "upgrade" error instead of a cryptic namespace error when the
+  payload is absent.
+
+### Changed
+
+- **LangChain docs updated** to cover all five drop-in surfaces (including the
+  new checkpointer) and the LCEL components; the version note now states the
+  supported `langchain-core` / `langgraph` compatibility band (major-version
+  range) rather than describing it as a hard pin.
+
+---
+
 ## [2026.7.15.0] — 2026-07-15 — Documentation accuracy, positioning, and a CUDA install guide
 
 Documentation and repository-hygiene release. No changes to the installable

INSTALL.md

@@ -81,12 +81,12 @@ wiring, chatlog hooks, and a final `doctor` verification. Power users can
 still run each step manually with `m3 install-m3`, `m3 embedder install`,
 etc. — see `m3 --help`.
 
-> **Tool catalog stays small in your context.** m3 ships 87 MCP tools but
-> groups them into 8 domains (memory, chatlog, files, entity, agent, tasks,
-> conversations, admin). Only ~6 essentials load at MCP startup
-> (~2,400 tokens vs ~16,100 if all 87 loaded eagerly). The agent pulls in
-> a domain on demand — just say "load the files tools" and it does. Set
-> `M3_TOOLS_LAZY=0` to disable.
+> **Tool catalog stays small in your context.** m3 ships 100+ MCP tools but
+> groups them into 9 domains (memory, chatlog, files, entity, agent, tasks,
+> conversations, diagnostics, admin). Only the ~18 essentials load at MCP
+> startup (~3,540 tokens, ~1.8% of a 200K window; the full catalog loads on
+> demand). The agent pulls in a domain on demand — just say "load the files
+> tools" and it does. Set `M3_TOOLS_LAZY=0` to disable.
 
 If the TL;DR worked, stop here. The rest of this file explains why and what
 gets installed.

MANIFEST.in

@@ -27,3 +27,11 @@ recursive-include memory/migrations *.sql
 global-exclude __pycache__/*
 global-exclude *.py[cod]
 global-exclude *.pdb
+
+# Prune colocated hermetic tests (e.g. integrations/langchain/test_provider_logic.py)
+# from BOTH the sdist and the wheel. Verified against an ISOLATED release build
+# (`python -m build`, which fetches the pinned setuptools>=77,<82): the resulting
+# wheel contains 0 test_*.py. (A non-isolated build using a newer local setuptools
+# — e.g. 82.x, outside our pin — can ignore this exclude for wheel package modules,
+# so always validate packaging with an isolated build, not the dev setuptools.)
+global-exclude test_*.py

bin/catalog/tools_admin.py

@@ -26,8 +26,8 @@
         name="tools_list_domains",
         description=(
             "List m3 tool domains (memory, chatlog, files, entity, agent, tasks, "
-            "conversations, admin) and their tool counts. Call `tools_load_domain` "
-            "to expose a domain's full tool surface."
+            "conversations, diagnostics, admin) and their tool counts. Call "
+            "`tools_load_domain` to expose a domain's full tool surface."
         ),
         parameters={"type": "object", "properties": {}, "required": []},
         impl=_tool_loader.list_domains,
@@ -43,7 +43,7 @@
             "Use when you need tools beyond the essentials (memory_search, "
             "memory_write, memory_get, chatlog_search, chatlog_write, files_search). "
             "Valid domains: memory, chatlog, files, entity, agent, tasks, "
-            "conversations, admin."
+            "conversations, diagnostics, admin."
         ),
         parameters={
             "type": "object",
@@ -96,7 +96,8 @@
             "(name/type/required). Use this to discover the exact args for any "
             "tool before calling it via m3_call — cheaper than a failed call. "
             "Read-only catalog metadata; never returns tool output. Domains: "
-            "memory, chatlog, files, entity, agent, tasks, conversations, admin."
+            "memory, chatlog, files, entity, agent, tasks, conversations, "
+            "diagnostics, admin."
         ),
         parameters={
             "type": "object",

bin/gen_features_json.py

@@ -131,9 +131,11 @@ def main() -> int:
                 "install_extra": "pip install m3-memory[langchain]",
                 "mem0_drop_in": True,          # Memory/M3Memory/MemoryClient, one-line import swap
                 "langgraph_basestore": True,   # M3Store — backs LangMem / any BaseStore consumer
+                "langgraph_checkpointer": True,  # M3Saver — BaseCheckpointSaver (pause/resume/time-travel)
                 "langmem_compatible": True,
                 "chat_message_history": True,  # M3ChatMessageHistory, with_m3_history
                 "rag_retriever": True,         # M3Retriever (LangChain BaseRetriever)
+                "lcel_runnables": True,        # MemoryWrite / MemoryRetrieve / with_m3_memory
                 "docs": "docs/integrations/LANGCHAIN.md",
                 "examples": "examples/langchain-agent/",
                 "note": (

bin/gen_mcp_inventory.py

@@ -144,7 +144,7 @@ def get_category_map():
     }
     return mapping
 
-def generate_markdown(all_tools):
+def generate_markdown(all_tools, counts=None):
     cat_map = get_category_map()
 
     # Sort tools by category then name
@@ -165,7 +165,21 @@ def generate_markdown(all_tools):
     ]
 
     md = "# MCP Tool Inventory\n\n"
-    md += f"This document provides a comprehensive inventory of all {len(all_tools)} MCP tools available in the M3 Memory system.\n\n"
+    md += (
+        f"This document provides a comprehensive inventory of all {len(all_tools)} "
+        "MCP tools available in the M3 Memory system.\n\n"
+    )
+    if counts:
+        md += (
+            f"> **How this reconciles with other docs.** The **{counts['catalog']}** "
+            "in-catalog tools (the number the README/COMPARISON quote as \"100+\", and "
+            f"the `count` in `docs/tools/MCP_CATALOG.json`, which excludes the two "
+            f"`tools_*` meta-tools → **{counts['catalog'] - 2}** there) plus "
+            f"**{counts['proxy']}** MCP-proxy protocol/debug tools (`mcp_proxy.py`) = "
+            f"**{len(all_tools)}** total listed here. The prose \"100+ tools\" refers to "
+            "the in-catalog surface; this inventory additionally documents the proxy "
+            "transport tools.\n\n"
+        )
 
     # Summary Table
     md += "## Summary Table\n\n"
@@ -230,7 +244,10 @@ def main():
     # 2. Proxy tools
     protocol_tools, debug_tools = extract_proxy_tools()
 
-    all_tools = catalog_tools
+    # NOTE: copy, don't alias — `all_tools = catalog_tools` would make the two the
+    # SAME list, so appending proxy tools below would also grow catalog_tools and
+    # corrupt the len(catalog_tools) used for the reconciliation counts.
+    all_tools = list(catalog_tools)
     for t in protocol_tools:
         f = t['function']
         all_tools.append({
@@ -257,8 +274,14 @@ def main():
     if len(all_tools) != EXPECTED_TOOL_COUNT:
         print(f"Warning: Expected {EXPECTED_TOOL_COUNT} tools, found {len(all_tools)} — update EXPECTED_TOOL_COUNT in gen_mcp_inventory.py if a tool was added/removed.")
 
-    # 3. Render
-    markdown = generate_markdown(all_tools)
+    # 3. Render — pass the catalog/proxy split so the intro can disclose exactly
+    # what the total counts (otherwise "122 tools" silently disagrees with the
+    # catalog's 111 and the "100+" prose — three numbers, no reconciliation).
+    counts = {
+        "catalog": len(catalog_tools),
+        "proxy": len(protocol_tools) + len(debug_tools),
+    }
+    markdown = generate_markdown(all_tools, counts)
 
     # 4. Save
     output_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), "docs", "MCP_TOOLS.md")

bin/gen_star_history.py

@@ -37,7 +37,9 @@ def _get(url: str, token: str) -> tuple[list, dict]:
     req.add_header("Authorization", f"Bearer {token}")
     req.add_header("X-GitHub-Api-Version", "2022-11-28")
     req.add_header("User-Agent", "m3-star-history-generator")
-    with urllib.request.urlopen(req, timeout=30) as resp:  # noqa: S310 (trusted host)
+    # nosec B310 — URL is the hardcoded https://api.github.com host (fixed scheme),
+    # not attacker-controlled; noqa suppresses ruff, nosec suppresses bandit.
+    with urllib.request.urlopen(req, timeout=30) as resp:  # noqa: S310  # nosec B310
         body = json.loads(resp.read().decode("utf-8"))
         return body, dict(resp.headers)