feat: v0.1.87

README.md

@@ -218,6 +218,8 @@ This collaborative approach ensures that the final output leverages collective i
 ---
 
 > 📖 **Complete Documentation:** For comprehensive guides, API reference, and detailed examples, visit **[MassGen Official Documentation](https://docs.massgen.ai/)**
+>
+> 🤖 **For AI agents:** A curated [`llms.txt`](https://docs.massgen.ai/en/latest/llms.txt) index and full [`llms-full.txt`](https://docs.massgen.ai/en/latest/llms-full.txt) dump are published with the docs ([llmstxt.org spec](https://llmstxt.org)).
 
 ---
 

README_PYPI.md

@@ -217,6 +217,8 @@ This collaborative approach ensures that the final output leverages collective i
 ---
 
 > 📖 **Complete Documentation:** For comprehensive guides, API reference, and detailed examples, visit **[MassGen Official Documentation](https://docs.massgen.ai/)**
+>
+> 🤖 **For AI agents:** A curated [`llms.txt`](https://docs.massgen.ai/en/latest/llms.txt) index and full [`llms-full.txt`](https://docs.massgen.ai/en/latest/llms-full.txt) dump are published with the docs ([llmstxt.org spec](https://llmstxt.org)).
 
 ---
 

docs/source/_extra/llms.txt

@@ -0,0 +1,56 @@
+# MassGen
+
+> MassGen is a multi-agent framework that coordinates AI agents through redundancy and iterative refinement. Multiple agents tackle the same task in parallel, observe each other's progress, and vote to converge on the best answer. Agents can use tools (MCP), execute code, and read/write files in your project.
+
+This file follows the [llms.txt convention](https://llmstxt.org). It is a curated index of canonical MassGen documentation for AI agents and crawlers. A larger, concatenated dump of the same documentation lives at [`/llms-full.txt`](llms-full.txt).
+
+## Project
+
+- [Project README](https://github.com/Leezekun/MassGen): What MassGen is, screenshots, quickstart, the "How It Works" section.
+- [AI_USAGE.md](https://github.com/Leezekun/MassGen/blob/main/AI_USAGE.md): How LLM agents should invoke MassGen via the CLI (always with `--automation`).
+- [Changelog](changelog.html): Release notes for every MassGen version.
+
+## Quickstart
+
+- [Installation](quickstart/installation.html): Install MassGen via uv / pip.
+- [Running MassGen](quickstart/running-massgen.html): First run, CLI, WebUI, automation mode.
+- [Configuration](quickstart/configuration.html): YAML config structure, agents, backends, orchestrator settings.
+
+## User guide
+
+- [Core concepts](user_guide/concepts.html): Agents, orchestrator, voting, consensus, refinement.
+- [Backends](user_guide/backends.html): Supported model providers and how to configure each (Claude, Gemini, GPT, Grok, Azure, LM Studio, OpenRouter, Codex, Claude Code SDK).
+- [Skills](user_guide/skills.html): MassGen-as-a-skill for AI coding agents (Claude Code, Cursor, Copilot, …).
+- [WebUI](user_guide/webui.html): Browser UI for side-by-side agent visualization and voting.
+- [Filesystem & project integration](user_guide/files/index.html): Context paths, permissions, workspaces, snapshots.
+- [Tools (MCP, code execution, custom tools)](user_guide/tools/index.html): How agents call tools and how to add your own.
+- [Multimodal](user_guide/multimodal.html): Image and audio inputs.
+- [Sessions and multi-turn](user_guide/sessions/index.html): Multi-turn conversations, memory, restart, cancellation.
+- [Integration](user_guide/integration/index.html): Python API, LiteLLM custom provider, HTTP server, automation mode.
+- [Validating configs](user_guide/validating_configs.html): Config schema validation.
+- [Logging](user_guide/logging.html): How to read MassGen run logs.
+
+## Reference
+
+- [CLI reference](reference/cli.html): All `massgen` command flags.
+- [Python API reference](reference/python_api.html): Async API for embedding MassGen.
+- [YAML schema](reference/yaml_schema.html): Full configuration schema.
+- [Configuration examples](reference/configuration_examples.html): Worked examples per use case.
+- [MCP server registry](reference/mcp_server_registry.html): MCP servers known to MassGen.
+- [Supported models](reference/supported_models.html): Model registry.
+- [Timeouts](reference/timeouts.html): Timeout configuration.
+- [Status file](reference/status_file.html): The `status.json` file emitted in automation mode.
+- [Comparisons](reference/comparisons.html): MassGen vs LLM Council, CrewAI, LangGraph, AutoGen.
+
+## Examples
+
+- [Available configs](examples/available_configs.html): Index of YAML configs shipped with MassGen.
+- [Basic examples](examples/basic_examples.html): Single-agent and small multi-agent setups.
+- [Advanced patterns](examples/advanced_patterns.html): Complex coordination patterns.
+- [Case studies](examples/case_studies.html): Side-by-side visual comparisons of MassGen vs single-agent solutions.
+
+## Optional
+
+- [Development & contributing](development/contributing.html): How to contribute to MassGen.
+- [Architecture](development/architecture.html): Internal architecture notes.
+- [Roadmap](development/roadmap.html): What's planned.

docs/source/conf.py

@@ -107,6 +107,10 @@
 templates_path = ["_templates"]
 exclude_patterns = ["case_studies"]  # Exclude standalone HTML from Sphinx processing
 
+# Files in _extra/ are copied verbatim to the build output root (e.g. llms.txt).
+# Use this for files that need to live at the site root, not under /_static/.
+html_extra_path = ["_extra"]
+
 # Autodoc settings
 autodoc_default_options = {
     "members": True,
@@ -214,3 +218,68 @@
     }
     hoverxref_tooltip_maxwidth = 600
     hoverxref_tooltip_theme = "tooltipster-shadow"
+
+
+# -- llms-full.txt generation (https://llmstxt.org) --------------------------
+# At build-finish, walk the curated documentation roots and concatenate their
+# source files into a single llms-full.txt at the build output root. The
+# hand-curated index lives at _extra/llms.txt and is copied via html_extra_path.
+
+_LLMS_FULL_ROOTS = ("quickstart", "user_guide", "reference")
+_LLMS_FULL_EXTS = (".rst", ".md")
+
+
+def _generate_llms_full_txt(app, exception):
+    if exception is not None:
+        return
+    if app.builder.name != "html":
+        return
+
+    source_root = os.path.abspath(app.srcdir)
+    out_path = os.path.join(app.outdir, "llms-full.txt")
+
+    header = (
+        "# MassGen — full documentation dump\n\n"
+        "> Concatenated source of MassGen's quickstart, user guide, and reference\n"
+        "> documentation. For a curated index see /llms.txt. Generated at\n"
+        "> Sphinx build time from docs/source/{quickstart,user_guide,reference}.\n\n"
+    )
+
+    sections = []
+    for root in _LLMS_FULL_ROOTS:
+        root_path = os.path.join(source_root, root)
+        if not os.path.isdir(root_path):
+            continue
+        for dirpath, _dirnames, filenames in os.walk(root_path):
+            for name in sorted(filenames):
+                if not name.endswith(_LLMS_FULL_EXTS):
+                    continue
+                file_path = os.path.join(dirpath, name)
+                rel_path = os.path.relpath(file_path, source_root)
+                try:
+                    with open(file_path, encoding="utf-8") as fh:
+                        body = fh.read()
+                except (OSError, UnicodeDecodeError):
+                    continue
+                sections.append((rel_path, body))
+
+    sections.sort(key=lambda item: item[0])
+
+    try:
+        with open(out_path, "w", encoding="utf-8") as fh:
+            fh.write(header)
+            for rel_path, body in sections:
+                fh.write(f"\n\n---\n\n## {rel_path}\n\n")
+                fh.write(body)
+                if not body.endswith("\n"):
+                    fh.write("\n")
+    except OSError as exc:
+        print(f"Warning: failed to write llms-full.txt: {exc}")
+        return
+
+    print(f"Wrote llms-full.txt ({len(sections)} files) -> {out_path}")
+
+
+def setup(app):
+    app.connect("build-finished", _generate_llms_full_txt)
+    return {"parallel_read_safe": True, "parallel_write_safe": True}

docs/source/index.rst

@@ -80,11 +80,20 @@ See visual comparisons between MassGen and single-agent solutions, highlighting
 
 Use MassGen from Claude Code, Codex, Copilot, Cursor, and other AI coding agents.
 
+.. note::
+
+   **For AI agents and crawlers:** This site publishes a curated `llms.txt <llms.txt>`_ index following the `llmstxt.org spec <https://llmstxt.org>`_, plus a concatenated `llms-full.txt <llms-full.txt>`_ dump of the user guide and reference docs.
+
 
 How Does MassGen Compare?
 -------------------------
 
-**MassGen vs LLM Council:** While LLM Council follows a fixed 3-stage pipeline, MassGen agents autonomously decide to contribute new answers or vote for others, reaching consensus organically. Plus, MassGen agents can use tools, execute code, and read/write files in your codebase — backed by active development with regular releases. :doc:`See full comparison → <reference/comparisons>`
+MassGen sits in a different design space than typical multi-agent frameworks. The core differentiator across the board is *parallel attempts with voting and consensus* — agents tackle the same task in parallel, observe each other, and converge on a winner — backed by tools, code execution, filesystem integration, and active development.
+
+- :doc:`MassGen vs LLM Council <reference/comparisons>` — dynamic voting / consensus vs a fixed 3-stage pipeline (responses → ranking → chairman synthesis).
+- :doc:`MassGen vs CrewAI <reference/comparisons/crewai>` — parallel refinement on one task vs role-based decomposition into sub-tasks.
+- :doc:`MassGen vs LangGraph <reference/comparisons/langgraph>` — a pre-built parallel + voting protocol vs a low-level graph runtime you author yourself.
+- :doc:`MassGen vs AutoGen / AG2 <reference/comparisons/autogen>` — parallel attempts with collective validation vs conversation-based multi-agent message passing.
 
 
 Quick Start

docs/source/reference/comparisons.rst

@@ -145,11 +145,19 @@ Technical Architecture Differences
 
 The key difference: LLM Council uses a fixed 3-stage pipeline with a designated chairman, while MassGen uses dynamic coordination where agents naturally converge on the best solution through voting.
 
-.. note::
+More Comparisons
+----------------
 
-   More comparisons coming soon:
+Dedicated comparison pages for the most common "MassGen vs …" questions:
 
-   - MassGen vs Claude Code
-   - MassGen vs AG2 (and AutoGen)
-   - MassGen vs LangGraph
-   - MassGen vs CrewAI
+- :doc:`comparisons/crewai` — role-based decomposition with a hosted control plane
+- :doc:`comparisons/langgraph` — low-level graph orchestration with the LangChain stack
+- :doc:`comparisons/autogen` — multi-agent conversations (Microsoft AutoGen and the community AG2 continuation)
+
+.. toctree::
+   :hidden:
+   :maxdepth: 1
+
+   comparisons/crewai
+   comparisons/langgraph
+   comparisons/autogen

docs/source/reference/comparisons/autogen.rst

@@ -0,0 +1,147 @@
+==========================
+MassGen vs AutoGen / AG2
+==========================
+
+`AutoGen <https://github.com/microsoft/autogen>`_ is Microsoft's multi-agent conversation framework (CC-BY-4.0 docs / MIT code, ~58K GitHub stars as of May 2026). It pioneered the "agents that chat with each other and tools" pattern that much of the field now builds on. `AG2 <https://github.com/ag2ai/ag2>`_ is the community-governed continuation of AutoGen (Apache 2.0 with original MIT components, ~4.6K stars, hosted under the new ``ag2ai`` organization). Both descend from the same codebase but have diverged in stewardship and roadmap.
+
+.. note::
+
+   **Maintenance status — read this first.**
+
+   - **Microsoft AutoGen** is in maintenance mode. Microsoft has positioned `microsoft/agent-framework <https://github.com/microsoft/agent-framework>`_ as the enterprise successor, with documented migration paths from both AutoGen and Semantic Kernel, supporting Python + .NET with graph-based orchestration. AutoGen continues to receive bug fixes but no new features are planned.
+   - **AG2** is actively developed and serves as the community continuation of the AutoGen lineage. It is the project MassGen's own README cites as a direct predecessor — the "multi-agent conversation" idea in AG2 is part of what MassGen builds on.
+
+   If you are choosing today: AG2 for the AutoGen-style API with active development, Microsoft Agent Framework for the new Microsoft-stack story, and AutoGen itself only for existing codebases pinned to it.
+
+This page compares MassGen with the AutoGen / AG2 lineage. Where AutoGen and AG2 differ, the differences are called out.
+
+.. contents:: On This Page
+   :local:
+   :depth: 2
+
+Overview
+--------
+
+.. list-table::
+   :header-rows: 1
+   :widths: 18 41 41
+
+   * - Aspect
+     - MassGen
+     - AutoGen / AG2
+   * - **Primary Goal**
+     - Parallel coordination of agents on the same task with voting and consensus
+     - Multi-agent conversation: agents and tools exchange messages to solve a task
+   * - **Architecture**
+     - All agents tackle the full task in parallel and converge through voting
+     - ``ConversableAgent`` base + group chat / swarm / nested chats / society-of-mind patterns
+   * - **Maintenance**
+     - Actively developed with regular releases
+     - **AutoGen:** maintenance only (successor: Microsoft Agent Framework). **AG2:** actively developed.
+
+Architecture & Coordination Model
+---------------------------------
+
+Both **AutoGen** and **AG2** model multi-agent work as a *conversation*. The shared lineage gives them a common shape:
+
+- ``ConversableAgent`` is the base abstraction — agents send and receive messages.
+- Group chat coordinates multiple agents through a *speaker selection* policy (round-robin, manager-chosen, etc.).
+- Higher-level patterns (swarms, nested chats, society-of-mind) compose conversations into richer flows.
+- Tools are registered as Python functions and exposed to agents; MCP servers are supported via extensions.
+- Termination is rule-based (max turns, sentinel message, predicate) — there is no native voting / consensus primitive.
+
+AutoGen layers this as Core / AgentChat / Extensions APIs and also ships AutoGen Studio (a no-code GUI). AG2 keeps the same conceptual model but emphasizes open governance ("AgentOS" branding) and is iterating on the API independently of Microsoft.
+
+**MassGen** runs all agents in parallel on the *same* task. Coordination is voting-based: at each step every agent decides between submitting a new answer or voting for an existing one. The orchestrator detects consensus automatically and the winner presents.
+
+In one line: AutoGen / AG2 model multi-agent work as a *conversation* where turn-taking is the control primitive. MassGen models it as *parallel attempts with collective validation* where voting is the control primitive. Both are valid; they optimize for different shapes of problem.
+
+Feature Comparison
+------------------
+
+.. list-table::
+   :header-rows: 1
+   :widths: 22 22 22 34
+
+   * - Feature
+     - MassGen
+     - AutoGen / AG2
+     - Notes
+   * - **License**
+     - Apache 2.0
+     - AutoGen: MIT (code) / CC-BY-4.0 (docs). AG2: Apache 2.0 with original MIT components.
+     - Both lineages fully open source for self-hosted use
+   * - **Languages**
+     - Python
+     - AutoGen: Python, .NET / C#. AG2: Python.
+     - AutoGen's .NET track is one reason to prefer it on the Microsoft stack
+   * - **CLI**
+     - ✅ ``massgen``, ``massgen --automation``, ``massgen --web``
+     - AutoGen: ``autogenstudio ui``. AG2: Python-first; CLI present but less emphasized.
+     - Different focuses
+   * - **Python API**
+     - ✅ Async API
+     - ✅ Core, AgentChat, Extensions (AutoGen); ``ConversableAgent`` + orchestration patterns (AG2)
+     - Both layered; pick the level you want
+   * - **WebUI / Studio**
+     - ✅ Side-by-side agent panels with live streaming and vote/consensus view
+     - AutoGen Studio (no-code GUI; docs note it is not production-ready without extra hardening)
+     - Different roles
+   * - **MCP tools**
+     - ✅ First-class on every backend (Claude, Codex, Gemini, OpenAI-compatible, Grok, Claude Code SDK)
+     - ✅ MCP server support via extensions in both AutoGen and AG2
+     - Both work
+   * - **Model providers**
+     - 10+ direct backends with per-agent heterogeneity (Claude, Gemini, GPT, Grok, Azure, LM Studio, OpenRouter, Codex, Claude Code SDK)
+     - OpenAI primary; other providers via extension clients / generic ``LLMConfig``
+     - MassGen's backend matrix is broader and first-class
+   * - **Voting / consensus**
+     - ✅ Core mechanism; agents vote, winner presents
+     - ❌ Not built in (group chat uses speaker selection + termination, not voting)
+     - This is the central design difference
+   * - **Maintenance**
+     - Active development
+     - AutoGen: maintenance only. AG2: active.
+     - Affects long-term roadmap, not current functionality
+   * - **Successor / continuation**
+     - n/a
+     - AutoGen → `microsoft/agent-framework <https://github.com/microsoft/agent-framework>`_ (Python + .NET, graph-based; migration paths from both AutoGen and Semantic Kernel). AG2 is the community continuation.
+     - For new work, evaluate AG2 (Python-first) or Microsoft Agent Framework (Python + .NET)
+
+Voting and Consensus (the MassGen Differentiator)
+-------------------------------------------------
+
+AutoGen and AG2 group chats pick the *next speaker*; MassGen's protocol picks the *winner*. The two are not the same:
+
+- Speaker selection is a *turn-taking* mechanism — useful when one agent's output is the input to the next.
+- MassGen's voting is a *selection* mechanism — useful when you want N agents to attempt the same thing and the system to identify the strongest answer.
+
+If your task is genuinely conversational (an agent asks another agent to do something, they trade messages, the chat terminates on a condition), AutoGen / AG2 is well-shaped for it. If your task benefits from many parallel attempts converging on the best answer, MassGen is purpose-built for it.
+
+When to Use Each
+----------------
+
+**Choose AG2 when you need:**
+
+- An *AutoGen-style API* (``ConversableAgent``, group chats, swarms, nested chats) with active community-led development.
+- An open governance model independent of any single corporate steward.
+- Compatibility with the broader AutoGen ecosystem of notebooks and patterns.
+
+**Choose Microsoft AutoGen when you need:**
+
+- Compatibility with an existing AutoGen codebase you cannot migrate.
+- The .NET / C# code path alongside Python on the Microsoft stack. (Note: for new Microsoft-stack work, Microsoft Agent Framework is the recommended forward path.)
+
+**Choose MassGen when you need:**
+
+- *Parallel attempts + voting* as a first-class control flow with iterative refinement.
+- Side-by-side live visualization of every agent's reasoning and answer.
+- Heterogeneous backends per agent (Claude + Gemini + GPT + Grok all on the same task).
+- An actively developed open-source project with regular releases and a broad backend matrix.
+
+Related
+-------
+
+- :doc:`crewai` — role-based decomposition framework
+- :doc:`langgraph` — graph-based orchestration substrate
+- :doc:`../comparisons` — back to comparisons hub