Skip to content

Commit 38ff3d3

Browse files
Gemini CLIclaude
Gemini CLI
and
claude
committed
feat(mcp): v5.1 MCP Tool Market — config-driven MCP client integration
Add McpHubManager singleton for connecting to external MCP servers, McpHubToolset wrapper for ADK agent integration, 4 REST API endpoints for server management, and a frontend Tools tab in DataPanel. - mcp_hub.py: config loading, connection lifecycle, tool aggregation - mcp_servers.yaml: declarative server config (stdio/sse/http) - McpHubToolset: BaseToolset bridge for general + planner agents - REST API: GET servers/tools, POST toggle/reconnect (admin-gated) - DataPanel: 6th tab with server status cards + tool list - Health check: MCP Hub status in startup banner + system status - i18n: 5 mcp.* keys in zh.yaml + en.yaml - 47 new tests covering hub, toolset, API, health, i18n, routes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 592ef0d commit 38ff3d3

15 files changed

Lines changed: 1547 additions & 9 deletions

File tree

‎CLAUDE.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -75,13 +75,13 @@ Custom React SPA replacing Chainlit's default UI. Three-panel layout: Chat (320p
7575

7676
- **ChatPanel**: Messages, streaming, action cards, NL layer control relay
7777
- **MapPanel**: Leaflet.js map with GeoJSON layers, layer control, annotations, basemap switcher (Gaode/Tianditu/CartoDB/OSM), legend
78-
- **DataPanel**: 5 tabs — files, CSV preview, data catalog, pipeline history, token usage dashboard
78+
- **DataPanel**: 6 tabs — files, CSV preview, data catalog, pipeline history, token usage dashboard, MCP tools
7979
- **LoginPage**: Login + in-app registration mode toggle
8080
- **AdminDashboard**: Metrics, user management, audit log (admin only)
8181
- **UserSettings**: Account info + self-deletion modal (danger zone)
8282
- **App.tsx**: Auth state, map/data state, layer control, user menu dropdown
8383

84-
### Frontend API (17 REST endpoints in `frontend_api.py`)
84+
### Frontend API (21 REST endpoints in `frontend_api.py`)
8585
All endpoints use JWT cookie auth. Routes mounted before Chainlit catch-all via `mount_frontend_api()`.
8686

8787
| Method | Path | Purpose |
@@ -94,14 +94,16 @@ All endpoints use JWT cookie auth. Routes mounted before Chainlit catch-all via
9494
| GET/POST | `/api/annotations`, PUT/DELETE `/api/annotations/{id}` | Map annotations CRUD |
9595
| GET | `/api/config/basemaps` | Basemap configuration |
9696
| GET/PUT/DELETE | `/api/admin/users`, `/api/admin/users/{username}/role`, `/api/admin/metrics/summary` | Admin endpoints |
97+
| GET | `/api/mcp/servers`, `/api/mcp/tools` | MCP server status + tool listing |
98+
| POST | `/api/mcp/servers/{name}/toggle`, `/api/mcp/servers/{name}/reconnect` | MCP server management (admin) |
9799

98100
### Key Modules
99101

100102
| Module | Purpose |
101103
|---|---|
102104
| `agent.py` | Agent definitions, pipeline assembly, tool functions |
103105
| `app.py` | Chainlit UI, auth, semantic router, RBAC, file uploads, layer control |
104-
| `frontend_api.py` | 17 REST API endpoints for React frontend |
106+
| `frontend_api.py` | 21 REST API endpoints for React frontend |
105107
| `auth.py` | Password hashing, registration, account deletion, Chainlit auth callbacks |
106108
| `user_context.py` | `ContextVar` for user_id/session_id/role propagation; `get_user_upload_dir()` |
107109
| `db_engine.py` | Singleton SQLAlchemy engine with connection pooling |
@@ -118,9 +120,11 @@ All endpoints use JWT cookie auth. Routes mounted before Chainlit catch-all via
118120
| `memory.py` | Persistent per-user spatial memory |
119121
| `report_generator.py` | Markdown → Word (.docx) report generation |
120122
| `toolsets/skill_bundles.py` | 5 named toolset groupings for agent configuration |
123+
| `mcp_hub.py` | MCP Hub Manager — config-driven MCP server connection + tool aggregation |
124+
| `toolsets/mcp_hub_toolset.py` | BaseToolset wrapper bridging MCP Hub to ADK agents |
121125

122-
### Toolsets (16 modules in `toolsets/`)
123-
Exploration, GeoProcessing, Visualization (9 tools incl. `control_map_layer`), Analysis, Database, SemanticLayer (9 tools), DataLake (8 tools), Streaming (5 tools), Team (8 tools), Location, Memory, Admin, File, RemoteSensing, SpatialStatistics, SkillBundles.
126+
### Toolsets (17 modules in `toolsets/`)
127+
Exploration, GeoProcessing, Visualization (9 tools incl. `control_map_layer`), Analysis, Database, SemanticLayer (9 tools), DataLake (8 tools), Streaming (5 tools), Team (8 tools), Location, Memory, Admin, File, RemoteSensing, SpatialStatistics, SkillBundles, McpHub.
124128

125129
### Data Loading (`_load_spatial_data` in `agent.py`)
126130
Supported formats: CSV, Excel (.xlsx/.xls), Shapefile, GeoJSON, GPKG, KML, KMZ. CSV/Excel auto-detect coordinate columns (lng/lat, lon/lat, longitude/latitude, x/y).

‎data_agent/agent.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@
4545
StreamingToolset,
4646
TeamToolset,
4747
DataLakeToolset,
48+
McpHubToolset,
4849
)
4950

5051
# ArcPy conditional function lists (for governance agents needing specific subsets)
@@ -291,6 +292,7 @@
291292
StreamingToolset(),
292293
TeamToolset(),
293294
DataLakeToolset(),
295+
McpHubToolset(pipeline="general"),
294296
] + _arcpy_tools,
295297
)
296298

@@ -376,6 +378,7 @@ def _make_planner_processor(name: str, **overrides) -> LlmAgent:
376378
StreamingToolset(),
377379
DataLakeToolset(tool_filter=_DATALAKE_READ),
378380
DatabaseToolset(tool_filter=["import_to_postgis"]),
381+
McpHubToolset(pipeline="planner"),
379382
] + _arcpy_tools,
380383
)
381384
defaults.update(overrides)

‎data_agent/app.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,17 @@
159159
if DYNAMIC_PLANNER:
160160
logger.info("Dynamic Planner mode enabled")
161161

162+
# --- MCP Hub: load config (async connections deferred to on_chat_start) ---
163+
try:
164+
from data_agent.mcp_hub import get_mcp_hub
165+
_mcp_hub = get_mcp_hub()
166+
_mcp_hub.load_config()
167+
_mcp_hub_loaded = True
168+
except Exception as _mcp_err:
169+
logger.warning("MCP Hub config loading failed: %s", _mcp_err)
170+
_mcp_hub_loaded = False
171+
_mcp_started = False
172+
162173
# --- Chainlit Data Layer: thread/message persistence in PostgreSQL ---
163174
try:
164175
from data_agent.session_storage import get_chainlit_db_url as _get_cl_db_url
@@ -2243,6 +2254,16 @@ async def start():
22432254
"""Initialize session with authenticated user."""
22442255
# Set i18n language from env (default: zh)
22452256
set_language(os.environ.get("UI_LANGUAGE", "zh"))
2257+
2258+
# Start MCP Hub connections (once, on first chat start)
2259+
global _mcp_started
2260+
if not _mcp_started and _mcp_hub_loaded:
2261+
try:
2262+
await get_mcp_hub().startup()
2263+
except Exception as e:
2264+
logger.warning("MCP Hub startup failed: %s", e)
2265+
_mcp_started = True
2266+
22462267
# Get authenticated user from Chainlit (set by auth callbacks in auth.py)
22472268
cl_user = cl.user_session.get("user")
22482269

‎data_agent/frontend_api.py‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -552,6 +552,78 @@ async def _api_session_delete(request: Request):
552552
return JSONResponse({"error": str(e)}, status_code=500)
553553

554554

555+
# ---------------------------------------------------------------------------
556+
# MCP Tool Market API
557+
# ---------------------------------------------------------------------------
558+
559+
async def _api_mcp_servers(request: Request):
560+
"""GET /api/mcp/servers — list configured MCP servers with status."""
561+
user = _get_user_from_request(request)
562+
if not user:
563+
return JSONResponse({"error": "Unauthorized"}, status_code=401)
564+
565+
from .mcp_hub import get_mcp_hub
566+
hub = get_mcp_hub()
567+
servers = hub.get_server_statuses()
568+
return JSONResponse({"servers": servers, "count": len(servers)})
569+
570+
571+
async def _api_mcp_tools(request: Request):
572+
"""GET /api/mcp/tools — list all tools from connected MCP servers."""
573+
user = _get_user_from_request(request)
574+
if not user:
575+
return JSONResponse({"error": "Unauthorized"}, status_code=401)
576+
577+
server_name = request.query_params.get("server")
578+
from .mcp_hub import get_mcp_hub
579+
hub = get_mcp_hub()
580+
581+
if server_name:
582+
tools = await hub.get_tools_for_server(server_name)
583+
else:
584+
tools = []
585+
for status in hub.get_server_statuses():
586+
if status["status"] == "connected":
587+
server_tools = await hub.get_tools_for_server(status["name"])
588+
tools.extend(server_tools)
589+
590+
return JSONResponse({"tools": tools, "count": len(tools)})
591+
592+
593+
async def _api_mcp_toggle(request: Request):
594+
"""POST /api/mcp/servers/{name}/toggle — enable/disable a server (admin only)."""
595+
user, username, role, err = _require_admin(request)
596+
if err:
597+
return err
598+
599+
server_name = request.path_params.get("name", "")
600+
try:
601+
body = await request.json()
602+
except Exception:
603+
return JSONResponse({"error": "Invalid JSON body"}, status_code=400)
604+
605+
enabled = body.get("enabled", False)
606+
from .mcp_hub import get_mcp_hub
607+
hub = get_mcp_hub()
608+
result = await hub.toggle_server(server_name, enabled)
609+
status_code = 200 if result.get("status") == "ok" else 404
610+
return JSONResponse(result, status_code=status_code)
611+
612+
613+
async def _api_mcp_reconnect(request: Request):
614+
"""POST /api/mcp/servers/{name}/reconnect — force reconnect (admin only)."""
615+
user, username, role, err = _require_admin(request)
616+
if err:
617+
return err
618+
619+
server_name = request.path_params.get("name", "")
620+
from .mcp_hub import get_mcp_hub
621+
hub = get_mcp_hub()
622+
result = await hub.reconnect_server(server_name)
623+
status_code = 200 if result.get("status") == "ok" else 404
624+
return JSONResponse(result, status_code=status_code)
625+
626+
555627
# ---------------------------------------------------------------------------
556628
# Route Mounting
557629
# ---------------------------------------------------------------------------
@@ -578,6 +650,10 @@ def get_frontend_api_routes():
578650
Route("/api/user/account", endpoint=_api_user_delete_account, methods=["DELETE"]),
579651
Route("/api/sessions", endpoint=_api_sessions_list, methods=["GET"]),
580652
Route("/api/sessions/{session_id}", endpoint=_api_session_delete, methods=["DELETE"]),
653+
Route("/api/mcp/servers", endpoint=_api_mcp_servers, methods=["GET"]),
654+
Route("/api/mcp/tools", endpoint=_api_mcp_tools, methods=["GET"]),
655+
Route("/api/mcp/servers/{name}/toggle", endpoint=_api_mcp_toggle, methods=["POST"]),
656+
Route("/api/mcp/servers/{name}/reconnect", endpoint=_api_mcp_reconnect, methods=["POST"]),
581657
]
582658

583659

‎data_agent/health.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,25 @@ def _get_feature_flags() -> dict:
131131
return flags
132132

133133

134+
def check_mcp_hub() -> dict:
135+
"""Check MCP Hub connection status."""
136+
try:
137+
from .mcp_hub import get_mcp_hub
138+
hub = get_mcp_hub()
139+
statuses = hub.get_server_statuses()
140+
connected = sum(1 for s in statuses if s["status"] == "connected")
141+
total = len(statuses)
142+
if total == 0:
143+
return {"status": "unconfigured", "connected": 0, "total": 0}
144+
return {
145+
"status": "ok" if connected > 0 else "disconnected",
146+
"connected": connected,
147+
"total": total,
148+
}
149+
except Exception:
150+
return {"status": "unconfigured", "connected": 0, "total": 0}
151+
152+
134153
# ---------------------------------------------------------------------------
135154
# Aggregated endpoints
136155
# ---------------------------------------------------------------------------
@@ -169,6 +188,7 @@ def get_system_status(session_svc=None) -> dict:
169188
"cloud_storage": check_cloud_storage(),
170189
"redis": check_redis(),
171190
"session_service": check_session_service(session_svc),
191+
"mcp_hub": check_mcp_hub(),
172192
},
173193
}
174194

@@ -229,6 +249,16 @@ def _icon(status):
229249
session_detail = f"{session['backend']}" + (" (fallback)" if session["status"] == "degraded" else "")
230250
lines.append(f" [{_icon(session['status'])}] Session: {session_detail}")
231251

252+
# MCP Hub
253+
mcp = check_mcp_hub()
254+
if mcp["status"] == "unconfigured":
255+
mcp_detail = "Not configured"
256+
elif mcp["status"] == "ok":
257+
mcp_detail = f"{mcp['connected']}/{mcp['total']} servers connected"
258+
else:
259+
mcp_detail = f"0/{mcp['total']} servers connected"
260+
lines.append(f" [{_icon(mcp['status'])}] MCP Hub: {mcp_detail}")
261+
232262
lines.append("-" * 50)
233263

234264
# Feature flags

‎data_agent/locales/en.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,3 +179,10 @@ auth.delete_admin_blocked: "Administrators cannot delete their own account this
179179
auth.delete_user_not_found: "User not found"
180180
auth.delete_success: "Account deleted"
181181
auth.delete_failed: "Deletion failed: {error}"
182+
183+
# --- MCP Hub ---
184+
mcp.server_connected: "MCP server '{name}' connected: {count} tools discovered"
185+
mcp.server_failed: "MCP server '{name}' connection failed: {error}"
186+
mcp.server_disconnected: "MCP server '{name}' disconnected"
187+
mcp.hub_startup: "MCP Tool Market started: {connected}/{total} servers connected"
188+
mcp.no_config: "No MCP server configuration file found"

‎data_agent/locales/zh.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,3 +179,10 @@ auth.delete_admin_blocked: "管理员不能通过此方式删除账户，请联
179179
auth.delete_user_not_found: "用户不存在"
180180
auth.delete_success: "账户已删除"
181181
auth.delete_failed: "删除失败: {error}"
182+
183+
# --- MCP Hub ---
184+
mcp.server_connected: "MCP 服务器 '{name}' 已连接，发现 {count} 个工具"
185+
mcp.server_failed: "MCP 服务器 '{name}' 连接失败: {error}"
186+
mcp.server_disconnected: "MCP 服务器 '{name}' 已断开"
187+
mcp.hub_startup: "MCP 工具市场启动完成：{connected}/{total} 个服务器已连接"
188+
mcp.no_config: "未找到 MCP 服务器配置文件"

0 commit comments

Comments
 (0)