|
1 | 1 | **English** | [中文](./README.md) |
2 | 2 |
|
3 | | -# GIS Data Agent (ADK Edition) v6.0 |
| 3 | +# GIS Data Agent (ADK Edition) v7.0 |
4 | 4 |
|
5 | | -An AI-powered geospatial analysis platform that turns natural language into spatial intelligence. Built on **Google Agent Developer Kit (ADK)** with semantic intent routing, four specialized pipelines, a React three-panel frontend, and enterprise-grade security. Features multi-source data fusion, multimodal input, 3D visualization, and workflow orchestration. |
| 5 | +An AI-powered geospatial analysis platform that turns natural language into spatial intelligence. Built on **Google Agent Developer Kit (ADK)** with semantic intent routing, four specialized pipelines, a React three-panel frontend, and enterprise-grade security. Features multi-source data fusion, multimodal input, 3D visualization, workflow orchestration, and geographic knowledge graph. |
6 | 6 |
|
7 | 7 | ## Core Capabilities |
8 | 8 |
|
9 | | -### Multi-Source Data Fusion (v5.5–v6.0) |
| 9 | +### Multi-Source Data Fusion (v5.5–v7.0) |
10 | 10 | - **Five-stage pipeline**: Profile → Assess → Align → Fuse → Validate |
11 | 11 | - **10 fusion strategies**: spatial join, attribute join, zonal statistics, point sampling, band stack, overlay, temporal fusion, point cloud height assignment, raster vectorize, nearest join |
12 | 12 | - **5 data modalities**: vector, raster, tabular, point cloud (LAS/LAZ), real-time stream |
13 | | -- **Intelligent semantic matching**: catalog-driven equivalence groups + tokenized similarity (camelCase↔snake_case) + type compatibility + automatic unit conversion |
| 13 | +- **Intelligent semantic matching**: |
| 14 | + - Five-tier progressive matching: exact → equivalence groups → embedding similarity → unit-aware → fuzzy |
| 15 | + - **v7.0 Vector embedding matching**: Gemini text-embedding-004 cosine similarity (opt-in) |
| 16 | + - Catalog-driven equivalence groups + tokenized similarity + type compatibility + auto unit conversion |
| 17 | +- **LLM-enhanced strategy routing (v7.0)**: Gemini 2.0 Flash intent-aware strategy recommendation |
| 18 | +- **Distributed/out-of-core computing (v7.0)**: Auto-chunked processing for large datasets (>500K rows / >500MB) |
| 19 | +- **Geographic knowledge graph (v7.0)**: networkx entity-relationship modeling, spatial adjacency/containment detection, N-hop neighbor queries |
14 | 20 | - **Raster auto-processing**: CRS reprojection, resolution resampling, windowed sampling for large rasters |
15 | 21 | - **Enhanced quality validation**: 10 checks (null rate, geometry validity, topology, KS distribution shift, etc.) |
16 | 22 |
|
@@ -120,7 +126,11 @@ Default login: `admin` / `admin123` (seeded on first run). In-app self-registrat |
120 | 126 | | | NL Layer Control | Natural language show/hide/style/remove map layers via `control_map_layer` tool | |
121 | 127 | | | MCP Tool Market | Config-driven MCP server connection + tool aggregation | |
122 | 128 | | **Data Fusion** | Fusion Engine (MMFE) | Five-stage pipeline (Profile→Assess→Align→Fuse→Validate), 10 strategies, 5 modalities | |
123 | | -| | Semantic Matching | Catalog-driven equivalence groups + tokenized similarity + type compatibility + auto unit conversion | |
| 129 | +| | Semantic Matching | Five-tier progressive: exact → equivalence groups → embedding similarity → unit-aware → fuzzy | |
| 130 | +| | Embedding Matching (v7.0) | Gemini text-embedding-004 vector semantic matching (opt-in) | |
| 131 | +| | LLM Strategy Routing (v7.0) | Gemini 2.0 Flash intent-aware strategy recommendation (`strategy="llm_auto"`) | |
| 132 | +| | Knowledge Graph (v7.0) | networkx spatial entity-relationship modeling, N-hop queries, shortest path | |
| 133 | +| | Distributed Computing (v7.0) | Auto-chunked processing for large datasets (>500K rows) | |
124 | 134 | | | Raster Processing | Auto CRS reprojection, resolution resampling, windowed sampling for large rasters | |
125 | 135 | | | Point Cloud & Stream | LAS/LAZ height assignment, CSV/JSON stream temporal fusion (time window + spatial aggregation) | |
126 | 136 | | | Quality Validation | 10 checks: null rate, geometry, topology, CRS, micro-polygons, outliers, KS distribution shift | |
@@ -180,22 +190,24 @@ data_agent/ |
180 | 190 | ├── workflow_engine.py # Workflow engine: CRUD, execution, webhook, cron |
181 | 191 | ├── multimodal.py # Multimodal input: image/PDF classification, Gemini Parts |
182 | 192 | ├── mcp_hub.py # MCP Hub Manager: config-driven MCP server management |
183 | | -├── fusion_engine.py # Multi-modal Data Fusion Engine (MMFE, ~1750 lines) |
| 193 | +├── fusion_engine.py # Multi-modal Data Fusion Engine (MMFE, ~2100 lines) |
| 194 | +├── knowledge_graph.py # Geographic Knowledge Graph Engine (networkx, ~625 lines) |
184 | 195 | ├── pipeline_runner.py # Headless pipeline executor (run_pipeline_headless) |
185 | | -├── toolsets/ # 18 BaseToolset modules |
| 196 | +├── toolsets/ # 19 BaseToolset modules |
186 | 197 | │ ├── visualization_tools.py # 10 tools: choropleth, heatmap, 3D, layer control |
187 | 198 | │ ├── fusion_tools.py # Data fusion toolset (4 tools) |
| 199 | +│ ├── knowledge_graph_tools.py # Knowledge graph toolset (3 tools) |
188 | 200 | │ ├── mcp_hub_toolset.py # MCP tool bridge |
189 | 201 | │ ├── skill_bundles.py # 5 named toolset groupings |
190 | 202 | │ └── ... # exploration, geo processing, analysis, database, etc. |
191 | 203 | ├── prompts/ # 3 YAML prompt files |
192 | | -├── migrations/ # 18 SQL migration scripts (001-018) |
| 204 | +├── migrations/ # 19 SQL migration scripts (001-019) |
193 | 205 | ├── locales/ # i18n: zh.yaml + en.yaml |
194 | 206 | ├── db_engine.py # Connection pool singleton |
195 | 207 | ├── health.py # K8s health check API |
196 | 208 | ├── observability.py # Structured logging + Prometheus |
197 | 209 | ├── i18n.py # i18n: YAML dict + t() function |
198 | | -├── test_*.py # 61 test files (1290+ tests) |
| 210 | +├── test_*.py # 62 test files (1330+ tests) |
199 | 211 | └── run_evaluation.py # Agent evaluation runner |
200 | 212 |
|
201 | 213 | frontend/ |
@@ -271,7 +283,7 @@ Custom React SPA replacing Chainlit's default UI: |
271 | 283 | ## Running Tests |
272 | 284 |
|
273 | 285 | ```bash |
274 | | -# All tests (1290+ tests) |
| 286 | +# All tests (1330+ tests) |
275 | 287 | python -m pytest data_agent/ --ignore=data_agent/test_knowledge_agent.py -q |
276 | 288 |
|
277 | 289 | # Single module |
@@ -306,9 +318,10 @@ GitHub Actions workflow (`.github/workflows/ci.yml`) runs on push to `main`/`dev |
306 | 318 | | v5.4 | Workflow Builder (Engine + Editor + Cron + Webhook) | Done | |
307 | 319 | | v5.5 | Multi-Modal Data Fusion Engine MMFE (5 modalities, 10 strategies, semantic matching) | Done | |
308 | 320 | | v5.6 | MGIM-Inspired Enhancements (fuzzy matching, unit conversion, data-aware scoring, multi-source) | Done | |
309 | | -| v6.0 | Fusion Engine Improvements (raster reprojection, point cloud, stream temporal, semantic+quality) | **Current** | |
| 321 | +| v6.0 | Fusion Engine Improvements (raster reprojection, point cloud, stream temporal, semantic+quality) | Done | |
310 | 322 | | v6.1 | Advanced Analytics Engine (spatiotemporal prediction, scenario simulation, network analysis) | Planned | |
311 | | -| v7.0 | Real-time Collaboration, Knowledge Graph, Edge Deployment | Future | |
| 323 | +| v7.0 | Vector Embedding Matching, LLM Strategy Routing, Knowledge Graph, Distributed Computing | **Current** | |
| 324 | +| v8.0 | Real-time Collaboration, Edge Deployment, Data Connector Ecosystem | Future | |
312 | 325 |
|
313 | 326 | ## License |
314 | 327 |
|
|
0 commit comments