Skip to content

Commit 48a7af7

Browse files
committed
feat: enrich outputs and docs; add topology backends, adapter, and CLI flags
- Implement Rips/Alpha persistence via GUDHI with field_to_point_cloud adapter - Guard TDA feature extraction against NaN/Inf; add topology backend to stage results - Add attractor classification/characterization and expose types in outputs - Extend dashboard with topology summary and attractor type chart - Add mneme.data.quality, mneme.data.parallel, mneme.utils.monitoring, results_generator - CLI: add topology and attractor flags; mneme info shows backend and PySR status - Docs: update README (examples and flags), API_DESIGN, DATA_PIPELINE, PROJECT_STRUCTURE, TESTING_STRATEGY - Packaging: setup.py align to Python 3.12
1 parent 6333003 commit 48a7af7

23 files changed

Lines changed: 890 additions & 127 deletions

CLAUDE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ python -c "import numpy as np; import pandas as pd; import matplotlib.pyplot as
6666

6767
- Package imports cleanly on Python 3.12
6868
- CI: lint passes; mypy runs non-blocking; docs deploy to `gh-pages`
69-
- `create_bioelectric_pipeline()` stub available; minimal model placeholders added
69+
- `create_bioelectric_pipeline()` implemented with lightweight defaults; minimal model placeholders remain
7070

7171
## Contributor Guidance
7272

README.md

Lines changed: 53 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ Mneme seeks to uncover attractor states, regulatory logic, and latent architectu
88

99
## Key Features
1010

11-
- **Field Reconstruction**: Information Field Theory implementations for continuous field interpolation
12-
- **Topology Analysis**: Persistent homology for identifying stable structures
13-
- **Attractor Detection**: Methods for finding and characterizing dynamical attractors
14-
- **Symbolic Regression**: Discovery of mathematical rules governing field behavior
15-
- **Latent Space Analysis**: Autoencoders for dimensionality reduction and pattern discovery
11+
- **Field Reconstruction (MVP-ready)**: Basic IFT and GP reconstruction APIs; identity fallback when sparse observations are not provided
12+
- **Topology Analysis (MVP-ready)**: Cubical persistence via GUDHI when installed; simple fallback otherwise
13+
- **Attractor Detection (experimental)**: Recurrence-based detector usable on temporal data; Lyapunov and clustering detectors are stubs with NotImplemented methods
14+
- **Symbolic Regression (placeholder)**: PySR is installed optionally; shipped `SymbolicRegressor` is a placeholder. Integrations are roadmap
15+
- **Latent Space Analysis (placeholder)**: `FieldAutoencoder` class is a minimal placeholder; not a production model
1616

1717
## Installation
1818

@@ -52,14 +52,51 @@ from mneme.data import generators
5252
generator = generators.SyntheticFieldGenerator(seed=42)
5353
field = generator.generate_dynamic(shape=(64, 64), timesteps=10, parameters={'noise_level': 0.1})
5454

55-
# Create analysis pipeline
55+
# Create analysis pipeline (lightweight defaults)
5656
pipe = pipeline.create_bioelectric_pipeline()
5757
results = pipe.run({'field': field})
5858

5959
# Access results
6060
print("Pipeline executed successfully!")
6161
```
6262

63+
### CLI usage
64+
65+
Run analysis on a saved array and choose a topology backend:
66+
67+
```bash
68+
# Topology backend options: cubical (default), rips, alpha
69+
mneme analyze data/synthetic/test_small.npz \
70+
--pipeline bioelectric \
71+
--topology-backend rips \
72+
-o results
73+
```
74+
75+
Run analysis with clustering-based attractor detection (example):
76+
77+
```bash
78+
mneme analyze data/synthetic/test_small.npz \
79+
--pipeline bioelectric \
80+
--attractor-method clustering \
81+
--attractor-threshold 0.2 \
82+
--attractor-min-samples 20 \
83+
-o results_clustering
84+
```
85+
86+
#### Attractor CLI flags
87+
88+
| Flag | Description |
89+
|------|-------------|
90+
| --attractor-method {none,recurrence,lyapunov,clustering} | Choose attractor detector (use none to disable) |
91+
| --attractor-threshold FLOAT | Detection threshold (method-specific) |
92+
| --attractor-min-persistence FLOAT | Recurrence: minimum persistence fraction |
93+
| --attractor-embedding-dim INT | Recurrence/Clustering: embedding dimension for 1D series |
94+
| --attractor-time-delay INT | Recurrence/Clustering: time delay for embedding |
95+
| --attractor-n-neighbors INT | Lyapunov: number of neighbors |
96+
| --attractor-evolution-time INT | Lyapunov: evolution time steps |
97+
| --attractor-min-samples INT | Clustering: minimum samples per cluster |
98+
| --attractor-clustering-method {dbscan,kmeans} | Clustering: algorithm selection |
99+
63100
## Project Structure (MVP)
64101

65102
```
@@ -76,11 +113,16 @@ See [docs/PROJECT_STRUCTURE.md](docs/PROJECT_STRUCTURE.md) for detailed structur
76113

77114
## Documentation
78115

79-
- [Project Structure](docs/PROJECT_STRUCTURE.md) - Code organization and architecture
80-
- [Development Setup](docs/DEVELOPMENT_SETUP.md) - Environment setup and dependencies
81-
- [API Design](docs/API_DESIGN.md) - Module interfaces and usage
82-
- [Data Pipeline](docs/DATA_PIPELINE.md) - Data processing workflows
83-
- [Testing Strategy](docs/TESTING_STRATEGY.md) - Testing approach and guidelines
116+
- [Project Structure](docs/PROJECT_STRUCTURE.md) — Code organization and architecture
117+
- [Development Setup](docs/DEVELOPMENT_SETUP.md) — Environment setup and dependencies
118+
- [API Design](docs/API_DESIGN.md) — Module interfaces and usage
119+
- [Data Pipeline](docs/DATA_PIPELINE.md) — MVP note: several sections are illustrative/roadmap and not yet implemented (quality, features, parallel, monitoring, recovery)
120+
- [Testing Strategy](docs/TESTING_STRATEGY.md) — Current suite is a smoke test; more tests are roadmap
121+
122+
### Capabilities vs Roadmap
123+
124+
- MVP capabilities: importable package, CLI (`mneme info`, `mneme analyze`), lightweight preprocessing, identity or basic reconstruction, cubical persistence (with GUDHI), simple recurrence attractor detection on temporal inputs, plotting utilities
125+
- Roadmap (not fully implemented): rich loaders/quality/feature extractors; Lyapunov/clustering attractors; real autoencoders; symbolic regression integration; parallel/distributed pipeline; monitoring/recovery utilities
84126
- [Contributing](CONTRIBUTING.md) - How to contribute
85127

86128
## Roadmap

docs/API_DESIGN.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ The Mneme API follows these principles:
1010

1111
## Module APIs
1212

13+
> MVP note: Some classes shown below (e.g., rich attractor characterization, full models) are placeholders or partially implemented. Methods explicitly marked with `NotImplementedError` are roadmap.
14+
1315
### 1. Field Theory Module (`mneme.core.field_theory`) — MVP
1416

1517
```python
@@ -77,6 +79,18 @@ class AttractorDetector:
7779

7880
def characterize(self, attractor: Attractor) -> Dict[str, Any]:
7981
"""Compute attractor properties (dimension, stability, basin)."""
82+
83+
### 2b. Point-cloud topology backends — MVP
84+
85+
```python
86+
from mneme.core.topology import RipsComplex, AlphaComplex, field_to_point_cloud
87+
88+
# Convert 2D field to point cloud and run Rips
89+
pc = field_to_point_cloud(field2d, method='peaks', percentile=95.0)
90+
tda = RipsComplex(max_dimension=1)
91+
diagrams = tda.compute_persistence(pc)
92+
features = tda.extract_features(diagrams)
93+
```
8094
```
8195
8296
### 3. Models Module (`mneme.models`) — placeholders

docs/DATA_PIPELINE.md

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Mneme Data Pipeline Documentation
22

3+
> Accuracy note (MVP): Several sections below (quality module, feature extractor, parallel pipeline, monitoring, recovery) are illustrative/roadmap and not yet implemented in `src/`. Where code references non-existent modules, treat them as examples for future work.
4+
35
## Overview
46

57
The Mneme data pipeline handles the flow of data from raw bioelectric measurements and synthetic generation through preprocessing, analysis, and final results. The pipeline is designed to be modular, reproducible, and scalable.
@@ -84,7 +86,7 @@ for experiment in loader:
8486
metadata = experiment.metadata
8587
```
8688

87-
### Stage 2: Quality Control
89+
### Stage 2: Quality Control (roadmap)
8890

8991
```python
9092
from mneme.data import quality
@@ -143,7 +145,7 @@ processed = preprocessor.fit_transform(voltage_field)
143145
- Bicubic interpolation for upsampling
144146
- Gaussian process interpolation for missing data
145147

146-
### Stage 4: Feature Extraction
148+
### Stage 4: Feature Extraction (roadmap)
147149

148150
```python
149151
from mneme.analysis import features
@@ -171,19 +173,25 @@ reconstructor = field_theory.FieldReconstructor(method='ift')
171173
continuous_field = reconstructor.fit_reconstruct(processed_field)
172174

173175
# 2. Topology analysis
176+
# Cubical for 2D fields (default), or use Rips/Alpha with adapter
174177
tda = topology.PersistentHomology()
175178
persistence_diagrams = tda.compute_persistence(continuous_field)
176179

180+
# Point-cloud backends
181+
pc = topology.field_to_point_cloud(continuous_field, method='peaks', percentile=95.0)
182+
rips = topology.RipsComplex(max_dimension=1)
183+
rips_diagrams = rips.compute_persistence(pc)
184+
177185
# 3. Latent space embedding
178186
autoencoder = autoencoders.FieldAutoencoder(latent_dim=32)
179187
latent_representation = autoencoder.encode(continuous_field)
180188

181-
# 4. Attractor detection
182-
detector = topology.AttractorDetector()
189+
# 4. Attractor detection (recurrence default; lyapunov/clustering also available)
190+
detector = topology.AttractorDetector(method='recurrence')
183191
attractors = detector.detect(latent_trajectory)
184192
```
185193

186-
### Stage 6: Results Generation
194+
### Stage 6: Results Generation (roadmap)
187195

188196
```python
189197
from mneme.analysis import results
@@ -285,15 +293,15 @@ results = pipe.run_batch(
285293
)
286294
```
287295

288-
## Parallel Processing
296+
## Parallel Processing (MVP)
289297

290298
```python
291299
from mneme.data import parallel
292300

293301
# Parallel pipeline for large datasets
294302
parallel_pipeline = parallel.ParallelPipeline(
295303
pipeline=pipe,
296-
backend='multiprocessing', # or 'dask', 'ray'
304+
backend='multiprocessing', # MVP
297305
n_workers=8
298306
)
299307

@@ -334,7 +342,7 @@ field_cache = cache.FieldCache(max_size="10GB")
334342
field_cache.put("exp_001", processed_field)
335343
```
336344

337-
## Monitoring and Logging
345+
## Monitoring and Logging (MVP)
338346

339347
```python
340348
from mneme.utils import monitoring
@@ -348,10 +356,10 @@ with monitor.track_stage("preprocessing"):
348356

349357
# Get performance metrics
350358
metrics = monitor.get_metrics()
351-
print(f"Preprocessing took: {metrics['preprocessing']['duration']}s")
359+
print(f"Preprocessing durations: {metrics['stage_durations_s']}")
352360
```
353361

354-
## Error Handling and Recovery
362+
## Error Handling and Recovery (roadmap)
355363

356364
```python
357365
from mneme.data import recovery

docs/PROJECT_STRUCTURE.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ mneme/
3636
│ │ │ ├── __init__.py
3737
│ │ │ ├── pipeline.py # Main analysis pipeline
3838
│ │ │ ├── visualization.py # Plotting and visualization
39+
│ │ │ ├── features.py # Basic field feature extraction (MVP)
3940
│ │ │ └── metrics.py # Evaluation metrics
4041
│ │ │
4142
│ │ └── utils/ # Utilities
@@ -100,12 +101,15 @@ mneme/
100101
- **bioelectric.py**: Specialized handlers for bioelectric imaging data
101102

102103
### Analysis Modules (`src/mneme/analysis/`)
103-
- **pipeline.py**: Orchestrates the complete analysis workflow
104-
- **visualization.py**: Publication-quality plotting and interactive visualizations
105-
- **metrics.py**: Coherence metrics, validation measures, and evaluation tools
104+
- **pipeline.py**: Orchestrates the analysis workflow (MVP-ready)
105+
- **visualization.py**: Plotting utilities (MVP-ready)
106+
- **features.py**: Basic feature extractor (MVP-ready)
107+
- **metrics.py**: Evaluation utilities (minimal)
106108

107109
## Development Workflow
108110

111+
> Note: Some items referenced in `docs/DATA_PIPELINE.md` (e.g., `quality`, `features`, `parallel`, `monitoring`) are roadmap and not yet implemented.
112+
109113
1. **Feature Development**: Create feature branches from `main`
110114
2. **Testing**: Write tests alongside new features
111115
3. **Documentation**: Update relevant docs with changes

docs/TESTING_STRATEGY.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# Mneme Testing Strategy
22

3+
> MVP note: Current repository includes a smoke test for imports. The sections below describe the intended testing strategy as the project grows.
4+
35
## Testing Philosophy
46

57
The Mneme project employs comprehensive testing to ensure:

results/analysis_results.hdf5

26.6 KB
Binary file not shown.

results_attr/analysis_results.hdf5

26.6 KB
Binary file not shown.

results_cli/analysis_results.hdf5

34.9 KB
Binary file not shown.

setup.py

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,14 +34,12 @@
3434
"Topic :: Scientific/Engineering :: Physics",
3535
"License :: OSI Approved :: MIT License",
3636
"Programming Language :: Python :: 3",
37-
"Programming Language :: Python :: 3.8",
38-
"Programming Language :: Python :: 3.9",
39-
"Programming Language :: Python :: 3.10",
37+
"Programming Language :: Python :: 3.12",
4038
"Operating System :: OS Independent",
4139
],
4240
package_dir={"": "src"},
4341
packages=find_packages(where="src"),
44-
python_requires=">=3.8",
42+
python_requires=">=3.12",
4543
install_requires=requirements,
4644
extras_require={
4745
"dev": dev_requirements,

0 commit comments

Comments
 (0)