Metacognitive Memory

Reflection-enhanced memory for AI agents.

Metacognitive Memory is a focused research prototype for reflective memory in AI agents. Instead of treating memory as a passive store-and-retrieve component, the system attaches explicit metamemory tags to stored items and runs a reflection loop that inspects the memory store for curated contradictions, likely knowledge gaps, and confidence-related signals.

This repository is a clean open-source subset of the larger Mnemosyne workspace. It is the part of the project that maps most directly to the Metacognitive Memory paper and its controlled pilot evaluation.

Why This Repository Exists

• to make the metacognitive-memory idea executable rather than purely conceptual
• to provide a compact companion codebase for workshop or arXiv circulation
• to offer a reproducible starting point for future benchmark-scale work on reflective agent memory

What This Release Includes

• memory objects for semantic, episodic, and procedural memory
• explicit metamemory tags for confidence, reliability, validation history, and retrieval success signals
• a periodic reflection loop over the memory store
• heuristic factual conflict detection
• heuristic temporal conflict detection
• knowledge-gap surfacing from low-success metamemory cues
• a reproducible pilot evaluation script
• smoke tests and a minimal runnable demo

Project Status

This release is intended as:

• a research prototype
• an executable reference implementation
• a controlled pilot-evaluation artifact

It is not intended as:

• a benchmark-winning system
• a production memory service
• a full long-horizon agent evaluation suite

Quickstart

1. Install

pip install -r requirements.txt

If you prefer an editable local install:

pip install -e ".[dev]"

2. Run the Minimal Demo

python examples/minimal_demo.py

Expected output:

Reflection summary
  conflicts: 1
  gaps: 0
  suggestions: 1

3. Reproduce the Pilot Evaluation

python experiments/metacognitive_pilot_eval.py

4. Run Smoke Tests

pytest -q

Minimal Example

The runnable example lives at examples/minimal_demo.py. A shortened version is shown below:

import asyncio

from mnemosyne.core.memory import MemoryType
from mnemosyne.metacognitive import MetacognitiveMemory


async def main():
    memory = MetacognitiveMemory(use_ollama=False)

    await memory.remember(
        "The package is open source",
        memory_type=MemoryType.SEMANTIC,
    )
    await memory.remember(
        "The package is not open source",
        memory_type=MemoryType.SEMANTIC,
    )

    insight = await memory.reflect()
    print(f"conflicts: {len(insight.conflicts)}")

    memory.shutdown()


asyncio.run(main())

Reflection Loop at a Glance

store memory
  -> attach metamemory tag
  -> accumulate memory history
  -> run reflection on demand
  -> surface:
       - factual conflicts
       - temporal conflicts
       - likely knowledge gaps
       - improvement suggestions

Pilot Evaluation Snapshot

All numbers below are from the included controlled pilot evaluation using the local mock embedding backend.

Capability	Setting	Result
Factual conflict detection	16 curated cases	Precision 1.00 / Recall 1.00 / F1 1.00 / Accuracy 1.00
Temporal conflict detection	16 curated cases	Precision 1.00 / Recall 1.00 / F1 1.00 / Accuracy 1.00
Knowledge-gap surfacing	12 curated cases	Precision 1.00 / Recall 1.00 / F1 1.00 / Accuracy 1.00
Reflection latency	10 memories	6.95 ms +- 2.14
Reflection latency	50 memories	41.88 ms +- 0.27
Reflection latency	100 memories	156.61 ms +- 7.87

These results should be read as proof-of-concept evidence on implementation-aligned controlled cases, not as benchmark-level generalization claims.

Repository Layout

Path	Purpose
mnemosyne/	Core implementation for this release
experiments/metacognitive_pilot_eval.py	Controlled pilot evaluation script
examples/minimal_demo.py	Small end-to-end demo
tests/test_release_smoke.py	Smoke tests for the public subset
docs/research_overview.md	Scope and positioning notes
CITATION.cff	Citation metadata for the software release
CONTRIBUTING.md	Contribution guidelines for public collaborators
RELEASE_NOTES_v0.1.0.md	Release summary for the first public drop

Limitations

• The evaluation is controlled and implementation-aligned rather than benchmark-scale.
• Conflict detection relies on lightweight heuristics and embedding similarity thresholds.
• Knowledge-gap surfacing is driven by metamemory cues, not autonomous external verification.
• Reflection currently uses pairwise scans and does not yet scale as a production design.

Roadmap

• broaden conflict detection beyond the current heuristic scope
• add stronger end-to-end agent evaluations on public benchmarks
• improve scaling behavior for larger memory stores
• add cleaner packaging and citation metadata for long-term public maintenance

Release Notes

The first public release summary is available in RELEASE_NOTES_v0.1.0.md.

Contributing

Guidelines for contributors are available in CONTRIBUTING.md.

License

This project is released under the MIT License. See LICENSE.

Citation

Citation metadata for the software release is available in CITATION.cff.

Once the accompanying paper is public, you can update both this section and CITATION.cff with the final publication metadata.