Curated knowledge ingestion and large session summarization

[jdandrea]

Scenario:

Perhaps related/tangential to discussion thread #633, I'm tire-kicking agentmemory (v0.9.27) for a prose-heavy workflow. I write technical courseware (AsciiDoc), respond to reviewer comments inline in source files or via GitHub (reviewer's preference), perform various quality checks along the way (using Vale and other rules, alongside curated Claude skills), and write a bit of Python (using documented, bespoke helper libraries) to set up, grade, and tear down our various lab exercises.

What I've got:

• agentmemory on macOS (M1 Max, 64GB)
• Claude Code and VS Code in a Linux VM (because reasons)
• Connected via an SSH tunnel
• MCP and hooks all wired up, skills installed, CLAUDE.md updated
• Claude Code and agentmemory connected via Gemini Enterprise Agent (aka Vertex)
• That is, no Claude API key, only Vertex; afaik Claude is called as an integrated partner model

There are two issues I'm wrestling with. Thoughts and ideas welcomed:

1. Pre-agentmemory, I've been curating and maintaining a bunch of markdown files containing plenty of insight, guidance, and lessons learned. Things like style rules, peer/editor review patterns, course conventions, team agreements, procedures, references to documentation, and so on. And so it's not raw session history.

   MEMORY.md, in turn, points to all those files using topic keywords and links. Claude will only ever grep MEMORY.md, of course, so when retrieval works well, I get by, though it feels a bit bootstrappy. (And when it doesn't work well, um... you already know what happens, that's why we're all here, right?)

   So this info is super important, and it's not JSONL transcripts from existing sessions. How can I reliably ingest/rehydrate/summarize/etc. these files in to agentmemory and have them turned into something usable and useful?

   Using /remember on each file (one big blob!) doesn't seem like a good idea at first glance. I've also tried using Claude to pre-digest the files and create manifests, which I then run through a li'l python script to slurp it all in. I'm not sure if that was a comprehensive approach either, but I figured it was worth a try.

2. I tried importing/summarizing all my session transcripts. (Side note: I bumped up cleanupPeriodDays in my settings.json for now.) All but two appear to have been summarized successfully, and the remaining (largest) sessions fail.

   For the ~6,600 observation session: [agentmemory] warn Summary validation failed {"sessionId":"...","errors":["narrative: Too small: expected string to have >=20 characters"]}. For the ~25,000 observation one (62 chunks): The worker appears to crash and reconnect during chunk handling, then I see {"error":"Invocation stopped"}.

   I've tried mitigating by halving SUMMARIZE_CHUNK_SIZE to 200 and SUMMARIZE_CHUNK_CONCURRENCY to 3, and running it as NODE_OPTIONS="--max-old-space-size=16384" npx @agentmemory/agentmemory. (I've since reverted the SUMMARIZE changes.)

   Of course those two remaining sessions also have the most valuable domain-specific work outside of my supplementary memory files. Is there another approach I ought to be taking to summarize these?

And thanks again for building this, @rohitg00 and team. Something about the overarching architecture is resonating with me, and the design comes across to me as very intentional and extra thoughtful. Curious to follow along as it evolves.

[jdandrea]

An update: After happening upon PR #472 (thank you @efenex !) I tried setting SUMMARIZE_CHUNK_SIZE=200 and SUMMARIZE_CHUNK_CONCURRENCY=50 in ~/.agentmemory/.env and re-running:

$ NODE_OPTIONS="--max-old-space-size=16384" npx @agentmemory/agentmemory

But those SUMMARIZE settings seemed to be ignored. Until I did this:

$ SUMMARIZE_CHUNK_SIZE=200 SUMMARIZE_CHUNK_CONCURRENCY=50 \
  NODE_OPTIONS="--max-old-space-size=16384" npx @agentmemory/agentmemory

And then the values took effect. I'm not sure why those two settings were being ignored otherwise.

And then after that I tried re-summarizing the two large sessions. It worked. Then I ran a consolidation and that worked too. (I can't vouch for the results, but what I mean is it ran to completion this time.)

So that would appear to take care of my second issue. (Although now I see that there's a script I could have run so perhaps I try that for completeness sake?) And yes async handling of all this to work around the 180s iii timeout would be great; separate concern tho.

As for my first issue... still wondering if I handled that well enough, or if there's a more sensible approach.