pluggable storage

[hwchase17]

Pluggable Backend

We want to make sure that people can bring and plug in their own filesystems, for both short term and long term storage.

Right now we do this by using State for short term memory, and Store for long term.

But we should let people choose their own store.

Long Term Memory

We should keep some opinions about long term memory in the core deepagents. Specifically, if long term memory is True, we should use a long term memory subdir (/memories/) and route that to the pluggable storage system that is specified by long term memory. There should be something in the prompt about /memories/, and we should handle the routing between normal store and long term memory store.

APIs for pluggable backend

The APIs for the filesystems backing short and long term memory should be pretty similar. Should expose all the same functionality that the tools in existing filesystem do (ls, read, write, edit) with the same API. this is so that each impelmentation can implement it however it wants.

They can return either:

• String (will be used as tool message)
• None (will use a default message)
• Command (will just return this)

Shared stuff

So if the filesystem is pluggable, what is left? The existing tool definitons for one. And then the tools should do the routing between short and long term memory

Implementations

Implement:

• filesystem (both current dir, and then ability to use a diff dir (for long term memory))
• state (already in there)
• store (the current long term memory)
• sqlite

Default

The defaults should be same as now (state, store)