Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for CRDT updates #441

Open
wants to merge 40 commits into
base: main
Choose a base branch
from
Open

Add support for CRDT updates #441

wants to merge 40 commits into from

Conversation

rjwebb
Copy link
Contributor

@rjwebb rjwebb commented Feb 12, 2025

This PR adds support for the y.js Doc CRDT to Canvas.

This is a second pass, following #439 and the earlier proof of concept at https://github.com/canvasxyz/gossiplog-yjs.

The document update type is integrated into a a new Message type, where payload.type === "updates" so we now have Message<Action | Session | Snapshot | Updates>.

Updates may include yjs Insert, yjs Delete, and yjs ApplyDelta operations. Yjs operations are applied to Yjs tables, which always have the form { id: string, [key]: "yjs-doc" }, so a document editor would have a schema like:

export const models = {
  documents: {
    id: "primary",
    content: "ytext",
  },
}

export const actions = {
  async applyDeltaToDoc(db, index, text) {
    await db.ytext.applyDelta("documents", "0", index, text)
  },
}

For the relevant code, look at packages/core/src/DocumentStore.ts and action handlers inside AbstractRuntime.


"Update" message type implementation

  • extend MessageType with Update, which describes a message that contains a Y.js diff
  • update Secp256k1DelegateSigner to expect Action | Session | Snapshot | Update
  • update AbstractRuntime:
    • store yjs text docs
    • add new yjs methods (e.g. insert, delete, format on text)
    • when a yjs method is called from an append, compute the diff
    • and then append another message with the diffs
    • when a Update message is handled, update the yjs doc
  • add test to show that CRDTs converge on concurrent apps
  • update the yjs state when a snapshot is loaded
  • add ytext column to the snapshot test

How has this been tested?

  • CI tests pass
  • Tested with example-chat (including login, all signers, and exchanging messages)
  • Tested with @canvas-js/test-network: (optional)

Does this contain any breaking changes to external interfaces?

  • Contract interfaces
  • Core interface
  • CLI
  • Data storage formats, including IndexedDB, SQLite, or filesystem storage (will this break existing apps?)

…e them in a field on the AbstractRuntime object
@raykyri raykyri changed the title WIP: add MessageType and Updates types [WIP] CRDTs: add MessageType and Updates types Mar 17, 2025
@rjwebb rjwebb changed the title [WIP] CRDTs: add MessageType and Updates types Add support for CRDT updates Mar 18, 2025
@rjwebb rjwebb marked this pull request as ready for review March 19, 2025 14:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant