Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for CRDT updates #441

Open
wants to merge 40 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
0c34a26
add MessageType and Updates types
rjwebb Feb 12, 2025
944181b
fix typo
rjwebb Feb 12, 2025
61475f1
add yjsInsert and yjsDelete methods to runtime, when these are called…
rjwebb Feb 12, 2025
4f2ba98
Merge branch 'main' into rjwebb/update-message-type
rjwebb Feb 12, 2025
e10e39e
remove signalInvalidType call
rjwebb Feb 12, 2025
7c70cf6
return an extra 'additionalMessages' value from the gossiplog apply m…
rjwebb Feb 13, 2025
8b35b93
append additional messages if they have been returned by apply
rjwebb Feb 13, 2025
b2a87bd
implement handleUpdates
rjwebb Feb 13, 2025
00a4e12
Merge branch 'main' into rjwebb/update-message-type
rjwebb Mar 3, 2025
e863949
don't return additional updates in the gossiplog apply function, stor…
rjwebb Mar 3, 2025
95c6bea
add yjs dependency to core
rjwebb Mar 3, 2025
ff6715a
reset change to gossiplog
rjwebb Mar 3, 2025
c0f894d
fix schema definition
rjwebb Mar 3, 2025
34f9431
only append updates if there are any updates
rjwebb Mar 3, 2025
7b044d1
add test to assert that documents converge
rjwebb Mar 3, 2025
3507417
implement snapshot for updates
rjwebb Mar 4, 2025
0f2ffec
add yjs to snapshot test
rjwebb Mar 4, 2025
ad01488
add contract function to apply a quill delta to the yjs document
rjwebb Mar 5, 2025
1b80319
Add yjs-doc to model schema expected column types
rjwebb Mar 5, 2025
547848b
more fixes for applyDelta
rjwebb Mar 5, 2025
c87bdd3
call apply delta with ops
rjwebb Mar 5, 2025
303edd8
WIP basic collaborative editor demo
rjwebb Mar 5, 2025
63c6c6c
autoformatting index.html
rjwebb Mar 5, 2025
c51d706
don't store ydoc states in modeldb, pass isAppend to message handlers…
rjwebb Mar 10, 2025
6ef479d
create a class to manage the stored yjs documents, don't apply a quil…
rjwebb Mar 11, 2025
3e67b8f
create reusable useDelta hook
rjwebb Mar 11, 2025
e7f405c
Merge branch 'main' into rjwebb/update-message-type
rjwebb Mar 11, 2025
f4cbc6b
remove unused imports
rjwebb Mar 11, 2025
427c9e4
comment out spam button
rjwebb Mar 12, 2025
a2b5638
apply updates directly to quill ref object
rjwebb Mar 17, 2025
c0ac328
add deleteDB to global for deleting the database during development
rjwebb Mar 18, 2025
334346a
use an integer cursor so that we don't apply the same delta twice
rjwebb Mar 18, 2025
724ad1b
remove print statement
rjwebb Mar 18, 2025
86d74b4
set db field on DocumentStore
rjwebb Mar 18, 2025
533c3a6
add basic test to save/load a document
rjwebb Mar 18, 2025
a948b26
code for getting/setting id
rjwebb Mar 18, 2025
661ccad
Merge branch 'main' into rjwebb/update-message-type
rjwebb Mar 18, 2025
758942f
add missing dependencies
rjwebb Mar 18, 2025
b3fc76b
store squashed document updates in snapshot
rjwebb Mar 18, 2025
e61a164
rename ytext functions
rjwebb Mar 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples/document/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
data/
54 changes: 54 additions & 0 deletions examples/document/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Document Editor Example

[Github](https://github.com/canvasxyz/canvas/tree/main/examples/document)

This example app demonstrates collaborative document editing using the CRDT provided by [Yjs](https://github.com/yjs/yjs). It allows users to collaboratively edit a single text document with persistence over libp2p.

```ts
export const models = {
documents: {
id: "primary",
content: "yjs-doc",
},
}

export const actions = {
async applyDeltaToDoc(db, index, text) {
await db.ytext.applyDelta("documents", "0", index, text)
},
}
```

## Server

Run `npm run dev:server` to start a temporary in-memory server, or
`npm run start:server` to persist data to a `.cache` directory.

To deploy the replication server:

```
$ cd server
$ fly deploy
```

If you are forking this example, you should change:

- the Fly app name
- the `ANNOUNCE` environment variable to match your Fly app name

## Running the Docker container locally

Mount a volume to `/data`. Set the `PORT`, `LISTEN`, `ANNOUNCE`, and
`BOOTSTRAP_LIST` environment variables if appropriate.

## Deploying to Railway

Create a Railway space based on the root of this Github workspace (e.g. canvasxyz/canvas).

- Custom build command: `npm run build && VITE_CANVAS_WS_URL=wss://chat-example.canvas.xyz npm run build --workspace=@canvas-js/example-chat`
- Custom start command: `./install-prod.sh && canvas run /tmp/canvas-example-chat --port 8080 --static examples/chat/dist --topic chat-example.canvas.xyz --init examples/chat/src/contract.ts`
- Watch paths: `/examples/chat/**`
- Public networking:
- Add a service domain for port 8080.
- Add a service domain for port 4444.
- Watch path: `/examples/chat/**`. (Only build when chat code is updated, or a chat package is updated.)
31 changes: 31 additions & 0 deletions examples/document/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<!doctype html>
<html lang="en">

<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta charset="utf-8" />
<title>Canvas Document Editor</title>
<script type="module" src="/src/index.tsx"></script>

<meta name="fc:frame" content='{
"version": "next",
"imageUrl": "https://document-example.canvas.xyz/image.png",
"button":{
"title": "Canvas Document Editor",
"action": {
"type": "launch_frame",
"name": "Open document",
"url": "https://document-example.canvas.xyz",
"splashImageUrl": "https://document-example.canvas.xyz/image.png",
"splashBackgroundColor": "#ddd"
}
}
}' />
</head>

<body>
<div id="root"></div>
</body>

</html>
51 changes: 51 additions & 0 deletions examples/document/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
{
"private": true,
"name": "@canvas-js/document",
"version": "0.14.0-next.1",
"type": "module",
"scripts": {
"build": "npx vite build",
"dev": "npx vite",
"dev:server": "canvas run --port 8080 --static dist --network-explorer --topic document-example.canvas.xyz src/contract.ts"
},
"dependencies": {
"@canvas-js/chain-atp": "0.14.0-next.1",
"@canvas-js/chain-cosmos": "0.14.0-next.1",
"@canvas-js/chain-ethereum": "0.14.0-next.1",
"@canvas-js/chain-ethereum-viem": "0.14.0-next.1",
"@canvas-js/chain-solana": "0.14.0-next.1",
"@canvas-js/chain-substrate": "0.14.0-next.1",
"@canvas-js/cli": "0.14.0-next.1",
"@canvas-js/core": "0.14.0-next.1",
"@canvas-js/gossiplog": "0.14.0-next.1",
"@canvas-js/hooks": "0.14.0-next.1",
"@canvas-js/interfaces": "0.14.0-next.1",
"@canvas-js/modeldb-sqlite-wasm": "0.14.0-next.1",
"@farcaster/auth-kit": "^0.6.0",
"@farcaster/frame-sdk": "^0.0.26",
"@libp2p/interface": "^2.7.0",
"@multiformats/multiaddr": "^12.3.5",
"@noble/hashes": "^1.7.1",
"@types/react": "^18.3.9",
"@types/react-dom": "^18.3.0",
"buffer": "^6.0.3",
"comlink": "^4.4.1",
"ethers": "^6.13.5",
"idb": "^8.0.2",
"multiformats": "^13.3.2",
"near-api-js": "^2.1.4",
"process": "^0.11.10",
"quill": "^2.0.3",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"vite-plugin-wasm": "^3.3.0"
},
"devDependencies": {
"autoprefixer": "^10.4.19",
"postcss": "^8.4.38",
"tailwindcss": "^3.4.4",
"typescript": "~5.6.0",
"vite": "^5.4.8",
"vite-plugin-node-polyfills": "^0.22.0"
}
}
6 changes: 6 additions & 0 deletions examples/document/postcss.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
}
99 changes: 99 additions & 0 deletions examples/document/src/App.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
import React, { useRef, useState } from "react"

import type { SessionSigner } from "@canvas-js/interfaces"
import { Eip712Signer, SIWESigner, SIWFSigner } from "@canvas-js/chain-ethereum"
import { ATPSigner } from "@canvas-js/chain-atp"
import { CosmosSigner } from "@canvas-js/chain-cosmos"
import { SolanaSigner } from "@canvas-js/chain-solana"
import { SubstrateSigner } from "@canvas-js/chain-substrate"

import { useCanvas } from "@canvas-js/hooks"

import { AuthKitProvider } from "@farcaster/auth-kit"
import { JsonRpcProvider } from "ethers"

import { AppContext } from "./AppContext.js"
import { ControlPanel } from "./ControlPanel.js"
import { SessionStatus } from "./SessionStatus.js"
import { ConnectEIP712Burner } from "./connect/ConnectEIP712Burner.js"
import { ConnectionStatus } from "./ConnectionStatus.js"
import { LogStatus } from "./LogStatus.js"
import * as contract from "./contract.js"
import { Editor } from "./Editor.js"
import { useQuill } from "./useQuill.js"

export const topic = "document-example.canvas.xyz"

const wsURL = import.meta.env.VITE_CANVAS_WS_URL ?? null
console.log("websocket API URL:", wsURL)

const config = {
// For a production app, replace this with an Optimism Mainnet
// RPC URL from a provider like Alchemy or Infura.
relay: "https://relay.farcaster.xyz",
rpcUrl: "https://mainnet.optimism.io",
domain: "document-example.canvas.xyz",
siweUri: "https://document-example.canvas.xyz",
provider: new JsonRpcProvider(undefined, 10),
}

export const App: React.FC<{}> = ({}) => {
const [sessionSigner, setSessionSigner] = useState<SessionSigner | null>(null)
const [address, setAddress] = useState<string | null>(null)

const topicRef = useRef(topic)

const { app, ws } = useCanvas(wsURL, {
topic: topicRef.current,
contract,
signers: [
new SIWESigner(),
new Eip712Signer(),
new SIWFSigner(),
new ATPSigner(),
new CosmosSigner(),
new SubstrateSigner({}),
new SolanaSigner(),
],
})

const quillRef = useQuill({ modelName: "documents", modelKey: "0", app })

return (
<AppContext.Provider value={{ address, setAddress, sessionSigner, setSessionSigner, app: app ?? null }}>
<AuthKitProvider config={config}>
{app && ws ? (
<main>
<div className="flex flex-row gap-4 h-full">
<div
id="quill-editor"
className="sm:min-w-[300px] md:min-w-[480px] flex-1 flex flex-col justify-stretch gap-2"
>
<Editor
ref={quillRef}
readOnly={false}
defaultValue={app.getYDoc("documents", "0").getText().toDelta()}
onSelectionChange={() => {}}
onTextChange={async (delta, oldContents, source) => {
if (source === "user") {
await app.actions.applyDeltaToDoc(JSON.stringify(delta))
}
}}
/>
</div>
<div className="flex flex-col gap-4 w-[480px] break-all">
<ConnectEIP712Burner />
<SessionStatus />
<ConnectionStatus topic={topicRef.current} ws={ws} />
<LogStatus />
<ControlPanel />
</div>
</div>
</main>
) : (
<div className="text-center my-20">Connecting to {wsURL}...</div>
)}
</AuthKitProvider>
</AppContext.Provider>
)
}
28 changes: 28 additions & 0 deletions examples/document/src/AppContext.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import { createContext } from "react"

import type { SessionSigner } from "@canvas-js/interfaces"
import { Canvas } from "@canvas-js/core"

export type AppContext = {
app: Canvas | null

address: string | null
setAddress: (address: string | null) => void

sessionSigner: SessionSigner | null
setSessionSigner: (signer: SessionSigner | null) => void
}

export const AppContext = createContext<AppContext>({
app: null,

address: null,
setAddress: (address: string | null) => {
throw new Error("AppContext.Provider not found")
},

sessionSigner: null,
setSessionSigner: (signer) => {
throw new Error("AppContext.Provider not found")
},
})
90 changes: 90 additions & 0 deletions examples/document/src/ConnectionStatus.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
import React, { useContext, useEffect, useState } from "react"

import type { Canvas, NetworkClient } from "@canvas-js/core"

import { AppContext } from "./AppContext.js"

export interface ConnectionStatusProps {
topic: string
ws: NetworkClient<any>
}

export const ConnectionStatus: React.FC<ConnectionStatusProps> = ({ topic, ws }) => {
const { app } = useContext(AppContext)

const [, setTick] = useState(0)
useEffect(() => {
const timer = setInterval(() => {
setTick(t => t + 1)
}, 1000)
return () => clearInterval(timer)
}, [])

if (app === null) {
return null
}

return (
<div className="p-2 border rounded flex flex-col gap-2">
<div>
<span className="text-sm">Topic</span>
</div>
<div>
<code className="text-sm">{topic}</code>
</div>

<hr />
<div>
<span className="text-sm">Connection</span>
</div>
<div>
<code className="text-sm">{import.meta.env.VITE_CANVAS_WS_URL}</code>
<span className="text-sm ml-2 text-gray-500">({ws.isConnected() ? 'Connected' : 'Disconnected'})</span>
</div>
</div>
)
}

interface ConnectionListProps {
app: Canvas
}

// const ConnectionList: React.FC<ConnectionListProps> = ({ app }) => {
// const [peers, setPeers] = useState<string[]>([])

// useEffect(() => {
// if (app === null) {
// return
// }

// const handleConnectionOpen = ({ detail: { peer } }: CustomEvent<{ peer: string }>) =>
// void setPeers((peers) => [...peers, peer])

// const handleConnectionClose = ({ detail: { peer } }: CustomEvent<{ peer: string }>) =>
// void setPeers((peers) => peers.filter((id) => id !== peer))

// app.messageLog.addEventListener("connect", handleConnectionOpen)
// app.messageLog.addEventListener("disconnect", handleConnectionClose)

// return () => {
// app.messageLog.removeEventListener("connect", handleConnectionOpen)
// app.messageLog.removeEventListener("disconnect", handleConnectionClose)
// }
// }, [app])

// if (peers.length === 0) {
// return <div className="italic">No connections</div>
// } else {
// return (
// <ul className="list-disc pl-4">
// {peers.map((peer) => {
// return (
// <li key={peer}>
// <code className="text-sm">{peer}</code>
// </li>
// )
// })}
// </ul>
// )
// }
// }
Loading