Presence 2 #288

curran · 2019-04-18T12:18:56Z

Summary of changes

This PR includes most commits from WIP Presence #286 (hit a a dead end of red tests there)
Split out presence logic completely from ShareDB core.
The implementation of presence is exposed as statelessPresence, which can be used like this:

var ShareDB = require('sharedb');
var statelessPresence = require('sharedb/lib/presence/stateless');
var share = new ShareDB({ presence: statelessPresence })`).

The value of the presence option is expected to contain implementations of the following classes:
- DocPresence, which is instantiated at doc._docPresence.
- ConnectionPresence, which is instantiated at doc._connectionPresence.
- AgentPresence, which is instantiated at doc._agentPresence.
- BackendPresence, which is instantiated at doc._backendPresence.
Doc, Connection, Agent and Backend invoke methods on their respective presence classes.
By default a dummyPresence is used, which implements methods that mostly no-ops.
100% of the presence logic is encapsulated within statelessPresence, so
it should be possible to swap out better implementations in the future, and
it's possible to develop third party presence implementations external to ShareDB.
API compatibility is mostly retained between this and the Teamwork ShareDB fork.
- The main difference is that the presence implementation needs to be passed into the constructor.
- Presence is exposed as doc.presence, as in the original implementation.
See this PR to compare these changes to the original Presence PR: WIP Presence continuation Teamwork/sharedb#3

See See https://github.com/nodejs/Release

I had to add the --exit flag workaround to mocha.opts to make it exit when tests are done. A better long-term solution would be to ensure that nothing keeps node running after all tests are done, see https://boneskull.com/mocha-v4-nears-release/#mochawontforceexit.

The problem was that unsubscribe re-added the doc to the connection. Now the doc is removed from the connection after unsubscribe. Additionally, we're no longer waiting for the unsubscribe response before executing the callback. It is consistent with Query, unsubscribe can't fail anyway and the subscribed state is updated synchronously on the client side.

The issue could not cause problems in practice because ot-json0 does not support presence.

See share#219

alecgibson · 2019-10-22T16:34:22Z

README.md

@@ -91,6 +108,8 @@ __Options__
 * `options.pubsub` _(instance of `ShareDB.PubSub`)_
  Notify other ShareDB processes when data changes
  through this pub/sub adapter. Defaults to `ShareDB.MemoryPubSub()`.
+* `options.presence` _(implementation of presence classes)_
+  Enable user presence synchronization. The value of `options.presence` option is expected to contain implementations of the classes `DocPresence`, `ConnectionPresence`, `AgentPresence`, and `BackendPresence`. Logic related to presence is encapsulated within these classes, so it is possible develop additional third party presence implementations external to ShareDB.


I'm not sure about this API. It feels atypical both of normal JS, and also of this library. I think I'd rather pass in a single presence object which exposes API methods, much like the DB adapters.

alecgibson · 2019-10-22T16:38:13Z

README.md

@@ -308,6 +327,9 @@ Unique document ID
 `doc.data` _(Object)_
 Document contents. Available after document is fetched or subscribed to.

+`doc.presence` _(Object)_
+Each property under `doc.presence` contains presence data shared by a client subscribed to this document. The property name is an empty string for this client's data and connection IDs for other clients' data. The structure of the presence object is defined by the OT type of the document (for example, in [ot-rich-text](https://github.com/Teamwork/ot-rich-text#presence) and [@datavis-tech/json0](https://github.com/datavis-tech/json0#presence)).


I'm not sure I agree with handing this off to the types, because:

I'm not sure it's strictly part of "OT", and it feels like a violation of Single Responsibility

This relies on us being able to update the type libraries (can we?)

There are potentially multiple desired implementations for a given type. Especially if you consider - for example - json0, which can sub-type other arbitrary types. What is a "generic" shape for this presence?

To me, it feels more natural for us to define some sort of separate presence transformers, probably being passed in to the presence constructor option?

alecgibson · 2019-10-22T16:38:53Z

README.md

@@ -467,6 +497,10 @@ Additional fields may be added to the error object for debugging context dependi
 * 4022 - Database adapter does not support queries
 * 4023 - Cannot project snapshots of this type
 * 4024 - Invalid version
+* 4025 - Passing options to subscribe has not been implemented


(NB: We'll need to change these to string error codes: #309)

alecgibson · 2019-10-22T16:40:24Z

lib/backend.js

+
+  // Expose the DocPresence passed in through the constructor
+  // to the Doc class, which has access to the connection.
+  connection.DocPresence = this.presence.DocPresence;


This will only work when the client and the server are on the same box. It's not going to work for the mainline case, where the client resides on a remote machine, and we can't directly access Doc from Backend.

alecgibson · 2019-10-22T16:41:35Z

lib/backend.js

+  // to the Doc class, which has access to the connection.
+  connection.DocPresence = this.presence.DocPresence;
+
+  connection._connectionPresence = new this.presence.ConnectionPresence(connection);


As I stated above, I don't think this looks "normal". I'd possibly expect something more along the lines of:

presence = this.presence.connectionPresence(connection);

alecgibson · 2019-10-22T16:44:18Z

lib/client/connection.js

@@ -255,6 +253,9 @@ Connection.prototype.handleMessage = function(message) {
      return;

    default:
+      if (this._connectionPresence.isPresenceMessage(message)) {


I don't like leaning on default for this - why can't we give it its own message type?

alecgibson · 2019-10-22T16:46:09Z

lib/client/doc.js

@@ -58,6 +68,8 @@ function Doc(connection, collection, id) {
  this.type = null;
  this.data = undefined;

+  this._docPresence = new connection.DocPresence(this);


As I've mentioned above, Doc won't have access to connection.DocPresence the way this is currently put together, because connection.DocPresence is set by Backend, which might be on a different machine altogether.

alecgibson · 2019-10-22T16:46:39Z

lib/backend.js

@@ -48,6 +55,10 @@ function Backend(options) {
  if (!options.disableSpaceDelimitedActions) {
    this._shimAfterSubmit();
  }
+
+  this.presence = options.presence || dummyPresence;


What's the reason for having this off by default? Performance?

alecgibson · 2019-10-22T16:48:05Z

lib/presence/index.js

@@ -0,0 +1,6 @@
+module.exports = {


What's the point of this file?

alecgibson · 2019-10-22T16:52:05Z

lib/presence/stateless.js

+  // The processed presence is removed to avoid leaking memory, in case peers keep connecting and disconnecting a lot.
+  // The processed presence is not removed immediately to enable avoiding race conditions, where messages with lower
+  // sequence number arrive after messages with higher sequence numbers.
+  this.receivedTimeout = 60000;


Can we get some justification about why a minute is a good default amount of time? (ie is it because it's a long time compared to the "standard" lifecycle of a submit request?). How does this work if we're on a patchy connection?

Also nit: can we please write 60 * 1000?

alecgibson · 2019-10-22T17:33:15Z

Okay, I've tried to actually use this, but it doesn't currently work if your client isn't on the same box as your server (as outlined inline). I've also had a quick skim through the code, and here are my personal thoughts:

I like that this attempts to stay as generic as possible, but I worry that it's gone too far. I actually think that DocPresence should be its own class, beside Doc and Query, and that all the other logic (AgentPresence, BackendPresence and ConnectionPresence) should just be baked into the core. Let's be pragmatic about this: presence is non-trivial, and I'm not sure we'll have many full rewrites of all of the presence logic: more just tweaks to its performance.

I think I've said in the past that I'd expected presence to act more like its own entity, and that you should be able to just do things like connection.getPresence(collection, id) in a similar way to connection.get(collection, id) for a Doc. Obviously, behind the scenes, the Presence object would be tightly coupled to Doc, but I think it's still conceptually its own thing.

In the future, if we wish to add pluggable behaviour, I think we should expose presence middleware hooks. For example, if you wanted to keep a stateful cache of presence on the server, then perhaps there would be a requestPresence hook or similar, which by default no-ops, and then requests presence from all connected clients. However, if you inject middleware into the hook, then it can query a cache, and fast return from there, rather than request from clients. This approach would both be architecturally simpler, and more similar to how ShareDB already operates.

An alternative to middleware might be that we have a PresenceCacheDB, which again by default no-ops, but can be replaced by an instance that actually serves cached presence. But - in my opinion - this is all stuff that can be added after implementing the basics (this was very much how we approached document history and milestones).

I'm also not a fan of delegating presence functionality to types. I don't think it should be their responsibility, and I think there might be many "shapes" for a presence object for a given type, depending on application. I personally think we should configure presence transformers in the Backend constructor, a bit like:

new Backend({
  presence: {
    cacheDb: new MemoryPresenceCacheDB(),
    transformers: {
      'rich-text': (presence, delta, isOwnOp) => { ... },
    },
  },
});

curran · 2019-10-23T06:29:06Z

All excellent ideas and feedback @alecgibson !

/cc @gkubisa , the original author of this work. All I did with this PR was to try to rearrange his work such that the PR could be accepted.

I do love the idea that presence should be a different responsibility than the OT types. It was a pain to fork JSON0 and add presence to it.

alecgibson · 2019-10-23T06:33:25Z

Okay, I might try to have a stab at this today with a blank piece of paper, but drawing heavily on what we've done here.

curran · 2019-10-23T06:37:35Z

That would be amazing! Happy to review your work.

If you haven't seen it, the presence feature in https://github.com/SyncOT/SyncOT might be interesting.

I do not understand it, just have seen there's work done. Presence is implemented there as its own package under https://github.com/SyncOT/SyncOT/tree/master/packages/presence/src

gkubisa · 2019-10-23T09:36:39Z

Just to give a bit of background, SyncOT is my take on OT with somewhat different trade-offs in some key areas. As I could design it from the ground up, I came up with a bit different way of handling presence, which you might want to adapt to ShareDB. Note that SyncOT is not complete and the ideas presented below have not been tested in real-life situations, apart from Global Presence which we're currently using in production as a stand-alone component separate from OT.

Back to presence in the context of OT... the main idea is to decouple Global Presence and Local Presence:

Global Presence represents user sessions/connections. It contains 3 mandatory properties: sessionId, userId and locationId, which could be the ID of an OT document, location within the app, etc. It also has the lastModified timestamp and optional custom data. Global Presence can be synchronized between clients based on sessionId, userId or locationId, where the last one is by far the most useful. It is cached and shared using Redis.

Local Presence represents users' presence within OT documents, eg caret positions, etc. It is stored as part of OT documents, updated through operations and transformed as needed. It is associated with Global Presence using sessionId. Having Local Presence as part of operations and snapshots greatly simplifies the whole system, as it reuses storage/caching, synchronization, conflict resolution and undo/redo.

From the above you could notice that Global Presence is transient and Local Presence is persistent, which leads to some problems:

text selections of offline users could be visible. The solution is to simply skip displaying selections for sessions which are not active in Global Presence.
latency/CPU/bandwidth overhead of storing presence events, eg in MongoDB. This can be minimized by throttling presence updates and optimizing storage - the current MongoDB driver in ShareDB: reads the snapshot, applies an operation, writes the operation and writes the snapshot. In SyncOT the usual process will be: apply the operation to the snapshot cashed in the local memory and write the operation to the database.
storage overhead. This would be minimized by MongoDB compression. Additionally, if necessary, I will implement a utility which can remove obsolete Local Presence from the database. It will be simple and safe, as Local Presence will be stored in a separate operation property as a presence snapshot - I mean for example, "setCursorPosition X", rather than "moveCursorBy X".

I hope the above makes sense and might help implement presence in ShareDB.

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

curran · 2019-12-21T05:57:20Z

FWIW I'm in favor of the work over in #322 . It looks a lot cleaner than this implementation at first glance.

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

curran · 2020-02-05T16:52:01Z

Closing this in favor of #322

@gkubisa

This change adds the ability for clients to broadcast information about "Presence" - the notion of a client's position or state in a particular document. This might be represent a cursor in a text document, or a highlighted field in a more complex JSON document, or any other transient, current information about a client that shouldn't necessarily be stored in the document's chain of ops. The main complication that this feature solves is the issue of keeping presence correctly associated with the version of a `Doc` it was created at. For example, in a "naive" implementation of presence, presence information can arrive ahead of or behind ops, which - in a text-based example - can cause the cursor to "jitter" around the change. Using the ShareDB implementation will ensure that the presence is correctly transformed against any ops, and will ensure that presence information is always consistent with the version of the document. We also locally transform existing presence, which should help to keep (static) remote presence correctly positioned, independent of latency. In order to facilitate this, the feature must be used with an OT type that supports presence. The only requirement for supporting presence is the support of a `transformPresence` method: ```javascript type.transformPresence(presence, op, isOwnOperation): presence; ``` * `presence` _Object_: the presence data being transformed. The type will define this shape to be whatever is appropriate for the type. * `op` _Op_: the operation against which to transform the presence * `isOwnOperation`: _boolean_: whether the presence and the op have the same "owner". This information can be useful for some types to break ties when transforming a presence, for example as used in [`rich-text`][1] This work is based on the [work][2] by @gkubisa and @curran, but with the following aims: - avoid modifying the existing `Doc` class as much as possible, and instead use lifecycle hooks - keep presence separate as its own conceptual entity - break the presence subscriptions apart from `Doc` subscriptions (although in practice, the two are obviously tightly coupled) - allow multiple presences on a single `Doc` on the same `Connection` [1]: https://github.com/quilljs/delta#tranformposition [2]: #288

gkubisa and others added 30 commits April 26, 2018 12:47

Update tested nodejs versions in .travis.yml

9121baf

See See https://github.com/nodejs/Release

Add .editorconfig

4dbefd1

Update mocha

2ef8181

I had to add the --exit flag workaround to mocha.opts to make it exit when tests are done. A better long-term solution would be to ensure that nothing keeps node running after all tests are done, see https://boneskull.com/mocha-v4-nears-release/#mochawontforceexit.

Fix hasWritePending in op's callback

1489e36

Implement ephemeral "presence" data sync

a4499a5

Execute some callbacks asynchronously

33c7264

Don't send presence unnecessarily

8ff4b33

Re-sync presence after re-subscribe and re-connect

0ff380d

Emit presence asynchronously

d67dd6a

Add submitted param to presence event

e8ec215

Merge branch 'share/master' into sync-presence

9c291b2

Use the correct variable

173bf3a

The issue could not cause problems in practice because ot-json0 does not support presence.

Small test update

054d34d

Merge branch 'fix-doc-destroy' into sync-presence

642ded6

Make hasPending depend on inflightPresence and pendingPresence

56b726b

Remove cached ops without using setTimeout

762496a

See share#219

Remove --exit mocha option

e4c5e6d

Workaround for circular dependency

428c46a

Resolve merge conflicts

33450ae

Restore tests to working order

f43b752

Remove extraneous .editorconfig

9409429

Revert extraneous changes in .travis.yml and package.json

c4cf1b8

Use lolex to make 'expires cached ops' test more stable.

237d2ad

Move doc.presence to doc.presence.current

c8d35c5

Move doc.receivedPresence to doc.presence.received

3efb82c

Move doc.requestReplyPresence to doc.presence.requestReply

f0451e3

Move doc.cachedOps to doc.presence.cachedOps

5217635

Move doc.inflightPresenceSeq to doc.presence.inflightSeq

ac26dae

Move doc.inflightPresence to doc.presence.inflight

48acccc

alecgibson reviewed Oct 22, 2019

View reviewed changes

lib/presence/index.js

@@ -0,0 +1,6 @@

module.exports = {

Copy link

Collaborator

alecgibson Oct 22, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the point of this file?

alecgibson reviewed Oct 22, 2019

View reviewed changes

gkubisa mentioned this pull request Oct 23, 2019

Example App SyncOT/SyncOT#11

Open

alecgibson mentioned this pull request Nov 4, 2019

Add Presence functionality #322

Merged

alecgibson mentioned this pull request Jan 30, 2020

Move during transformation can lead to wrong result reedsy/quill-cursors#56

Closed

curran closed this Feb 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Presence 2 #288

Presence 2 #288

curran commented Apr 18, 2019 •

edited

Loading

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson Oct 22, 2019

alecgibson commented Oct 22, 2019

curran commented Oct 23, 2019

alecgibson commented Oct 23, 2019

curran commented Oct 23, 2019 •

edited

Loading

gkubisa commented Oct 23, 2019

curran commented Dec 21, 2019

curran commented Feb 5, 2020

Presence 2 #288

Presence 2 #288

Conversation

curran commented Apr 18, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alecgibson commented Oct 22, 2019

curran commented Oct 23, 2019

alecgibson commented Oct 23, 2019

curran commented Oct 23, 2019 • edited Loading

gkubisa commented Oct 23, 2019

curran commented Dec 21, 2019

curran commented Feb 5, 2020

curran commented Apr 18, 2019 •

edited

Loading

curran commented Oct 23, 2019 •

edited

Loading