Add console.record and console.recordEnd

[dcrousso]

WebKit has added support for being able to record actions performed to a HTMLCanvasElement, CanvasRenderingContext2D, and WebGLRenderingContext via a Canvas tab in the Web Inspector frontend [1]. We are also prototyping a way for these recordings to be triggered via JavaScript, specifically through console.record and console.recordEnd [2]. console.record and console.recordEnd do not need to be limited to <canvas>, however, and we have ideas for using it with other objects, such as DOM nodes or storage objects, although these ideas are very much so just ideas at this point.

The proposal is to support the following:

console.record(object, [options]);
console.recordEnd(object);

where object is any JavaScript object and the optional options would be a dictionary of configuration values. Neither function would have any return value or throw an exception, instead silently doing nothing if no Web Inspector frontend exists or if object is already being recorded.

The keys we currently support for options are:

name: String
singleFrame: Boolean
frameCount: Integer
memoryLimit: Integer

• name is used to help distinguish multiple recordings taken, such as to identify a particular segment of logic (e.g. if I want to record the drawing of the background vs an object in the scene).
• singleFrame indicates whether the recording should automatically stop after one frame (recordEnd can still end the recording early)
• frameCount is used to limit the recording to a certain number of frames (recordEnd can still end the recording early)
  • this takes precedence over singleFrame
• memoryLimit is used to limit the amount of data that is captured during the recording, and will cause the recording to end if a new action would save more than the allotted amount of data

I don't think that these options should be required, and I am listing them here more so for greater explanation/understanding.

The main use-case of providing this would be to enable developers to identify and record specific sections of logic within a single frame. As an example:

let ctx = document.createElement("canvas").getContext("2d");

console.record(ctx, {name: "Line"});
ctx.moveTo(10, 10);
ctx.lineTo(20, 20);
ctx.stroke();
console.recordEnd(ctx);

console.record(ctx, {name: "Box"});
ctx.fillRect(20, 20, 10, 10);
console.recordEnd(ctx);

After the last action fillRect, two recordings would be available to view/replay: "Line" and "Box". This is a very simple example, but I believe that it is possible to extrapolate the usefulness of this feature to much larger and more complex drawings.

[1] Web Inspector: [META] Allow canvas actions to be recorded/replayed
[2] Web Inspector: provide method for recording CanvasRenderingContext2D from JavaScript

[domfarolino]

Admittedly my backlog (of mostly school right now :(... ) is fairly substantial but I will dig into this more as soon as I can! From a brief look though, it seems like a bit of a compatibility issue perhaps. Typically we'd prefer to have a couple major vendors commit to implementing a feature before landing it in the spec, and it sounds like (again, I have not dug into the above links or conversations) this is a WebKit-specific feature right now that does not return anything or throw so that it can be "compatible" right out of the bat (other impls technically don't break). This is perfect for Node since we don't have elaborate UI functionality that we can trigger via the console object, but I have a feeling we might want to get other implementations on board (or at least looking at it) before adding to spec? Then again it is mainly just UI stuff that doesn't affect other spec'd stuff so....hmm. My judgement on this might not be great though, so in the meantime @domenic might have some good insight!

[JosephPecoraro]

From a brief look though, it seems like a bit of a compatibility issue perhaps.

Could you elaborate on that? Compatibility issues when introducing new APIs in Web Standards normally falls into a few buckets:

• An issue exposed by different behavior between implementations
• Breaking functionality in existing environments (such as naming conflicts)

I don't think either apply here.

we might want to get other implementations on board (or at least looking at it) before adding to spec?

Yep, that is exactly what this issue is meant to encourage.

We (WebKit) would like to hear if any other browsers / browser tools have an interest in a feature like this, and if so maybe refining the name / API to work more generally for cases we hadn't thought of or considered.

Further, I think strict compatibility of functions on Console is probably held to a lower bar than other APIs. At least on the Web, many Console APIs are tools used by developers in development when debugging or investigating issues (console.profile, console.takeHeapSnapshot, console.trace). console.record would fall into that bucket. I think there is an expectation that these APIs are tools to be used during development only and should be removed / excluded in actual production. When the tool is available, it will be useful, but it isn't (and shouldn't be) required in order for websites to function. The advantage of trying to standardize such tools is that it generally works the same in all environments so developers can use them with confidence when they are available.

---

WebKit's initial use case with Canvas recordings is a natural first application because we have the ability to make a recording of a canvas element / context in the tools. But you could imagine developer tools being able to "record" other objects, and having a common way to programmatically start/stop a recording for any kind of object would be nice.

Here are some examples off the top of my head for other Web exposed objects:

• console.record(localStorage) - Show modifications to a DOMStorage between record/recordEnd
• console.record(indexedDB) - Show modifications to an IndexedDB between record/recordEnd
• console.record(navigator.geolocation) - Show changes in location between record/recordEnd
• console.record(node) - Show mutations to a Node between record/recordEnd
• console.record(webSocket) - Show the list of WebSocket messages sent/received between record/recordEnd

I don't know of any tools that provide these kinds of debugging capabilities, but the proposed API would be sufficient to make this possible.

In the same way that console.inspect(obj) lets developers programmatically inform the developer tools to focus in on a particular object, console.record(obj) would provide the ability for developers to programmatically inform developer tools to focus in on changes to a particular object. We've seen that the ability to programmatically trigger events in code be useful to developers, especially if the only other way to do something is by clicking through a UI, which might be too slow to capture what is trying to be debugged.

At a pure JavaScript Context level (e.g. node.js) I haven't thought of any compelling use cases for this yet. JavaScriptCore's JSContext implementation of console.record will end up doing nothing for now. You could imagine recording a list of changes to an ArrayBuffer / Array / Object properties / WASM Memory, but I'm not sure how useful that would be in practice. I suspect lower level tools, like the ability to add "watchpoints" (the ability to pause in a debugger on changes to objects) would be more useful.

[domenic]

/cc @whatwg/canvas just in case any of the implementers there are super-enthused about this idea for their own canvas/devtools interaction.

I think the primary concern is indeed making sure other environments are at least interested in adding a no-op function; that shouldn't be too high of a bar. Ideally they'd be interested in using this in more areas. Perhaps on the Chrome side @paulirish has some thoughts.

[terinjokes]

@JosephPecoraro In addition to exposed objects, logging changes to a POJO would be appreciated. I've had to wire up setters and observers in the past for similar functionality.

I wouldn't be against adding it here if other implementations are keen.

[JosephPecoraro]

POJO

Is that a "Plain Old JavaScript Object"?

Yeah, I suspect logging modifications / accesses to a single object is possible in pure JavaScript with Proxy, but having tools make a task like that easier (without having to make a Proxy) would be nice for developers.

My opinion right now is that such a tool shouldn't be console.record because it is so general and powerful. You'd want a specific tool for that behavior, e.g. console.watch (just spitballing so we have a name for it).

Without derailing too much. A feature like watch is low level and works with every object. I'm considering a feature like record to be high level and work with some kinds of objects. By having both you would be able to get both high and low level features on the same object record(canvas) (get a canvas recording in tools) and watch(canvas) (watch the object for modifications / accesses). You may also want to provide custom actions with watch (like pause in a debugger) but record is more passive (let the tool do its thing).

But this is the start of a healthy discussion. I think an implementation could totally make console.record do something for every object. That isn't how I was thinking about it though. What do others think?

NOTE: I'm not proposing anything like console.watch now. It is something we've thought about but haven't experimented with enough to get a list of use cases, feel for what the right API would be, or time to implement.

[JosephPecoraro]

Some folks that may be interested: @auchenberg, @Fishrock123

[JosephPecoraro]

I think the primary concern is indeed making sure other environments are at least interested in adding a no-op function; that shouldn't be too high of a bar.

My hope is that other environments wouldn't add a no-op function but would find use cases that are appropriate to their tools, particularly if there are things that the user has to start/stop manually. In my opinion adding a no-op function would actually be worse then developers feature checking console.record.

Chrome and Firefox have the ability to start/stop recording network traffic. console.record could be used to programmatically toggle this or any other developer tools behavior. In this example there isn't a suitable Object that can be passed to console.record, so perhaps a string argument console.record("network") and console.recordEnd("network") could be used to start/stop developer tools features.

I think an unwritten goal here is that I don't want to see a long list of one off console.recordCanvas/End, console.recordStorage/End, console.recordNetwork/End functions appearing on Console. I think having one generic interface, like console.inspect is, would be preferable.

[dcrousso]

Similar to what @JosephPecoraro is saying, I see console.record as a way of triggering some tracking-esque functionality in the developer tools that will do something with its captured data once the recording is over (usually via console.recordEnd).

I also agree in that console.watch would provide greater control/depth over actions performed on any given object. This would also do in the developer tools, but it would be much more immediate and not wait until something tells the system to stop (as in the system would trigger exactly when an object is modified, not when a console.watchEnd is called).

My thinking is that console.watch would act almost like a more dynamic breakpoint, whereas console.record would be used to capture some stream of data/actions/logs that can be examined at a later time (such as once the recording is completed).

[domfarolino]

@JosephPecoraro @dcrousso

I don't think either apply here.

Agreed, sorry if I came off wrong. What you said makes sense. I agree with the usefulness of this feature, and am too willing to add if a few implementations are as well. cc'ing @xirzec for Edge as well. Also wondering if @nchevobbe or @bgrins might have any thoughts.

[nchevobbe]

My initial thought is that the API as it is laid out here seems to handle many different things, which are not really linked:

• Canvas record/replay
• Nodes Mutations
• Storage changes
• Play/Pause network recording
• …

All this cases would need different UIs to be consumed, and I guess this would require the user to remember each usage depending on what they pass as a first argument to console.record.

I can see the benefits of these in isolation, but it seems a bit blurry how this would fit under the same API.

However, I can see the point of a global console.record()/console.recordEnd() when WebRR becomes a thing.

[xirzec]

Personally, I think this sounds pretty neat, though I am curious about the scenario that motivated this API. Was the WebKit team having trouble debugging canvas issues? Were developers clamoring for a way for logging without needing to write their own layer of indirection around it?

As others have mentioned, it seems like various platform features would need to implement meaningful logging for this to be useful. Perhaps we could do something generic like have console.record access a method exposed by a particular Symbol on the object, that would instruct the object to begin logging (say by passing in a logging callback function in addition to the user-specified options.) Then both platform objects and user-created objects could decide to participate without needing to make changes to the console itself.