Possible alternative design of Discovering API

[relu91]

Currently, the Discovering API uses the ThingFilter's method field to select which discovery approach to use. Therefore, we are using a factory style to select different strategies using an Enumerative value. This is common in the OOP world but I rather use a more functional style to solve this problem. The fact is that method field is hinting us that the discovery process could be carried out in different ways, which means that we could model the API using different functions.

Consequently, my proposal is to remove the method field from ThingFilter and design a set of functions that represents the different method enumerative values. In concrete, I'd define a subnamespace of WoT called discovery which contains the directory, local, mulitCast, direct (?), and any functions. I'll describe the interface formally using Typescript since I am not so used to WebIDL:

declare namespace WOT {
    namespace discovery {
        function directory(url:[string|ThingDescription],query?:string):ThingDiscovery ;
        function local():ThingDiscovery ;
        function multiCast():ThingDiscovery ;
        function any():ThingDiscovery ;
        function direct(url:string,form?:Form):Promise<ThingDescription>;
    }
}

Note that function signatures are just examples, they are not the topic of this issue.
However, it can be seen how this solution adds flexibility to the inputs/outputs of the different discovery processes. We are not limited to use the ThingFilter object containing all the possible inputs of the algorithms. We can now fine-tune the signature to better convey the anatomy of the discovery method. For example, see the (proposal) signature of the direct algorithm, where the output is not a ThingDiscovery but just the ThingDescription requested. Or see the directory function where we define as input also a ThingDescription of the directory service (if directories will have a TD).

Practically, devs will be able to call the discovery in the following way:

const TD = await WOT.discovery.direct("http://somehost.some");
const thing = await WOT.consume(TD)

pro

• algorithm input/output fine-tuning
• Easier to describe each algorithm. we do not need a big switch and explain each case, but we can use different subsections.
• More expressive. we model the process as a real function, not as an Enum value.
• More extensible. it is easier to change/add discovery algorithms (i.e. adding a new function or change its signature)
• Easy feature detection (e.g., checking if local is supported: wot.discovery.local !== undefined)
• ...?

cons

• Model any function might be hard
• WebIDL does not seem to support subnamespaces
• Increase the number of functions in the API (i.e. increase complexity)
• Cannot easily express a combination of different discovery methods. (suggested by @zolkis in the comment below)
• ... ?

see also

This design is similar to vscode extension API (they are using module instead namespace)

[zolkis]

In general, it might indeed make sense to make small functions instead of dictionary options, if those get too big.
However, in this particular case those options are not mature enough yet to know what functions to make out of them, so at least we need to stabilize the API for that.

Some of the API rationale has been capture here: https://github.com/w3c/wot-scripting-api/blob/master/rationale.md

In addition (to be added there, too) the current API design reflects the basic interactions structure:

• discover TDs
• consume
• produce.
  These define the words (functions).
  The rest are parameters to the functions.

For discovery, IMHO the discovery methods are rather parameters (from client point of view, since the client is interested in discovering TD's, sometimes won't even provide parameters, just going with the defaults):
- by direct URL of the Thing
- by WoT directory (URL of the directory)
- by using the underlying protocols' discovery mechanisms (e.g. multicast).

That doesn't stop us from factoring out separate algorithms in the spec for each type of discovery. But IMHO it's better if we present these through one interface that basically just provides TDs. That also allows using sensible defaults, or fallback policies.

Also the filters are parameters to discovery. When represented in a single dictionary, they can be specified single or in combination, which is not possible if we separate the functions out.

So yes, I recognize the advantages of factoring out well defined functionality in smaller functions, but we don't have yet stable defined functionality for that yet. Then, I believe the current API design is more stable and more versatile (though less explicit).

We can definitely explore using small functions. Maybe the first function to try separately would be directory().
By understanding and fine-tuning the smaller functions, we may also be able to improve the integrated API as well :).
Then we will be in better position to decide.
Let's spend more time experimenting.

[relu91]

Thanks for the feedback!

For discovery, IMHO the discovery methods are rather parameters (from client point of view, since the client is interested in discovering TD's, sometimes won't even provide parameters, just going with the defaults):

• by direct URL of the Thing
• by WoT directory (URL of the directory)
• by using the underlying protocols' discovery mechanisms (e.g. multicast).

That doesn't stop us from factoring out separate algorithms in the spec for each type of discovery. But IMHO it's better if we present these through one interface that basically just provides TDs. That also allows using sensible defaults, or fallback policies.

yes, I think that probably clients would just go for the default in 80% of cases. However, we could factor these requirements in the WOT.discovery.any() function, which might be designed as the current discovery method.

When represented in a single dictionary, they can be specified single or in combination, which is not possible if we separate the functions out

I agree I should have added to the cons. Adding right now. Still, a combination could be achieved calling the different functions one at a time (not ideal I agree).

We can definitely explore using small functions. Maybe the first function to try separately would be directory().
By understanding and fine-tuning the smaller functions, we may also be able to improve the integrated API as well :).
Then we will be in better position to decide.
Let's spend more time experimenting.

Totally agree that we need more time to experiment and wait untill we have a better definition of the different discovery processes inside the Web Of Things world.

[danielpeintner]

At the moment the discover() call takes as argument a ThingFilter with a required method property (e.g., "any", "local", "directory", ..) and some more properties that are useful in on or the other kind of filter context.

What is done in the TD work for DataSchema and SecurityScheme is a bit different. A generic/abstract container is defined and depending on the actual need additional properties are added:

• see DataSchema classes in node-wot
• see SecurityScheme classes in node-wot

I am not saying this is the best approach but for developers it might be useful to have something in the same spirit.

Having a basic ThingFilter and extensions for it: LocalThingFilter, DirectoryThingFilter, ...

[zolkis]

Having a basic ThingFilter and extensions for it: LocalThingFilter, DirectoryThingFilter, ...

That is certainly a more structured way than now to describe the combination of accepted input.

Anyway, the first thing to clarify is the discovery methods. We need at least "direct" and "directory".
Also, we need filters: template-based, query based etc.
Also, it would be nice to have a capability discovery API: what kind of discovery methods and filters are supported.

[relu91]

Anyway, the first thing to clarify is the discovery methods. We need at least "direct" and "directory".
Also, we need filters: template-based, query based etc.

As I see this, we might end up modeling different methods using different functions (as proposed above) and different filters using what @danielpeintner has hinted. We certainly need more iterations and interactions with the Discovery WG.

Also, as discussed today, my proposal did not cover the two-phase discovery architecture. We have to evaluate if we need to include that structure also in the API. Today, the Discovery WG has committed the first draft version of the discovery specification. In the next call, we could start from there. One thing worth to mention is that Direct is considered an Introduction mechanism.

[zolkis]

One thing worth to mention is that Direct is considered an Introduction mechanism.

We need to stop using "direct" then, in order to avoid confusions. However that itself is in contradiction with protocols (like OCF) that do have a "direct" discovery method just like the one in the API.

In our case the result of discover() is TDs, not URLs. But "direct" in the discovery spec seems to cover the same use case with 2 steps, since the URL is meant to be subject to a GET in order to obtain the TD (of the self-describing Thing, or that of a TDD). I am not sure what is the advantage of splitting into 2 steps since the URL itself should not carry identifiable information (it should be as opaque as possible).

[relu91]

I re-read the discussion for the slides of the next vF2F, what we missed to consider as a pro is that the new APIs offer an easy feature detection mechanism. For example, checking if the runtime support local discovery: wot.discovery.local !== undefined. How can we achieve the same with the current spec? Adding this as a Pro.

[JKRhb]

One thing I wanted to add to this discussion is that I'd find it very useful to add some kind of registration interface for new methods to the Discovery API or making the list of possible methods dynamic as the discovery mechanisms can be very protocol specific (as already mentioned by @zolkis). For instance, a CoAP implementation could offer methods like:

• "direct" multicast discovery using /.well-known/wot-thing-description,
• multicast discovery using the CoRE Link Format and /.well-known/core?rt=wot.thing, followed by retrieving TDs from the discovered links,
• discovery using a single CoRE Resource Directory...
• ...or multiple ones which are themselves discovered using multicast (and then retrieving the TDs).

This alone are four different (sub)methods which could be supported by the ThingDiscovery object. In node-wot, for example, these could be provided by the binding implementation and then registered at the servient. Using this kind of approach could make it possible to only specify the direct and directory methods in the Scripting API and leaving the rest to discovery specification and/or a binding document.

I don't know how useful this approach is but I think it would be great to find a way to enable Scripting API implementations to integrate some kind of multicast discovery (this is one of the benefits of CoAP, after all). There are of course some rightful security concerns but hopefully these can be addressed by using approaches like Group OSCORE (which would be yet another submethod) in the future.

[zolkis]

add some kind of registration interface for new methods to the Discovery API or making the list of possible methods dynamic as the discovery mechanisms can be very protocol specific

Only that in Web APIs this is not a recommended design pattern. It's better to list the possible methods and the ones not available would fail. Introspection, if needed, can be solved by other means.

The purpose of the Scripting discovery API is to find TDs, by "any method" (Edit: which we have removed/replaced by "direct"), or by specifying a directory (as a method that is being standardized in WoT). The protocol-specific mechanisms can be grouped under various umbrellas, like "nearby" (covering e.g. Bluetooth, NFC, etc), or "local" (like anyone in the same network/security realm, or "multicast" which is a special case of "any method".

We started by supporting a broad array of discovery methods, but they were not implemented, nor well understood, so we have cut them to the bare minimum, essentially leaving it up to the implementations, except that apps can specify a directory (which is extensible with options about what kind of directory that would be).

I expect we'll discuss these, together with the security and privacy aspects, after the Discovery task force converges on the spec.

[egekorkan]

I would say that even if another protocol offers other ways to "to do stuff" with TDs, they will hopefully fall into the same abstraction of retrieving, searching, updating, deleting, registering etc. That way, the methods on the scripting level can be the same but (maybe with some options) can be used in different protocols?

[JKRhb]

Thank you for your replies! I think you made good points. I think the main question I was asking myself was how to deal with multiple similar discovery methods that all fall under the same umbrella term (for example multicast) -- should all of these methods be used at the same time?

I expect we'll discuss these, together with the security and privacy aspects, after the Discovery task force converges on the spec.

That sounds good -- could one possible (alternative) solution be to have one enum entry for each mechanism specified there and then let the implementation check if the given mechanism works with the available protocols? Or would that be overkill?

[zolkis]

could one possible (alternative) solution be to have one enum entry for each mechanism specified there and then let the implementation check if the given mechanism works with the available protocols?

That sounds sensible, actually the current API covers that.
The function name is discover() and its filter argument may include a property method to specify various mechanisms that may fail with NotSupportedError or other.

We are trying to gather the sensible method names and what do they cover in protocol equivalents.
Suggestions are welcome, nothing is set in stone yet. One constraint: the released API versions should remain backwards compatible wrt later extensions, so whatever names we support, should be stable (well designed, with strong use cases, enough examples, tests etc).