-
Notifications
You must be signed in to change notification settings - Fork 10
The common API #23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I really like this approach
Why keeping them in tree is needed?
we could also proceed with both, on the sugondat-chain the more complex thing is to setup the RPCs that are able to able to use some runtime-apis so maybe created the base layer having both: common divisor and ah-hoc api should not be that complex
Asking to the da for ALL blobs of a namespace inside a specific block, is split in a number of rpc calls equals to 1+n where n is the number of blobs submitted in the block, why not something like |
well, to add another adapter would mean that the devs of that rollup stack will have to fork the project and implement their set of endpoints (assuming that they don't want to use the existing endpoints for other adapters). Then they either keep it out of tree or upstream it making it in-tree. Another alternative, is that we provide an extension point somewhere, but that feels kinda overkill and goes against the idea of having a single binary as I described in #24. But possible.
TBH, I am not sure if I follow. But generally going for both adhoc and the common denominator doesn't make sense to me, because we will have to spend time on it to make common denominator API reasonable (or do a half-assed approach), then support both versions. It's a no go.
Well, yes, that's the problem I was trying to describe. There is a natural tradeoff between the flexibility of the API and its efficiency. Maybe, it is the common way to ask blobs in such a way, and it does seem to me this way as well, but what are you going to do when that's not the case? Probably, you will add the bare versions of the API. But now you will have to maintain all of those new APIs (you don't want to break older clients do you?). Are those efficiency gains worth it? Maybe or maybe not. That also shows the advantage of the ad-hoc APIs. You don't touch the old APIs, if the adapter changes you add new API, if something requires change, that something is not public API. |
the added set of RPCs is upstreamed, not the adapter code itself, right ?¿
yeah, that's true, in my initial equation I imagined the difficulty of creating a common denominator API too low.... for sure satisfy all different adapters is not easy
I see your point now... decide something like this require some security on the functioning of the system and right now we don't have this
yeah, and this could also fit with the shim described in #24 where it changes overtime with the adapters evolving |
yep! |
Do I have this right? The proposed architecture is that every adapter is a thin wrapper around a spawned child process which is the API worker and which implements APIs for every adapter to be served over some serialized query/subscription protocol like JSON-RPC. Internally these APIs would be implemented with something like the common-denominator API, which is itself not exposed. |
Almost.
I think spawning a child process is a unnecessary complication. I was envisioning that the shim process would be launched by a docker-compose or another similar program.
Yes, althought I am not sure if those should be enabled/exposed by default. Maybe for e.g. opstack you may need to pass an
I was hoping to avoid JSON-RPC if possible. That's tangential anyway.
Yeah, kinda, but not necessarily. It should be just reasonably designed. |
Closed by #32 |
Uh oh!
There was an error while loading. Please reload this page.
There are several rollup stacks out there. E.g. #19 #8 #9 and sovereign-sdk.
Sov is Rust. Others in Go. Probably there may be yet others in completely different languages (although, not very likely). We don't want to reimplement all the intricacies in every adapter. Therefore, it may be worthwhile to push most of the heavy lifting into what we called previous an API.
There are several approaches to build such an API:
/sovereign_v1/finalized_at/:height
and/rollkit_v2/get_blob/:id
.While the common denominator approach seems cleaner/more elegant/more aesthetically pleasing, I think the ad-hoc API approach is the way to go, at least for the beginning.
I think getting the common denominator API is tricky. Every new adapter may change the whole thing completely and we may still need to add the ad-hoc things for specific adapters.
I am pretty sure we will have some breaking changes initially. With the ad-hoc APIs we don't have to spend too much time to foresee all the possible use-cases, we just implement the current API in the form of RPC and that's it. There still may be a new adapter that challenges some foundational assumptions, but that would affect the internal API/the implementations which should be easier to change compared to the common denominator RPC which a public API. Such a change may affect the adapters who were perfectly happy with the previous version of the API (most of them probably).
The elegance of the common denominator API also comes at a cost: while ad-hoc API will return all the required information for the adapter call in one go, the common denominator API may require several calls and further logic on the adapter side.
OTOH, the ad-hoc APIs allow making the adapters dumb. All the heavy lifting is performed in the implementation of API. Making the adapters dumb is desirable because it makes the whole system more amenable for testing. Arguably, dumb adapters don't require as much testing besides some integration tests and we can cover the common Rust logic more extensively than we could ever do in adapters.
The ad-hoc API has a downside, that probably the adapters will have to be in-tree. I think this is fine. We can provide the adapters for select rollup stacks. For example I wonder if #9 #8 #19 cover a big chunk of rollups already. Realistically, we cannot expect that stack devs will line up to implement integrations with sugondat, so we will have to kinda do it ourselves at least initially.
Also, picking the ad-hoc API way doesn't mean we should stick forever to it. Maybe, if there's enough demand, we could add a common denominator API later on, when we gain enough experience and things may stabilize.
With all that said, I will present my current vision of the common denominator API below. The reason is, it should be obvious how to structure the ad-hoc APIs, however, as I mentioned, the common denominator API is less obvious. Also, perhaps, this API may inform under the hood abstractions for the ad-hoc API approach.
Notes:
height
— a finalized block number. In every API so far it was u64 and I am ok with it. In JSON we may consider using String to encode it.id
is an identifier of an included blob, e.g. height and extrinsic index or height and extrinsic hash.AbridgedHeader
consists of:Blob
= VecNamespace
, TBD by Determine Namespace ID size #12The text was updated successfully, but these errors were encountered: