Faster, cheaper, OData JSON serialization #3236
Description
As part of a larger effort to make OData libraries highly performant, we should consider building a faster and more efficient serialization stack since the serialization is one of the biggest perf bottleneck and source of performance complaints in OData.
We're hitting the limits of improvements we can make to the current serializer without significant architectural and breaking changes. ODL 9 presents an opportunity to rethink the serializer.
The serializer could first be roll out as an opt-in alternative to the current one as it gets evaluated and battle-tested before considering making it default or phasing out the current one.
Is your feature request related to a problem? Please describe.
We have made efforts to improve OData library performance, but it's still far from desired. Customers take it as a given that they must sacrifice performance when they switch from JsonSerializer to OData serialization. While some performance gap may be expected due to additional work we do to implement and validate OData conventions and rules, experiments have shown that we can reduce the gap significantly, but we are limited by the existing architecture. In some ways, we have hit a wall when it comes to perf improvements with the current architecture and are sometimes resorting to micro-optimize at the margins
See:
- Performance concern on ODataOutputFormatter AspNetCoreOData#959
- Microsoft.AspNetCore.OData v8.0.3 Thoughts And Performance AspNetCoreOData#359
Here are some of the key bottlenecks in the current architecture
Push-based model based on weakly-typed ODataResource
The customer's input, usually a POCO object (but could be any type of representation) is transformed to an ODataResource object as an intermediate serialization step. The ODataResource objects will be passed to the intermediate layers of writers which will validate the object, add annotations, etc. then write properties and values to the lower-level IJsonWriter.
ODataResource (and related types) is a dynamic, weakly-typed bag of properties, annotations and other metadata. This allows OData to be dynamic and easily handle things like $select/$expand, dynamic properties, computed properties, arbitrary annotations, different subtypes, etc. But it comes at a great cost since we create an instance for each resource (entity, complex property) in the response. There are a lot of expensive operations related to this type, a lot of reflection, a lot of boxing, etc.:
See:
Repeated computations, lookups and redundant data
We perform a lot of metadata computation and validation per instance, even when dealing with instances of the same type and shape. In the common case, a lot of this is redundant and would only need to be performed once per type. In some cases, some validation is unnecessary when we know that the ODataResource was correctly constructed by AspNetCoreOData's serializer, for example property type verification, or duplicate name checker, or whether the type or property belongs to the schema, etc.
- ODataResourceSerializer and ODataResource.SkipPropertyVerification AspNetCoreOData#1258
- Perf: investigate repeated metadata lookups when reading/writing payloads #2480
Async overhead
When we added support for true asynchronous I/O, we create an async version of all methods that might perform an I/O call. Since we have a lot of intermediate layers of writers between ODataMessageWriter and the implementation of IJsonWriter, this results in a lot of intermediate async methods calls for any given write, even when the write doesn't result in actual I/O call (e.g. buffered writes). This resulted in a lot of overhead from the async state machines and Task allocations. As a result, the latency of an async call (without accounting for network latency) is considerably higher than the non-async version and considerable higher then async JsonSerializer. We made attempts to clean this up like:
- Basing a new
IJsonWriteronUtf8JsonWriterwhich only performs async calls when flushing to the output stream - Eliding async calls in some cases to avoid unnecessary async state machines
- Replacing Task with ValueTask
These made improvements, but it was applying bandaid to wound. There was only so much we could do without introducing breaking changes.
Overhead for things we don't use
There are cases in OData where we pay an overhead for things we don't use. For example, we allocate collections for Actions and Functions per resource, even though these are only needed when using full-metadata, which is not the common case.
Describe the solution you'd like
- A more efficient and more performant OData JSON serializer. A reasonable upper bound we could aim for is 2x the latency. I'd expect at least 80% reduction in latency and memory allocations from the current serializer in common scenarios.
- Integration with JsonSerializer and JsonConverter to make adoption smoother, especially in minimal APIs
- Don't pay for what you don't use: as a matter of principle, we should avoid overhead in the common case for things that are only beneficial in edge cases
- Support full range of OData capabilites:
- Flexble: can support different types of inputs: POCOs, untyped scenarios,
- Supports for selecting properties based on $select, $expand
- Customizable: user can determine how to map EDM types to domain types, EDM properties to domain properties
- Support for dynamic properties
- Support for computed properties
- Support for default annotations based on metadata mode
- Support for custom annotations
- Support for polymorphic payloads
- Support for streaming
We could release this in phases, as an opt-in feature until it reaches the maturity and real-world testing required to consider making it the default.
My proposal is to use a "pull" model based on custom writers and resolvers that define how to write the properties of a given type, given an instance and some context, similar to JsonConverter approach. We could create default implementations for common cases like POCO classes that could be used by default in AspNetCoreOData, but users can customize by providing their own implementations.
Describe alternatives you've considered
As mentioned, we've made a lot of progress in improving the performance of the current serializer, but we're reaching the limit of what we can do with the current architecture and without breaking changes.
Would you like to work on this feature?
- Yes
- No
- Maybe (need guidance)
🔹 Thanks for taking the time to suggest this feature! We appreciate your contributions. 🙌