Skip to content

anlumo/capnp-serde

Repository files navigation

capnp-serde

A library for serializing and deserializing Cap'n Proto messages with Serde. It relies on the capnp crate. Note that it does not use the Cap'n Proto message encoding, this crate is for serializing using another format.

Use Case

This crate allows directly converting Cap'n Proto messages to and from another serde-implementing codec, like JSON, YAML or CBOR without any intermediary data storage (such as needed by capnp_conv).

The reason I created this is to two-way translate Cap'n Proto messages for languages that don't have Cap'n Proto support themselves (like Dart in my case, via flutter_rust_bridge, but serde-wasm-bindgen is also a good use case). The crate can easily convert anything to some common schemaless codec and do interop that way.

Another use case might be easily debugging complex data structures, for example by converting them to JSON to investigate in a JSON viewer. Also, it's quite easy to quickly construct a Cap'n Proto message via serde_json::json! (much easier than the capnp API!). Look at the transcode example on how to do that.

Note that the Cap'n Proto schema is still required. This crate uses capnp and capnpc like normal, relying on its introspection feature and capnp::dynamic_value.

Usage

Add the following to your Cargo.toml:

[dependencies]
capnp-serde = "0.1.0"

Then use it like this:

let mut message = capnp::message::TypedBuilder::<my_type::Owned>::new_default();
let mut root = message.init_root();
// do something to `root` here

let serde_reader = CapnpSerdeReader::new(root.into_reader());

// Then you can convert to another format like this:
let json = serde_json::to_vec(&serde_reader).unwrap();
let yaml = serde_yml::to_string(&serde_reader).unwrap();

// Convert from the other format back to Cap'n Proto like this:
let serde_builder = CapnpSerdeBuilder<my_type::Owned> = serde_json::from_slice(&json).unwrap();
let message: capnp::message::TypedBuilder<my_type::Owned> = serde_builder.into_inner();

Format

The format expected by the deserialization is the same as the one generated by the serialization without any leniency. This is not designed as a general parser of any serialization format, but only for documents specifically crafted to adhere to a Cap'n Proto schema. The only exception is that number formats are freely converted when possible (also, JSON, CBOR and other formats only have a single number type).

That said, Cap'n Proto is very versatile, so it might be possible to convert a limited set of generic input data by purposefully crafting the schema in a certain way.

Limitations

Since Cap’n Proto uses arena-style memory allocation and builds the message in-place, it fundamentally requires you to know the size of lists ahead of time. There’s no real way around this with the official Rust capnp crate.

Many deserializers supply the array/list size upfront as a hint to the decoder, which solves the problem. However, other decoders do not. There's a workaround implemented for everything except lists and structs, which involves an extra copy of the whole list (via Rust's Vec<T>).

Anypointer and capabilities are not implemented at all.

Examples

There are a few examples included in the project to demonstrate (and test) the usage, as found in the examples directory. You can run them using

cargo run --example <name> --features examples

The examples feature is required, because only then the example capnp schema is built. Unforunately, there is no build.rs specific to examples, and the capnp crate needs to generate code.

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

About

Directly bridge between Cap'n Proto and serde

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published