Docs Question: Making a trigger plugin

[ethanfrey]

First of all, I have used WASM for some time and wanted to explore the potential of WASI when I found this project. Great stuff! Nice design and it really shows how to make some WASI-first architecture and re-design how we think of components and services. I am very inspired by the architecture.

I am looking to make a custom task manager, which could be grafted onto HTTP somehow I guess, but I would really love to make a custom trigger type for my project to make that work. I looked into the code and see how you can add triggers via plugins and also see this approach discussed in an issue.

I'd really like to understand how one could add such a plugin. The plugin documentation references one example: spin-timer. This is a huge step forward, and I can see how to implement the run of TriggerExecutor. However, it also raises a number of questions, especially around the macros (wasmtime::component::bindgen and wit_bindgen::generate), the *.wit files themselves, and this Guest implementation.

If there is any more documentation internally (or in wasmtime) that would help explain this, I would greatly appreciate a link. If not, it would be awesome if there was some more in the spin-timer example - either inline comments or in the README.

Thank you again for the project and I look forward to building on it

[lann]

Hi @ethanfrey; good question. The Extending and Embedding Spin section of the docs covers this in somewhat more detail, though it's still true that the topic isn't very well covered.

For the Component Model (and Wasmtime), you might be interested in the Component Model doc site.

[ethanfrey]

Hi @lann

Thank you for the response. Those links were very helpful. Indeed a deeper understanding of the component model and WIT format was very helpful for my mental model. And the first docs link and the attached sources answered many of those questions.

However, I have one specific question left. I see how the guest works - wasi_bindgen, along with the import and export. On the host side, I also see the bindgen that should create a SpinTimer interface type, like the bindgen rustdocs say.

This does explain how we get the nicely typed export to call into the guest. But I still don't see where the code injects the imports (variables.wit in this case) into the runtime. I am looking for something equivalent to HelloWorldImports in the bindgen rustdoc.

I guess if we magically inject all standard spin types, that is a fair answer. But I was also wondering how I could inject custom imports to the guest, so a deeper understanding of this would be helpful. I am guessing this is somehow done inside the spin_trigger::TriggerAppEngine type, which wraps the lower-level wasmtime::Engine type referenced in those bindgen docs. Maybe it is simple and I have just been staring at code too long, but any pointers here are appreciated.

I have used "vanilla" Wasm quite a bit but not the component model or WIT and this really makes a lot of things nice and clean. I appreciate the design and architecture of spin and am learning some new best practices.

[lann]

Ah, adding imports for custom triggers is indeed entirely undocumented afaik. I'll try to outline the necessary steps here; apologies if I miss something:

• Implement TriggerExecutor::configure_engine, which gives you access to the EngineBuilder at app start time
• Now you have two options:
  • The simplest (or intended to be simplest anyway...) option would be to implement a HostComponent. This is how Spin's own host imports are implemented; see OutboundMqttComponent for a relatively simple example. This HostComponent would be added via EngineBuilder::add_host_component
  • More complex (and flexible) would be to use EngineBuilder::link_import directly, which requires a somewhat deeper understanding of spin-core internals. In particular you'd need to store your Host impl in your TriggerExecutor::RuntimeData and figure out the proper add_to_linker closure to extract that from Data. I'd be happy to help here too if you want/need to go this route.

[lann]

I should also add: we are planning to embark on a major redesign of spin-core in the next ~quarter which will hopefully make some of the above more obvious and well-documented.

[ethanfrey]

Thanks for the quick reply. I just found where the variables interface is registered but that didn't seem to provide an extension point for custom code, as it uses a private member of TriggerExecutorBuilder

I will look more into the OutboundMqttComponent. That seems like a promising start.

I am not hacking code yet... still trying to figure out how to architecture this and what it is capable of. One I figure it out, I'd be happy to add some docs from my perspective - incomplete but maybe covering basic parts for novices. I am keeping a local markdown file of my learnings now.

[lann]

that didn't seem to provide an extension point for custom code, as it uses a private member of TriggerExecutorBuilder

Indeed there are some limitations here and in fact that exact block of code you pointed out is one of the major reasons for the aforementioned refactor. 🙂

[ethanfrey]

I got a custom plugin trigger working and made custom wit interfaces to call a WASI guest, which makes use of various spin APIs and it is working well. I haven't dug into adding the custom imports, but got everything else going pretty nicely and feel a much better understanding of the code. Thank you for your assistance.

If anyone else is reading this thread later, besides the docs linked above, I can recommend the following examples to use as basis for your own work:

• cron-plugin to shows not only how to write a plugin trigger, but also how to provide a nice sdk with macros and all
• spin-rust-sdk examples are the most up to date and complete demos of how to call the various spin APIs from within your WASI program

[ethanfrey]

I did hit some issues in using the outbound http requests from my custom guest interface, due to the use of async functions. Also one trick I found (which might be a hack?) was to wrap it inside http::run. The code looks something like...

wit_bindgen::generate!({
    world: "spin-periodic",
    path: "../../../wit",
});

struct OracleQuery;

impl Guest for OracleQuery {
    fn handle_periodic_request(meta: Metadata) -> Result<(), Error> {
        run_http(async { 
            let res = Self::_handle_periodic_request(meta).await;
            res.map_err(|e| Error::Other(e.to_string()))
        })
    }
}

impl OracleQuery {
    async fn _handle_periodic_request(meta: Metadata) -> anyhow::Result<()> {
       // outgoing http request here...
    }
}

The generated wit expects to call into a non-async function but http::run will allow us to safely convert an async function (which can make http and network calls) into a sync function (which is a valid wasm component export). #[http_component] seems to handle this for you magically, and I never saw async used in other contexts. Luckily I found http::run in cargo expand output, so I assume this is the proper way to handle running async inside of WASI.

[fibonacci1729]

@ethanfrey Definitely not a hack and is the right way to do it at the moment; http::run is just re-exported from the spin-executor crate which implements an async executor for the sdk.