WASM pipelines

[gedw99]

https://github.com/ConduitIO/conduit/releases/tag/v0.9.0

I am going to try to get conduit working with WASM go-SQLite.

All the conduit data source processors run as WASM and the platform has a CDC standard data format.

It useful to me because then you can “tap” all these external conduit data sources , and update higher level systems. It’s CQRS.

I will see if I can write a plugin for go-sqlite3. Then this is also a data source for the CDC system.

It has some bearing on the discussion of using Litestream , because it’s also a CDC pattern and so figured others may be curious etc.

It’s a great way to built data systems where only the changes at the base data cause rippling events from that origin outwards allowing layers and layers of caching outwards or workflows to react to data changes.