RFC: How should CDK support LDK going forward?

[thesimplekid]

CDK currently supports LDK through an in-process ldk-node backend. This gives operators a simple deployment model: run one cdk-mintd binary and get both the mint and Lightning node.

The downside is that CDK also owns more Lightning node operational surface than it does for other backends. For example, CDK has had to maintain its own LDK node dashboard and manage the node lifecycle inside the mint process.

ldk-server now provides an ldk-node based daemon with a gRPC API. This would make LDK work more like CDK's CLN, LND, and external payment processor integrations: the mint connects to a separate Lightning service.

This raises the question: what should CDK's long-term LDK support look like?

Options

1. Keep only the current embedded ldk-node backend

Pros:

• one binary for operators
• simple local development and testing

Cons:

• CDK keeps maintaining node management UI and lifecycle code
• mint upgrades stop the Lightning node
• can not run multiple mint instances against one node

2. Support both embedded ldk-node and ldk-server

Pros:

• preserves the simple single-binary option
• adds a service-separated option for production deployments
• gives CDK time to evaluate ldk-server as it matures

Cons:

• two LDK backends to maintain
• more docs, config, tests, and support complexity

3. Add ldk-server support and eventually remove embedded ldk-node

Pros:

• aligns LDK with CDK's other Lightning backends
• removes CDK's need to maintain an LDK dashboard
• separates mint and node lifecycles
• enables multiple mint instances to share one Lightning node

Cons:

• removes the current one-binary deployment model
• may be premature while ldk-server APIs are still stabilizing

Open questions

• Should ldk-server be an optional backend, a future replacement, or out of scope for now?
• If both are supported, which should CDK recommend for production?
• Should CDK keep maintaining the embedded LDK dashboard long term?

Possible conservative path

Add ldk-server as an experimental backend while keeping embedded ldk-node. Once ldk-server is more mature and CDK has integration experience, revisit whether the embedded backend and dashboard should be deprecated.

My current preference

I am leaning toward adding ldk-server support and removing support for the integrated ldk-node backend over time.

The main reason is that it removes the need for CDK to maintain a CDK-specific Lightning node dashboard. If ldk-server becomes the LDK deployment target, CDK can lean on LDK-native tools and other operational tooling instead of owning that surface itself.

I also think separating the mint and Lightning node is the better model at scale. It allows the mint layer to be horizontally scaled while sharing one underlying node, and it keeps the mint and Lightning node as separate services with separate lifecycles. They are distinct pieces of infrastructure, and treating them that way seems like the cleaner long-term architecture.

Already a PR adding support for it here: #2164

[gandlafbtc]

is there a performance/speed difference that needs to be considered?

[orangeshyguy21]

I'm in favor of a plan based on option 3), with the conservative path given ldk-servers current instability, that phases in ldk-server with a phase out plan for ldk-node for similar reasons to the ones stated.

"The main reason is that it removes the need for CDK to maintain a CDK-specific Lightning node dashboard. If ldk-server becomes the LDK deployment target, CDK can lean on LDK-native tools and other operational tooling instead of owning that surface itself.

I also think separating the mint and Lightning node is the better model at scale. It allows the mint layer to be horizontally scaled while sharing one underlying node, and it keeps the mint and Lightning node as separate services with separate lifecycles. They are distinct pieces of infrastructure, and treating them that way seems like the cleaner long-term architecture."

Going forward, an appropriate principle for the project to adopt when it comes to backend support could be to only own the implementation if cdk is doing something bespoke that an official implementation does not, or will not support.

I think this fits squarely in that framing where there is little to nothing gained, in terms of function, that justifies the increase in scope.

[prusnak]

I am leaning toward adding ldk-server support and removing support for the integrated ldk-node backend over time.

+1

[thesimplekid]

is there a performance/speed difference that needs to be considered?

It's a good question but I do not think a meaningful one they are both ldk-node under the hood; so should be pretty equal there but it there would be one extra grpc round trip for ldk-server. However, I don't think this would be noticeable given everything else that happens in the chain for ln operations.

[prusnak]

Maybe not the best place to discuss this, but here goes anyway:

1. Should we remove cln/lnd/ldk crates and replace with cdk-*-payment-processor?
2. What about spark/bark/etc? Should we keep them separate as grpc payment processors or should we just move them into main repo as cdk-spark/bark crates?
3. If answers for 1 and 2 are different, where do we draw the line?

[thesimplekid]

Maybe not the best place to discuss this, but here goes anyway:

1. Should we remove cln/lnd/ldk crates and replace with cdk-*-payment-processor?

2. What about spark/bark/etc? Should we keep them separate as grpc payment processors or should we just move them into main repo as cdk-spark/bark crates?

3. If answers for 1 and 2 are different, where do we draw the line?

Fair question.

My thinking is that we don’t want to force the use of the payment processor/gRPC model for things we consider first-class supported backends. For those, I think it makes sense to keep them in the main repo as native CDK crates.

As for where to draw the line, I think it should be around the things we support directly and are willing to maintain long-term. So for Lightning that would be CLN, LND, and LDK(ldk-server?), and for on-chain that would be BDK.

Then I think we can have another category of payment processors that we support, but more experimentally. I would put Bark and Spark here for now. They are both pretty new, and I don’t think we should fully commit to them as first-class in-tree crates yet.

That said, we should probably do a better job surfacing what secondary payment processors we support. Right now some of these are just in my personal GitHub and are not very discoverable or well maintained. Even if they stay outside the main repo, we should make it clear which ones exist, where they live, and what level of support they have.

LNbits is currently in the repo, but we already said for the next release we are removing it and moving it to a payment processor.

[prusnak]

Maye we could create a cdk-payment-processors monorepo and put the payment processors (bark, spark, lnbits, template) there?

This would simplify discoverability and maintenance. We can have great CI workflows which detect breakages in existing processors and would serve as acceptance criteria for accepting new processors into the monorepo. But we can still be very clear that these are not first-class in-tree crates.

[vincenzopalazzo]

I still need to follow all the discussion here, but I think ldk-server and ldk-node are just serving two different users.

In process ldk-node would be amazing for people who do not run a lightning node already or do not want to run one. You can provide an ldk-node out-of-the-box support, and this means that you can have a one-shot command to run the mint

However, people who run a lightning node for other purposes, like me, would benefit from just pointing the mint server to the already existing lightning node

with the conservative path given ldk-servers current instability,

I would not consider ldk-server instable due that use ldk-node under the hood, probably just the API could be unstable and some bugs here and there that would not be a problem if they are not inside ldk-node.

If I were you, I was going to yolo the support of ldk-server as another implementation due to the amount of review work that is going on in ldk and ldk-node. People should also consider this in contrast to other implementations that may have one or none reviewers.

[ConorOkus]

Just to follow up some more on what @vincenzopalazzo wrote. Embedded ldk-node is a great DX and shouldn't regress just because we add a service-separated option. I think ldk-server can cover that case too via docker compose/files, etc. If we get that right, supporting ldk-server is not really a trade-off against a simple deployment; it just becomes the same simple deployment with the node split out.

Option 3 seems reasonable to me.

[TheBlueMatt]

Another point @ConorOkus raised privately is how this might fit into a mint-in-TEE model. Obviously if, as a part of that shift, mints move to not trusting the lightning node (ala fedi's lightning gateway design) its somewhat moot, but if a goal is to enable TEE while retaining the same DX of "one-click run" an embedded LN node would be a hard requirement. Sadly, today, ldk node can't support running in a TEE, but we're working on supporting WASM which is likely to be the bulk of the refactors to enable TEE operation.

[callebtc]

also in favor of option 3

[prusnak]

if a goal is to enable TEE while retaining the same DX of "one-click run" an embedded LN node would be a hard requirement.

It is possible to run multiple processes in a TEE - Confidential Virtual Machines (e.g., AMD SEV, Intel TDX) have a VM as boundary. And DX is still very good (you essentially deploy a Dockerfile into a TEE).

Sadly, today, ldk node can't support running in a TEE

Should be trivial in a Confidential Virtual Machine.