Add wallet creation example using RPC #519

rajarshimaitra · 2022-01-06T20:13:52Z

Description

As mentioned in bitcoindevkit/bitcoindevkit.org#83 (comment) it would be helpful
to have wallet creation example code in the example directory.

This is an attempt to create the most simplistic bdk wallet with a RPC backend and demonstrate receiving, creating, signing
and broadcasting transaction.

The code is a refinement of the RPC tutorial https://bitcoindevkit.org/blog/bitcoin-core-rpc-demo/ with more elaborate doc comments.

Notes to the reviewers

To automate the background RPC process electrsd is used which is already a dev-dependency of bdk.

Checklists

All Submissions:

I've signed all my commits
I followed the contribution guidelines
I ran cargo fmt and cargo clippy before committing

rajarshimaitra · 2022-01-07T16:46:40Z

Apparently cargo test is trying to run the example which has some extra feature dependency and causing the test failure in CI.

I am not sure why thats not the case for other examples though (they also have special required deps).

@notmandatory @afilini am I missing something in the cargo configuration?

rajarshimaitra · 2022-01-14T13:48:22Z

Fixed build failure, all tests passing..

ConorOkus · 2022-01-14T20:20:01Z

Trying to run this locally at the moment, but failing. What command should I run? cargo run --example rpcwallet?

rajarshimaitra · 2022-01-15T08:13:00Z

@ConorOkus this should work cargo run --features="all-keys key-value-db rpc" --example rpcwallet.

ConorOkus · 2022-01-16T20:05:35Z

tACK

LLFourn · 2022-01-25T22:16:39Z

utACK does the example get run in the ci?

rajarshimaitra · 2022-01-27T10:28:00Z

utACK does the example get run in the ci?

I think currently it doesn't. But we can explicitly ask the CI to run examples I think. Haven't tried that out.

afilini · 2022-01-27T11:31:13Z

I think it's ok if we just compile this example in the CI, I'm guessing what it does is already covered by specific unit tests

ConorOkus · 2022-01-27T16:51:06Z

I'm trying to do a similar thing in LDK here

afilini · 2022-01-28T10:28:17Z

It's not as easy for BDK because different examples might require different features. So if you just compile them like you did, they will most likely fail.

You would have to get the list of features from the Cargo.toml and then compile using those

rajarshimaitra · 2022-02-09T13:28:44Z

Retested

All looks good on this one to me.. Its good to go I think..

notmandatory · 2022-02-11T04:06:56Z

This example LGTM but I'm going hold off merging it until #535 goes in so it can be refactored as needed first. Also removing from next milestone since we're already a bit behind and have some higher priority PRs to get in.

xraid · 2022-03-18T04:03:22Z

has this been massaged to accommodate new stuff in merged #535

rajarshimaitra · 2022-03-21T04:48:25Z

has this been massaged to accommodate new stuff in merged #535

Good point, This will definitely have effect on the example. Will update it accordingly..

notmandatory · 2022-03-21T19:29:12Z

I think this PR also needs a rebase to pick up the latest verify changes, in particular related to the CI Blockchain jobs.

rajarshimaitra · 2022-03-22T07:08:25Z

Fixed the example in light of #535 and rebased..

notmandatory

I tried testing but got an error, do I need to do something else to run it?:

% cargo run --features keys-bip39,key-value-db,rpc --example rpcwallet
warning: field is never read: `network`
  --> src/blockchain/rpc.rs:58:5
   |
58 |     network: Network,
   |     ^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: `bdk` (lib) generated 1 warning
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/examples/rpcwallet`
>> Setting up bitcoind
>> bitcoind setup complete
Available coins in Core wallet : 50.00000000 BTC

>> Setting up BDK wallet
Error: Rpc(JsonRpc(Transport(Timeout)))

examples/rpcwallet.rs

This adds an example wallet creation code with sled DB and RPC Blockchain. The backend RPC is managed using electrsd::bitcoind

rajarshimaitra · 2022-03-23T06:32:31Z

Thanks @notmandatory for the review.. I fixed the nits.

I tried testing but got an error, do I need to do something else to run it?:

This seems very weird. Its not happening in my local. And it seems the error is originating from blockchain::rpc not the example. Thats even more odd. But I am not sure how to reproduce it.. Probably because of a different rustc version you are using??

xraid · 2022-03-27T11:42:34Z

i get :

Setting up bitcoind
bitcoind setup complete
Available coins in Core wallet : 50.00000000 BTC

Setting up BDK wallet
Error: Rpc(JsonRpc(Transport(SocketError(Os { code: 35, kind: WouldBlock, message: "Resource temporarily unavailable" }))))

when i swap out in file rpcwallet.rs to use :

use bdk::bitcoincore_rpc::{Auth as rpc_auth, Client, RpcApi};

with Client invocation dance => instead of using :

use electrsd;

the BDK wallet finds bitcoind and all is good ...

While running a local regtest bitcoind with correct auth ...

rajarshimaitra · 2022-03-28T11:35:33Z

Thanks @xraid for reporting.. I am still not being able to reproduce any of the issues. Its all working fine in my local as normal.

Did you mean you have to do the bitcoin core rpc setup manually instead of using electrsd crate and then it worked?

xraid · 2022-03-28T15:06:32Z

the err says it blocks ? @afilini mentioned something about sharing connections to backends - that might not be the issue here, maybe it is the bitcoin.cookie in auth for the bdk-wallet ?

i changed and used Client instead of the electrsd lines [50...75]

line 103 reuse bitcoind object for bdk-wallet that was used for core-wallet prior
let rpc_config = RpcConfig { url: bitcoind.params.rpc_socket.to_string(), auth: bitcoind_auth,

i am still to fresh in Rust and the BDK code base to make any sense in try debug,

so i leave it to You and meanwhile watch ...

rajarshimaitra · 2022-03-29T10:54:10Z

the err says it blocks ? @afilini mentioned something about sharing connections to backends - that might not be the issue here, maybe it is the bitcoin.cookie in auth for the bdk-wallet ?

This should not be an issue because elecrsd deploys a local bitcoind only for the code in concern, and deletes it once the example ends.. So there should be only one instance of bdk trying to connect with the temporary bitcoind.

i changed and used Client instead of the electrsd lines [50...75]

In that case did you manually deploy the regtest node in the background?

Ya I will keep digging to see if I can repro this..

rajarshimaitra · 2022-03-29T10:54:16Z

the err says it blocks ? @afilini mentioned something about sharing connections to backends - that might not be the issue here, maybe it is the bitcoin.cookie in auth for the bdk-wallet ?

This should not be an issue because elecrsd deploys a local bitcoind only for the code in concern, and deletes it once the example ends.. So there should be only one instance of bdk trying to connect with the temporary bitcoind.

i changed and used Client instead of the electrsd lines [50...75]

In that case did you manually deploy the regtest node in the background?

Ya I will keep digging to see if I can repro this..

rajarshimaitra · 2022-03-29T10:54:37Z

the err says it blocks ? @afilini mentioned something about sharing connections to backends - that might not be the issue here, maybe it is the bitcoin.cookie in auth for the bdk-wallet ?

This should not be an issue because elecrsd deploys a local bitcoind only for the code in concern, and deletes it once the example ends.. So there should be only one instance of bdk trying to connect with the temporary bitcoind.

i changed and used Client instead of the electrsd lines [50...75]

In that case did you manually deploy the regtest node in the background?

Ya I will keep digging to see if I can repro this..

xraid · 2022-03-29T15:03:27Z

i run a local Nigiri regtest bitcoind electrum and used the bdk::bitcoincore_rpc::Client for core-wallet and for bdk-wallet i use :

bdk::blockchain::rpc::RpcBlockchain

where i added auth and url in config :

// Set RPC username and password
let auth = Auth::UserPass {
    username: "admin1".to_string(),
    password: "123".to_string()
};
// Set RPC url
let mut rpc_url = "http://".to_string();
rpc_url.push_str("127.0.0.1:18443");

// Create a RPC configuration of the running bitcoind backend we created in last step
// Note: If you are using custom regtest node, use the appropriate url and auth
let rpc_config = RpcConfig {
    url: rpc_url, // bitcoind.params.rpc_socket.to_string(),
    auth: auth, //bitcoind_auth,
    network: Network::Regtest,
    wallet_name,
    skip_blocks: None,
};

// Use the above configuration to create a RPC blockchain backend
let blockchain = RpcBlockchain::from_config(&rpc_config)?;

so there is 2 diff connections to bitcoind rpc - one for core and one for bdk

Why the example with using the use electrsd; succeeds with first connect to core wallet and fails when trying to connect from initiated bdk wallet, i dont know ...

notmandatory

tACK 8a98e69

I verified this runs on my Linux (ubuntu 20.04) box. I'm still getting an error on my macOS system, but I suspect it's something specific to the mac. I'd like to know what's blocking it on the mac but not a reason to hold up this PR.

rajarshimaitra force-pushed the example-rpc-wallet branch from 0aa3d65 to 48a250a Compare January 6, 2022 20:16

rajarshimaitra mentioned this pull request Jan 6, 2022

Document structure improvement bitcoindevkit/bitcoindevkit.org#83

Closed

rajarshimaitra force-pushed the example-rpc-wallet branch from 48a250a to 5c0315a Compare January 14, 2022 13:32

notmandatory assigned rajarshimaitra Feb 1, 2022

notmandatory added this to the Release 0.17.0 Feature Freeze milestone Feb 8, 2022

notmandatory modified the milestones: Release 0.17.0 Feature Freeze, Release 0.18.0 Feature Freeze Feb 11, 2022

rajarshimaitra force-pushed the example-rpc-wallet branch 2 times, most recently from 14cb771 to a722c52 Compare March 22, 2022 07:07

notmandatory reviewed Mar 22, 2022

View reviewed changes

examples/rpcwallet.rs Outdated Show resolved Hide resolved

examples/rpcwallet.rs Outdated Show resolved Hide resolved

examples/rpcwallet.rs Outdated Show resolved Hide resolved

examples/rpcwallet.rs Outdated Show resolved Hide resolved

Add rpc wallet creation example

8a98e69

This adds an example wallet creation code with sled DB and RPC Blockchain. The backend RPC is managed using electrsd::bitcoind

rajarshimaitra force-pushed the example-rpc-wallet branch from a722c52 to 8a98e69 Compare March 23, 2022 06:30

notmandatory approved these changes Mar 29, 2022

View reviewed changes

notmandatory merged commit ff6db18 into bitcoindevkit:master Mar 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add wallet creation example using RPC #519

Add wallet creation example using RPC #519

rajarshimaitra commented Jan 6, 2022

rajarshimaitra commented Jan 7, 2022

rajarshimaitra commented Jan 14, 2022

ConorOkus commented Jan 14, 2022

rajarshimaitra commented Jan 15, 2022

ConorOkus commented Jan 16, 2022

LLFourn commented Jan 25, 2022

rajarshimaitra commented Jan 27, 2022

afilini commented Jan 27, 2022

ConorOkus commented Jan 27, 2022

afilini commented Jan 28, 2022

rajarshimaitra commented Feb 9, 2022 •

edited

Loading

notmandatory commented Feb 11, 2022

xraid commented Mar 18, 2022

rajarshimaitra commented Mar 21, 2022

notmandatory commented Mar 21, 2022

rajarshimaitra commented Mar 22, 2022

notmandatory left a comment

rajarshimaitra commented Mar 23, 2022

xraid commented Mar 27, 2022 •

edited

Loading

rajarshimaitra commented Mar 28, 2022

xraid commented Mar 28, 2022 •

edited

Loading

rajarshimaitra commented Mar 29, 2022

rajarshimaitra commented Mar 29, 2022

rajarshimaitra commented Mar 29, 2022

xraid commented Mar 29, 2022 •

edited

Loading

notmandatory left a comment

Add wallet creation example using RPC #519

Add wallet creation example using RPC #519

Conversation

rajarshimaitra commented Jan 6, 2022

Description

Notes to the reviewers

Checklists

All Submissions:

rajarshimaitra commented Jan 7, 2022

rajarshimaitra commented Jan 14, 2022

ConorOkus commented Jan 14, 2022

rajarshimaitra commented Jan 15, 2022

ConorOkus commented Jan 16, 2022

LLFourn commented Jan 25, 2022

rajarshimaitra commented Jan 27, 2022

afilini commented Jan 27, 2022

ConorOkus commented Jan 27, 2022

afilini commented Jan 28, 2022

rajarshimaitra commented Feb 9, 2022 • edited Loading

notmandatory commented Feb 11, 2022

xraid commented Mar 18, 2022

rajarshimaitra commented Mar 21, 2022

notmandatory commented Mar 21, 2022

rajarshimaitra commented Mar 22, 2022

notmandatory left a comment

Choose a reason for hiding this comment

rajarshimaitra commented Mar 23, 2022

xraid commented Mar 27, 2022 • edited Loading

rajarshimaitra commented Mar 28, 2022

xraid commented Mar 28, 2022 • edited Loading

rajarshimaitra commented Mar 29, 2022

rajarshimaitra commented Mar 29, 2022

rajarshimaitra commented Mar 29, 2022

xraid commented Mar 29, 2022 • edited Loading

notmandatory left a comment

Choose a reason for hiding this comment

rajarshimaitra commented Feb 9, 2022 •

edited

Loading

xraid commented Mar 27, 2022 •

edited

Loading

xraid commented Mar 28, 2022 •

edited

Loading

xraid commented Mar 29, 2022 •

edited

Loading