Skip to content

Commit 2c15475

Browse files
benthecarmanclaude
andcommitted
Fix bitcoind shutdown hang during initial sync errors and polling
When using bitcoind chain source, `Node::stop()` could hang indefinitely if called during: 1. Initial sync error backoff (up to 5 minutes of sleep) 2. Active polling operations in the continuous sync loop This is a more severe variant of the background sync hang issue, as the bitcoind initial sync loop had NO stop signal checking at all. The `BitcoindChainSource::continuously_sync_wallets()` initial sync loop (lines 149-277) performs synchronization on startup. When sync fails (e.g., "Connection refused" when bitcoind is unavailable), it enters an exponential backoff retry loop: - Transient errors: Sleep for `backoff` seconds (10s, 20s, 40s, 80s, 160s, up to 300s) - Persistent errors: Sleep for MAX_BACKOFF_SECS (300 seconds) These sleeps had NO stop signal checking. When shutdown was requested: 1. Initial sync fails with connection error 2. Code sleeps for backoff period (up to 5 minutes) 3. User calls `stop()` 4. Stop signal sent but ignored - stuck sleeping 5. Shutdown times out after 5 seconds and aborts 6. Initial sync loop never exits cleanly The continuous polling loop (lines 296-349) had the same issue as electrum/esplora - no biased select and no cancellation of in-progress operations. 1. Added biased `tokio::select!` at loop start to check stop signal before beginning each sync attempt 2. Wrapped both error backoff sleeps in biased `tokio::select!` to allow immediate interruption when stop is requested This ensures that even if bitcoind is down and the node is retrying with exponential backoff, shutdown completes immediately instead of waiting up to 5 minutes. Applied the same biased nested `tokio::select!` pattern used for electrum/esplora: - Outer biased select prioritizes stop signal before polling intervals - Inner nested selects allow cancellation of in-progress operations Existing unit tests pass. The shutdown will now complete in milliseconds instead of hanging for minutes when bitcoind is unreachable. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent 1a134b4 commit 2c15475

File tree

1 file changed

+60
-10
lines changed

1 file changed

+60
-10
lines changed

src/chain/bitcoind.rs

Lines changed: 60 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,12 @@ impl BitcoindChainSource {
147147
const MAX_BACKOFF_SECS: u64 = 300;
148148

149149
loop {
150+
// if the stop_sync_sender has been dropped, we should just exit
151+
if stop_sync_receiver.has_changed().unwrap_or(true) {
152+
log_trace!(self.logger, "Stopping initial chain sync.");
153+
return;
154+
}
155+
150156
let channel_manager_best_block_hash = channel_manager.current_best_block().block_hash;
151157
let sweeper_best_block_hash = output_sweeper.current_best_block().block_hash;
152158
let onchain_wallet_best_block_hash =
@@ -226,7 +232,18 @@ impl BitcoindChainSource {
226232
e,
227233
backoff
228234
);
229-
tokio::time::sleep(Duration::from_secs(backoff)).await;
235+
// Sleep with stop signal check to allow immediate shutdown
236+
tokio::select! {
237+
biased;
238+
_ = stop_sync_receiver.changed() => {
239+
log_trace!(
240+
self.logger,
241+
"Stopping initial chain sync.",
242+
);
243+
return;
244+
}
245+
_ = tokio::time::sleep(Duration::from_secs(backoff)) => {}
246+
}
230247
backoff = std::cmp::min(backoff * 2, MAX_BACKOFF_SECS);
231248
} else {
232249
log_error!(
@@ -235,7 +252,18 @@ impl BitcoindChainSource {
235252
e,
236253
MAX_BACKOFF_SECS
237254
);
238-
tokio::time::sleep(Duration::from_secs(MAX_BACKOFF_SECS)).await;
255+
// Sleep with stop signal check to allow immediate shutdown
256+
tokio::select! {
257+
biased;
258+
_ = stop_sync_receiver.changed() => {
259+
log_trace!(
260+
self.logger,
261+
"Stopping initial chain sync during backoff.",
262+
);
263+
return;
264+
}
265+
_ = tokio::time::sleep(Duration::from_secs(MAX_BACKOFF_SECS)) => {}
266+
}
239267
}
240268
},
241269
}
@@ -260,6 +288,7 @@ impl BitcoindChainSource {
260288
let mut last_best_block_hash = None;
261289
loop {
262290
tokio::select! {
291+
biased;
263292
_ = stop_sync_receiver.changed() => {
264293
log_trace!(
265294
self.logger,
@@ -268,17 +297,38 @@ impl BitcoindChainSource {
268297
return;
269298
}
270299
_ = chain_polling_interval.tick() => {
271-
let _ = self.poll_and_update_listeners(
272-
Arc::clone(&channel_manager),
273-
Arc::clone(&chain_monitor),
274-
Arc::clone(&output_sweeper)
275-
).await;
300+
tokio::select! {
301+
biased;
302+
_ = stop_sync_receiver.changed() => {
303+
log_trace!(
304+
self.logger,
305+
"Stopping polling for new chain data.",
306+
);
307+
return;
308+
}
309+
_ = self.poll_and_update_listeners(
310+
Arc::clone(&channel_manager),
311+
Arc::clone(&chain_monitor),
312+
Arc::clone(&output_sweeper)
313+
) => {}
314+
}
276315
}
277316
_ = fee_rate_update_interval.tick() => {
278317
if last_best_block_hash != Some(channel_manager.current_best_block().block_hash) {
279-
let update_res = self.update_fee_rate_estimates().await;
280-
if update_res.is_ok() {
281-
last_best_block_hash = Some(channel_manager.current_best_block().block_hash);
318+
tokio::select! {
319+
biased;
320+
_ = stop_sync_receiver.changed() => {
321+
log_trace!(
322+
self.logger,
323+
"Stopping polling for new chain data.",
324+
);
325+
return;
326+
}
327+
update_res = self.update_fee_rate_estimates() => {
328+
if update_res.is_ok() {
329+
last_best_block_hash = Some(channel_manager.current_best_block().block_hash);
330+
}
331+
}
282332
}
283333
}
284334
}

0 commit comments

Comments
 (0)