Merge #26: Re-write in xshell

5533a45 tasks: add package filter so can run on a subset of crates (Nick Johnson)
ae24339 tasks: add lock task (Nick Johnson)
f5b2ffd tasks: add tests task (Nick Johnson)
99ea567 tasks: add docs and bench commands (Nick Johnson)
7e16550 tasks: add a toolchain check (Nick Johnson)
32ee8d6 tasks: add the lint task (Nick Johnson)
fe25b2f Make repo a workspace and add a tasks crate (Nick Johnson)

Pull request description:

  Re-write the ci/run_task.sh script in xshell, attempting to match as much functionality as possible, but some user interfaces are changed for easier maintenance. The largest change is dropping the config scripts, `contrib/crates.sh`, `contrib/whitelist_deps.sh`, and `contrib/test_vars.sh`, for `contrib/rbmt.toml`.

  The first patch makes the repo a workspace and initializes the new crate for the xshell based maintainer tools. The following patches add commands to match existing functionality. They also add a new `lock` command for lock file management as well as a `--package` filter (matching cargo's interface) to run commands on a subset of a workspace's crates.

  Closes #25, #6

ACKs for top commit:
  tcharding:
    ACK 5533a45

Tree-SHA512: 2281ca2aef4011708c3f1ccb5ca556df7d7ee9d66a81301242f12f6c3da18cde24bd0d49dac1e5806664744db162f68f9bb41f2f7694f024423a5771d420c774

Author: tcharding

Cargo.lock

@@ -0,0 +1,6 @@
+[workspace]
+members = [
+    "releases",
+    "tasks",
+]
+resolver = "2"

Cargo.toml

@@ -18,5 +18,5 @@ crates_io_api = { version = "0.11.0", default-features = false, features = ["rus
 semver = { version = "1.0.22", default-features = false, features = ["std"] }
 serde_json = { version = "1.0.116", default-features = false, features = [] }
 serde = { version = "1.0.198", features = ["derive"] }
-tokio = { version = "1.37.0", features = ["rt", "macros"] }
+tokio = { version = "1.37.0", features = ["rt-multi-thread", "macros"] }
 toml = { version = "0.8.12", default-features = false, features = ["display", "parse"] }

releases/Cargo.toml

@@ -0,0 +1,5 @@
+# Changelog
+
+## [0.1.0] -
+
+* Initial release of `rust-bitcoin-maintainer-tools` (executable: `rbmt`) matching functionality of shell scripts.

tasks/CHANGELOG.md

@@ -0,0 +1,18 @@
+[package]
+name = "rust-bitcoin-maintainer-tools"
+version = "0.1.0"
+authors = ["Nick Johnson <[email protected]>"]
+license = "CC0-1.0"
+edition = "2021"
+rust-version = "1.74.0"
+
+[[bin]]
+name = "rbmt"
+path = "src/main.rs"
+
+[dependencies]
+xshell = "0.2"
+clap = { version = "4", features = ["derive"] }
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+toml = "0.8"

tasks/Cargo.toml

@@ -0,0 +1,134 @@
+# Maintainer Tools
+
+Maintainer tools for Rust-based projects in the Bitcoin domain. Built with [xshell](https://github.com/matklad/xshell).
+
+## Configuration
+
+Configuration for `rbmt` is stored in `contrib/rbmt.toml`. The file can live at both the workspace root (e.g. `$ROOT/contrib/rbmt.toml`) as well as per-crate (e.g. `$ROOT/$CRATE/contrib/rbmt.toml`) within a repository.
+
+### Lint
+
+The `lint` command detects duplicate dependencies, but some may be unavoidable (e.g., during dependency updates where transitive dependencies haven't caught up). Configure the `[lint]` section to whitelist specific duplicates for a workspace (or a crate if only one crate in a repository).
+
+```toml
+[lint]
+allowed_duplicates = [
+    "syn",
+    "bitcoin_hashes",
+]
+```
+
+### Test
+
+The `test` command can be configured to run feature matrix testing for your crate. Configure with the `contrib/rbmt.toml` file at the crate level.
+
+```toml
+[test]
+# Examples to run with specific features enabled.
+# Format: "example_name:feature1 feature2"
+examples = [
+    "example1:serde",
+    "example2:serde rand",
+]
+
+# Features to test with the conventional `std` feature enabled.
+# Tests each feature alone with std, all pairs, and all together.
+# Example: ["serde", "rand"] tests: std+serde, std+rand, std+serde+rand
+features_with_std = ["serde", "rand"]
+
+# Features to test without the `std` feature.
+# Tests each feature alone, all pairs, and all together.
+# Example: ["serde", "rand"] tests: serde, rand, serde+rand
+features_without_std = ["serde", "rand"]
+
+# Exact feature combinations to test.
+# Use for crates that don't follow conventional `std` patterns.
+# Each inner array is tested as-is with no automatic combinations.
+# Example: [["serde", "rand"], ["rand"]] tests exactly those two combinations
+exact_features = [
+    ["serde", "rand"],
+    ["rand"],
+]
+
+# Features to test with an explicit `no-std` feature enabled.
+# Only use if your crate has a `no-std` feature (rust-miniscript pattern).
+# Tests each feature with no-std, all pairs, and all together.
+# Example: ["serde", "rand"] tests: no-std+serde, no-std+rand, no-std+serde+rand
+features_with_no_std = ["serde", "rand"]
+```
+
+### Environment Variables
+
+* `RBMT_LOG_LEVEL=quiet` - Suppress verbose output and reduce cargo noise.
+
+## Lock Files
+
+To ensure your crate works with the full range of declared dependency versions, `rbmt` requires two lock files in your repository.
+
+* `Cargo-minimal.lock` - Minimum versions that satisfy your dependency constraints.
+* `Cargo-recent.lock` - Recent/updated versions of dependencies.
+
+The `rbmt lock` command generates and maintains these files for you. You can then use `--lock-file` with any command to test against either version set.
+
+### Usage
+
+**Generate/update lock files**
+
+```bash
+rbmt lock
+```
+
+1. Verify that direct dependency versions aren't being bumped by transitive dependencies.
+2. Generate `Cargo-minimal.lock` with minimal versions across the entire dependency tree.
+3. Update `Cargo-recent.lock` with conservatively updated dependencies.
+
+**Use a specific lock file**
+
+```bash
+# Test with minimal versions.
+rbmt --lock-file minimal test stable
+
+# Test with recent versions.
+rbmt --lock-file recent test stable
+
+# Works with any command.
+rbmt --lock-file minimal lint
+rbmt --lock-file minimal docs
+```
+
+When you specify `--lock-file`, the tool copies that lock file to `Cargo.lock` before running the command. This allows you to test your code against different dependency version constraints.
+
+## Workspace Integration
+
+`rbmt` can simply be installed globally, or as a dev-dependency for more granular control of dependency versions.
+
+### 1. Install globally
+
+Install the tool globally on your system with `cargo install`.
+
+```bash
+cargo install [email protected]
+```
+
+Then run from anywhere in your repository.
+
+```bash
+rbmt lint
+```
+
+### 2. Add as a dev-dependency
+
+Add as a dev-dependency to a workspace member. This pins the tool version in your lockfile for reproducible builds.
+
+```toml
+[dev-dependencies]
+rust-bitcoin-maintainer-tools = "0.1.0"
+```
+
+Then run via cargo.
+
+```bash
+cargo run --bin rbmt -- lint
+```
+
+It might be worth wrapping in an [xtask](https://github.com/matklad/cargo-xtask) package for a clean interface.

tasks/README.md

@@ -0,0 +1,11 @@
+# List available recipes.
+_default:
+    @just --list
+
+# Run tests.
+test:
+    cargo test
+
+# Install rbmt from the local path.
+install:
+    cargo install --path .

tasks/justfile

@@ -0,0 +1,31 @@
+//! Benchmark testing tasks.
+
+use crate::environment::{get_crate_dirs, quiet_println};
+use crate::quiet_cmd;
+use crate::toolchain::{check_toolchain, Toolchain};
+use xshell::Shell;
+
+/// Run benchmark tests for all crates in the workspace.
+pub fn run(sh: &Shell, packages: &[String]) -> Result<(), Box<dyn std::error::Error>> {
+    check_toolchain(sh, Toolchain::Nightly)?;
+
+    let crate_dirs = get_crate_dirs(sh, packages)?;
+
+    quiet_println(&format!(
+        "Running bench tests for {} crates",
+        crate_dirs.len()
+    ));
+
+    for crate_dir in &crate_dirs {
+        quiet_println(&format!("Running bench tests in: {}", crate_dir));
+
+        // Use pushd pattern to change and restore directory.
+        let _dir = sh.push_dir(crate_dir);
+
+        quiet_cmd!(sh, "cargo bench")
+            .env("RUSTFLAGS", "--cfg=bench")
+            .run()?;
+    }
+
+    Ok(())
+}

tasks/src/bench.rs

@@ -0,0 +1,47 @@
+//! Documentation building tasks.
+
+use crate::quiet_cmd;
+use crate::toolchain::{check_toolchain, Toolchain};
+use xshell::Shell;
+
+/// Build documentation for end users with the stable toolchain.
+///
+/// This verifies that `cargo doc` works correctly for users with stable Rust.
+/// Uses basic rustdoc warnings to catch common documentation issues.
+pub fn run(sh: &Shell, packages: &[String]) -> Result<(), Box<dyn std::error::Error>> {
+    check_toolchain(sh, Toolchain::Stable)?;
+
+    let mut cmd = quiet_cmd!(sh, "cargo doc --all-features");
+
+    // Add package filters if specified.
+    for package in packages {
+        cmd = cmd.args(&["-p", package]);
+    }
+
+    cmd.env("RUSTDOCFLAGS", "-D warnings").run()?;
+
+    Ok(())
+}
+
+/// Build documentation for docs.rs with the nightly toolchain.
+///
+/// This emulates the docs.rs build environment by using the nightly toolchain
+/// with `--cfg docsrs` enabled. This catches docs.rs-specific issues.
+pub fn run_docsrs(sh: &Shell, packages: &[String]) -> Result<(), Box<dyn std::error::Error>> {
+    check_toolchain(sh, Toolchain::Nightly)?;
+
+    let mut cmd = quiet_cmd!(sh, "cargo doc --all-features");
+
+    // Add package filters if specified.
+    for package in packages {
+        cmd = cmd.args(&["-p", package]);
+    }
+
+    cmd.env(
+        "RUSTDOCFLAGS",
+        "--cfg docsrs -D warnings -D rustdoc::broken-intra-doc-links",
+    )
+    .run()?;
+
+    Ok(())
+}

tasks/src/docs.rs

@@ -0,0 +1,93 @@
+use std::env;
+use xshell::{cmd, Shell};
+
+/// Environment variable to control output verbosity.
+/// Set to "quiet" to suppress informational messages and reduce cargo output.
+/// Any other value (or unset) defaults to verbose mode.
+const LOG_LEVEL_ENV_VAR: &str = "RBMT_LOG_LEVEL";
+
+/// Path to the RBMT configuration file relative to workspace/crate root.
+pub const CONFIG_FILE_PATH: &str = "contrib/rbmt.toml";
+
+/// Check if we're in quiet mode via environment variable.
+pub fn is_quiet_mode() -> bool {
+    env::var(LOG_LEVEL_ENV_VAR).is_ok_and(|v| v == "quiet")
+}
+
+/// Helper macro to create commands that respect quiet mode.
+#[macro_export]
+macro_rules! quiet_cmd {
+    ($sh:expr, $($arg:tt)*) => {{
+        let mut cmd = xshell::cmd!($sh, $($arg)*);
+        if $crate::environment::is_quiet_mode() {
+            cmd = cmd.quiet();
+        }
+        cmd
+    }};
+}
+
+/// Print a message unless in quiet mode.
+pub fn quiet_println(msg: &str) {
+    if !is_quiet_mode() {
+        println!("{}", msg);
+    }
+}
+
+/// Configure shell log level and output verbosity.
+/// Sets cargo output verbosity based on LOG_LEVEL_ENV_VAR.
+pub fn configure_log_level(sh: &Shell) {
+    if is_quiet_mode() {
+        sh.set_var("CARGO_TERM_VERBOSE", "false");
+        sh.set_var("CARGO_TERM_QUIET", "true");
+    } else {
+        sh.set_var("CARGO_TERM_VERBOSE", "true");
+        sh.set_var("CARGO_TERM_QUIET", "false");
+    }
+}
+
+/// Change to the repository root directory.
+///
+/// # Panics
+///
+/// Panics if not in a git repository or git command fails.
+pub fn change_to_repo_root(sh: &Shell) {
+    let repo_dir = cmd!(sh, "git rev-parse --show-toplevel")
+        .read()
+        .expect("Failed to get repository root, ensure you're in a git repository");
+    sh.change_dir(&repo_dir);
+}
+
+/// Get list of crate directories in the workspace using cargo metadata.
+/// Returns fully qualified paths to support various workspace layouts including nested crates.
+///
+/// # Arguments
+///
+/// * `packages` - Optional filter for specific package names. If empty, returns all packages.
+pub fn get_crate_dirs(sh: &Shell, packages: &[String]) -> Result<Vec<String>, Box<dyn std::error::Error>> {
+    let metadata = cmd!(sh, "cargo metadata --no-deps --format-version 1").read()?;
+    let json: serde_json::Value = serde_json::from_str(&metadata)?;
+
+    let crate_dirs: Vec<String> = json["packages"]
+        .as_array()
+        .ok_or("Missing 'packages' field in cargo metadata")?
+        .iter()
+        .filter_map(|package| {
+            let manifest_path = package["manifest_path"].as_str()?;
+            // Extract directory path from the manifest path,
+            // e.g., "/path/to/repo/releases/Cargo.toml" -> "/path/to/repo/releases".
+            let dir_path = manifest_path.trim_end_matches("/Cargo.toml");
+            
+            // Filter by package name if specified.
+            if !packages.is_empty() {
+                let package_name = package["name"].as_str()?;
+                if !packages.contains(&package_name.to_string()) {
+                    return None;
+                }
+            }
+            
+            Some(dir_path.to_string())
+        })
+        .collect();
+
+    Ok(crate_dirs)
+}