diff --git a/README.md b/README.md index 84bf7f9..17bc1a4 100644 --- a/README.md +++ b/README.md @@ -66,7 +66,7 @@ This will: - Start storage provider(s) - Launch [Portainer UI](https://docs.docksal.io/use-cases/portainer/) for container management -**If you are have troubles**: Use `cargo run -- start`, removing parallelism during start, this may take longer. +**If you have troubles**: Use `cargo run -- start`, removing parallelism during start, this may take longer. **That's it!** Your local Filecoin network is running. @@ -109,7 +109,7 @@ From building Docker images to deploying contractsβ€”everything is automated: Built with modular steps for easy extension and customization: - Add custom deployment steps - Configure multiple PDP service providers -- Control "allowed" SP nodes via `~/.foc-devnet/config.toml` +- Control "allowed" SP nodes via `~/.foc-devnet/config.toml` (see [Configuration System](README_ADVANCED.md#configuration-system)) ### πŸ“œ Programmable Built for scripting and automation: @@ -139,12 +139,13 @@ Bundled with Portainer for browser-based Docker managementβ€”no terminal wizardr | **Docker** | Desktop (macOS) or CE (Linux) | | **tar** | Archive utility (usually pre-installed) | | **Disk Space** | ~20GB for images and blockchain data | +| **x86 Architecture** | Some dependencies require x86 (Intel) architecture. As a result, ARM-based architectures (including Apple Silicon) may not be supported. See [GitHub Issues](https://github.com/FilOzone/foc-devnet/issues) for current architecture compatibility status. | --- ## πŸ› οΈ Need More? -See **[ADVANCED_README.md](ADVANCED_README.md)** for comprehensive documentation on: +See **[README_ADVANCED.md](README_ADVANCED.md)** for comprehensive documentation on: - **All commands reference** (init, build, start, stop, status, version) - **Configuration system** (config.toml structure, parameters, editing) - **Complete directory structure** (what's stored where and why) @@ -156,7 +157,7 @@ See **[ADVANCED_README.md](ADVANCED_README.md)** for comprehensive documentation - **Lifecycle overview** (full startup sequence, step implementation) - **Service Provider examples** (1 SP with 0 authorized, 3 SPs with top 2 authorized, etc.) - **Troubleshooting guides** (port conflicts, build failures, network issues) -- **Advanced topics** (custom genesis, Lotus API access, contract interaction) +- **Additional user actions** (custom genesis, Lotus API access, contract interaction) --- diff --git a/ADVANCED_README.md b/README_ADVANCED.md similarity index 74% rename from ADVANCED_README.md rename to README_ADVANCED.md index 5d6462d..d6cd72c 100644 --- a/ADVANCED_README.md +++ b/README_ADVANCED.md @@ -21,8 +21,8 @@ foc-devnet init [OPTIONS] - `--yugabyte-url ` - Yugabyte download URL - `--yugabyte-archive ` - Local Yugabyte archive file - `--proof-params-dir ` - Local proof params directory -- `--force` - Force regeneration of config file -- `--rand` - Use random mnemonic instead of deterministic one +- `--force` - Force regeneration of config file. Useful when switching between configurations. +- `--rand` - Use random mnemonic instead of deterministic one. Use this for unique test scenarios. **Source Format:** - `gittag:v1.0.0` - Specific git tag (uses default repo) @@ -42,11 +42,16 @@ foc-devnet init \ ### `build` Builds Filecoin components in Docker containers. +> **Note:** This command must be run after `init` to ensure Docker images and environment are prepared. + ```bash foc-devnet build lotus [PATH] [--output-dir ] foc-devnet build curio [PATH] [--output-dir ] ``` +**Options:** +- `--output-dir ` - Directory for built binaries (default: `~/.foc-devnet/bin`) + **Example:** ```bash foc-devnet build lotus @@ -56,15 +61,31 @@ foc-devnet build curio /path/to/custom/curio --output-dir ~/bins ### `start` Starts the local Filecoin network cluster. +> **Note:** This command should be run after `build` of `lotus` and `curio` to ensure binaries are available. + ```bash foc-devnet start [OPTIONS] ``` **Options:** -- `--volumes-dir ` - Custom docker volumes directory -- `--run-dir ` - Custom run-specific data directory +- `--volumes-dir ` - Custom docker volumes directory. Use custom paths (e.g., faster SSD, network storage). +- `--run-dir ` - Custom run-specific data directory. Use custom paths (e.g., faster SSD, network storage). - `--parallel` - **⚑ Run steps in parallel for ~40% faster startup (recommended)** -- `--notest` - Skip end-to-end tests + +**Why `--parallel` (Recommended):** +- **⚑ Significant speedup:** Reduces startup time from ~5 min to ~3 min +- **Smart parallelization:** Steps that don't depend on each other run concurrently +- **Production-ready:** Thread-safe implementation with proper synchronization +- **Use case:** Default for most workflows, especially development iteration + +**When NOT to use `--parallel`:** +- Debugging step ordering issues +- Very low-resource machines (< 4GB RAM) +- First-time setup (sequential is easier to follow) + +See [Detailed Start Sequence](#detailed-start-sequence) for information about which steps are parallelized. + +- `--notest` - Skip end-to-end tests. Use when rapid iteration is needed. **Recommended for faster startup:** ```bash @@ -76,8 +97,6 @@ foc-devnet start --parallel foc-devnet start --parallel --notest ``` -> **πŸ’‘ Pro Tip:** Use `--parallel` by default! It runs independent steps concurrently (contract deployments, database startup, etc.) while respecting dependencies. This can reduce startup time from ~5 minutes to ~3 minutes. - **After successful start:** - Portainer UI available at http://localhost:5700 (uses first port in configured range) - Use Portainer to monitor containers, view logs, and debug issues @@ -182,6 +201,15 @@ tag = "synapse-sdk-v0.36.1" **Constraints:** - `approved_pdp_sp_count` ≀ `active_pdp_sp_count` ≀ `MAX_PDP_SP_COUNT` (5) +### How Defaults Work + +Defaults are defined in code (see [`src/config.rs`](src/config.rs) `Config::default()`) and written to `config.toml` during `init`. This means: + +- **First-time setup:** Running `foc-devnet init` creates `config.toml` with current defaults from code +- **Updating defaults:** When a new version of `foc-devnet` includes updated defaults (e.g., newer Lotus version), run `foc-devnet init --force` to regenerate `config.toml` with the new defaults +- **Preserving customizations:** After regenerating, you'll need to reapply any custom settings you had modified +- **Source of truth:** The code defines what defaults are available; `config.toml` stores your specific configuration + ### Editing Config ```bash @@ -223,7 +251,7 @@ foc-devnet init --force β”‚ └── genesis/ # Genesis block keys β”œβ”€β”€ logs/ # Container logs β”œβ”€β”€ run/ # Run-specific execution data -β”‚ └── / # e.g., 26jan02-1430_ZanyPip/ +β”‚ └── / # e.g., 20260102T1430_ZanyPip/ β”‚ β”œβ”€β”€ setup.log # Startup execution log β”‚ β”œβ”€β”€ version.txt # Component versions β”‚ β”œβ”€β”€ contract_addresses.json # Deployed contracts @@ -303,8 +331,8 @@ foc-devnet start # Creates new run, preserves old ones foc-devnet stop # Delete specific run by run ID -rm -rf ~/.foc-devnet/run/26jan01-1200_OldRun -rm -rf ~/.foc-devnet/docker/volumes/run-specific/26jan01-1200_OldRun +rm -rf ~/.foc-devnet/run/20260101T1200_OldRun +rm -rf ~/.foc-devnet/docker/volumes/run-specific/20260101T1200_OldRun ``` **Delete all old runs (keep only current):** @@ -329,20 +357,6 @@ ls | grep --invert-match "$CURRENT_RUN" | xargs rm -rf rm -rf ~/.foc-devnet ``` -### Manual Cleanup - -```bash -# Stop cluster -foc-devnet stop - -# Delete specific run -rm -rf ~/.foc-devnet/run/26jan02-1430_ZanyPip -rm -rf ~/.foc-devnet/docker/volumes/run-specific/26jan02-1430_ZanyPip - -# Complete nuclear reset (delete everything) -rm -rf ~/.foc-devnet -``` - --- ## Run ID and Step Context @@ -351,9 +365,9 @@ rm -rf ~/.foc-devnet **What:** A unique identifier for each cluster execution. -**Format:** `YYmmmDD-HHMM_RandomName` +**Format:** `YYYYMMDDTHHMM_RandomName` -**Example:** `26jan02-1430_ZanyPip` +**Example:** `20260102T1430_ZanyPip` **Why needed:** - **Isolation:** Separate concurrent runs without conflicts @@ -363,10 +377,10 @@ rm -rf ~/.foc-devnet **Generation:** ```rust -// Date: YYmmmDD (26jan02 = January 2, 2026) -// Time: HHMM (1430 = 2:30 PM) +// Date: YYYYMMDD (20260102 = January 2, 2026, condensed ISO8601) +// Time: HHMM (1430 = 2:30 PM, no colons for Docker compatibility) // Name: RandomAdjective + RandomNoun (ZanyPip) -"26jan02-1430_ZanyPip" +"20260102T1430_ZanyPip" ``` **Storage:** @@ -375,9 +389,12 @@ rm -rf ~/.foc-devnet ### Step Context (SetupContext) -**What:** Thread-safe shared state container that passes data between steps. +**What is a "step"?** A step is a discrete unit of work in the cluster startup process (e.g., starting Lotus daemon, deploying contracts, starting Curio SPs). See [Detailed Start Sequence](#detailed-start-sequence) for a complete list of all steps. -**Why needed:** +**What is "step context"?** Thread-safe shared state container that passes data between steps. + + +**Why is "step context" needed?** - **Dependency resolution:** Later steps need data from earlier steps - **Decoupling:** Steps don't directly call each other - **Parallelization:** Thread-safe for concurrent step execution @@ -386,42 +403,59 @@ rm -rf ~/.foc-devnet **Architecture:** ```rust pub struct SetupContext { - state: Arc>>, // Shared state - run_id: String, // Current run ID - run_dir: PathBuf, // Run directory - port_allocator: Arc>, // Port manager + // Thread-safe key-value store for sharing data between steps + // Keys are string literals (see src/commands/start/ for examples) + state: Arc>>, + // Unique identifier for this cluster run (format: YYYYMMDDTHHMM_RandomName) + run_id: String, + // Directory where all run-specific data is stored (~/.foc-devnet/run//) + run_dir: PathBuf, + // Manages dynamic port allocation to avoid conflicts with other services + port_allocator: Arc>, } ``` **Example flow:** ```rust -// Step 1: ETHAccFundingStep creates deployer address +// Step 1: ETHAccFundingStep creates deployer address and stores it for later steps fn execute(&self, context: &SetupContext) -> Result<(), Box> { + // Generate a new Ethereum address for contract deployment let address = create_eth_address()?; + // Store the address in context so other steps can retrieve it + // Context keys are string literals defined in each step's implementation context.set("deployer_mockusdfc_eth_address", &address); Ok(()) } -// Step 2: USDFCDeployStep uses that address +// Step 2: USDFCDeployStep retrieves the address from context and uses it fn execute(&self, context: &SetupContext) -> Result<(), Box> { + // Retrieve the deployer address that was set by ETHAccFundingStep + // Returns error if key not found (dependency not met) let deployer = context .get("deployer_mockusdfc_eth_address") .ok_or("Deployer not found")?; + // Use the deployer address to deploy the MockUSDFC contract let contract = deploy_mockusdfc(&deployer)?; + // Store the contract address for subsequent steps that need it context.set("mockusdfc_contract_address", &contract); Ok(()) } -// Step 3: USDFCFundingStep uses the contract address +// Step 3: USDFCFundingStep retrieves the contract address and uses it fn execute(&self, context: &SetupContext) -> Result<(), Box> { + // Retrieve the contract address set by USDFCDeployStep let contract = context.get("mockusdfc_contract_address")?; + // Use the contract address to fund test accounts with tokens fund_accounts(&contract)?; Ok(()) } ``` **Common context keys:** + +These keys are string literals used throughout the codebase (see step implementations in [`src/commands/start/`](src/commands/start/)). The definitive list is maintained in the code where steps use `context.get()` and `context.set()`. For an example, see [`src/commands/start/usdfc_deploy/prerequisites.rs`](src/commands/start/usdfc_deploy/prerequisites.rs). + - `deployer_mockusdfc_eth_address` - MockUSDFC deployer address - `deployer_foc_eth_address` - FOC contracts deployer address - `mockusdfc_contract_address` - MockUSDFC token contract @@ -448,7 +482,7 @@ fn execute(&self, context: &SetupContext) -> Result<(), Box> { **What is Portainer?** -Portainer is a lightweight container management UI that gives you visual, browser-based access to all your Docker containers, networks, and volumes. foc-devnet automatically starts Portainer using the first port in your configured range. +[Portainer](https://docs.docksal.io/use-cases/portainer/) is a lightweight container management UI that gives you visual, browser-based access to all your Docker containers, networks, and volumes. foc-devnet automatically starts Portainer using the first port in your configured range. **Access:** http://localhost:5700 (default, or first port from `port_range_start` in config.toml) @@ -517,14 +551,14 @@ Portainer is a lightweight container management UI that gives you visual, browse |-----------|-------|---------|-------| | `foc--lotus` | foc-lotus | Filecoin daemon (FEVM enabled) | 1234 (API), 1235 (P2P) | | `foc--lotus-miner` | foc-lotus-miner | First-gen miner (PoRep) | 2345 (API) | -| `foc--yugabyte` | foc-yugabyte | Database for Curio | 5433 (PostgreSQL) | -| `foc--curio-1` | foc-curio | First Curio SP (PDP) | Dynamic | -| `foc--curio-2` | foc-curio | Second Curio SP (PDP) | Dynamic | -| `foc--curio-N` | foc-curio | Nth Curio SP (PDP) | Dynamic | +| `foc--yugabyte-1` | foc-yugabyte | Database for Curio SP 1 | 5433 (PostgreSQL) | +| `foc--yugabyte-N` | foc-yugabyte | Database for Curio SP N (one per SP) | Dynamic from range | +| `foc--curio-1` | foc-curio | First Curio SP (PDP) | Dynamic from range | +| `foc--curio-N` | foc-curio | Nth Curio SP (PDP) | Dynamic from range | | `foc-builder` | foc-builder | Foundry tools (contract deployment) | Host network | | `foc-portainer` | portainer/portainer-ce | Container management UI | 5700 (first from range) | -**Note:** Container names include run-id for isolation (e.g., `foc-26jan02-1430_ZanyPip-lotus`). +**Note:** Container names include run-id for isolation (e.g., `foc-20260102T1430_ZanyPip-lotus`). ### Network Topology @@ -547,46 +581,42 @@ Docker's user-defined bridge networks are virtual networks that provide: ```mermaid graph TB subgraph host["Host Machine (localhost)"] - style host fill:#f0f0f0,stroke:#333,stroke-width:2px portainer["🌐 Portainer
:5700"] lotus_api["πŸ“‘ Lotus API
:5701"] miner_api["⛏️ Miner API
:5702"] yugabyte_api["πŸ—„οΈ Yugabyte
:5710"] end - subgraph lotus_net["foc-<run-id>-lot-net
(Lotus Network - Blockchain Communication)"] - style lotus_net fill:#e3f2fd,stroke:#1976d2,stroke-width:2px + subgraph lotus_net["foc-<run-id>-lot-net (Lotus Network)"] lotus["foc-lotus
(Filecoin Daemon)"] builder["foc-builder
(--net=host)"] - curio1_lot["foc-curio-1
(on lot-net)"] + curio_n_lot["foc-curio-n
(on lot-net)"] end - subgraph miner_net["foc-<run-id>-lot-m-net
(Lotus Miner Network)"] - style miner_net fill:#fff3e0,stroke:#f57c00,stroke-width:2px + subgraph miner_net["foc-<run-id>-lot-m-net (Lotus Miner Network)"] miner["foc-lotus-miner
(PoRep Miner)"] end - subgraph curio_net["foc-<run-id>-cur-m-net-1
(Curio SP 1 Network)"] - style curio_net fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px - yugabyte["foc-yugabyte
(Database)"] - curio1["foc-curio-1
(PDP Service Provider)"] + subgraph curio_net_n["foc-<run-id>-cur-m-net-n (Curio SP N Network)"] + yugabyte_n["foc-yugabyte-n
(Database)"] + curio_n["foc-curio-n
(PDP Service Provider)"] end %% Container to Host connections lotus -.->|exposes| lotus_api miner -.->|exposes| miner_api - yugabyte -.->|exposes| yugabyte_api + yugabyte_n -.->|exposes| yugabyte_api %% Network connections builder -->|uses host network| lotus - curio1 -->|same container| curio1_lot + curio_n -->|same container| curio_n_lot miner -->|connects to| lotus - curio1_lot -->|connects to| lotus - yugabyte <-->|database| curio1 + curio_n_lot -->|connects to| lotus + yugabyte_n <-->|database| curio_n %% Styling classDef container fill:#fff,stroke:#333,stroke-width:1px - class lotus,builder,curio1_lot,miner,yugabyte,curio1 container + class lotus,builder,curio_n_lot,miner,yugabyte_n,curio_n container ``` **Legend:** @@ -607,8 +637,8 @@ graph TB 3. **Curio Networks (`foc--cur-m-net-N`)**: - Each Curio SP gets its own network - - All share Yugabyte database via network membership - - Provides DNS: Curio can use `foc--yugabyte` as database host + - Each Curio SP has its own Yugabyte database instance on its network + - Provides DNS: Curio SP N can use `foc--yugabyte-N` as database host **Builder uses host network** (`--network host`) to access Lotus RPC at `http://localhost:1234/rpc/v1`. @@ -643,15 +673,25 @@ port_range_count = 100 ## Repository Management + + ### Required Repositories -| Repository | Default Source | Purpose | -|------------|---------------|---------| -| **lotus** | `github.com/filecoin-project/lotus:v1.34.0` | Filecoin daemon | -| **curio** | `github.com/filecoin-project/curio:pdpv0` | Storage provider (PDP) | -| **filecoin-services** | `github.com/FilOzone/filecoin-services:v1.0.0` | FOC smart contracts | -| **multicall3** | `github.com/mds1/multicall3:main` | Multicall3 contract | -| **synapse-sdk** | `github.com/FilOzone/synapse-sdk:synapse-sdk-v0.36.1` | PDP verification SDK | +- **[lotus](https://github.com/filecoin-project/lotus)** - Filecoin daemon +- **[curio](https://github.com/filecoin-project/curio)** - Storage provider (PDP) +- **[filecoin-services](https://github.com/FilOzone/filecoin-services)** - FOC smart contracts +- **[multicall3](https://github.com/mds1/multicall3)** - Multicall3 contract +- **[synapse-sdk](https://github.com/FilOzone/synapse-sdk)** - PDP verification SDK + +### Version Strategy + +Default versions for these repositories are defined in code (see [`src/config.rs`](src/config.rs) `Config::default()`). + +- **Git tags** (`GitTag`): Used for stable releases. Tags provide version pinning and stability. +- **Git commits** (`GitCommit`): Used for repositories where specific commits are required and there isn't a corresponding tag yet. (Generally tags should be preferred over commits.) +- **Git branches** (`GitBranch`): Used for development or when tracking latest changes. + +**Updating defaults:** See [How Defaults Work](#how-defaults-work) for information on how defaults are defined and the steps to apply updates. ### Using Local Repositories @@ -679,45 +719,19 @@ foc-devnet init \ **To share your exact setup with others:** -1. **Export config:** - ```bash - cat ~/.foc-devnet/config.toml - ``` - -2. **Document versions:** - ```toml - # Lotus v1.34.0 - [lotus] - url = "https://github.com/filecoin-project/lotus.git" - tag = "v1.34.0" - - # Curio pdpv0 branch (commit: abc123) - [curio] - url = "https://github.com/filecoin-project/curio.git" - branch = "pdpv0" - - # FilOzone services v1.0.0 - [filecoin_services] - url = "https://github.com/FilOzone/filecoin-services.git" - tag = "v1.0.0" - - # Synapse SDK - [synapse_sdk] - url = "git@github.com:FilOzone/synapse-sdk.git" - tag = "synapse-sdk-v0.36.1" - ``` - -3. **Share config file:** - ```bash - # Recipient copies config - mkdir -p ~/.foc-devnet - cp shared-config.toml ~/.foc-devnet/config.toml - - # Run init to download and build - foc-devnet init - ``` +```bash +# Copy config file +cp shared-config.toml ~/.foc-devnet/config.toml -**For reproducible builds, specify exact commits:** +# Run init to download and build +foc-devnet init +``` + +--- + +### Reproducible Builds + +For reproducible builds, specify exact commits in `config.toml`: ```toml [lotus] @@ -731,55 +745,40 @@ commit = "789012345678..." --- -## Command Flags - -### `init` Flags - -| Flag | Type | Description | -|------|------|-------------| -| `--curio` | String | Curio source location | -| `--lotus` | String | Lotus source location | -| `--filecoin-services` | String | Filecoin Services source location | -| `--synapse-sdk` | String | Synapse SDK source location | -| `--yugabyte-url` | String | Yugabyte download URL | -| `--yugabyte-archive` | Path | Local Yugabyte archive (.tar.gz) | -| `--proof-params-dir` | Path | Local proof parameters directory | -| `--force` | Boolean | Force config regeneration | -| `--rand` | Boolean | Use random mnemonic (non-deterministic keys) | +## Lifecycle Overview -**Why `--force`:** Regenerates `config.toml` even if it exists. Useful when switching between configurations. +The typical workflow follows: `init` β†’ `build` β†’ `start` β†’ `[running]` β†’ `stop` β†’ `start` (regenesis). Each `start` command performs a regenesis, creating a fresh blockchain state. -**Why `--rand`:** Generates random keys instead of deterministic ones. Use for unique test scenarios. +### Detailed Start Sequence -### `build` Flags +The `start` command orchestrates multiple phases to launch the cluster. See the [Step Implementation Pattern](#step-implementation-pattern) section for how steps are structured. -| Flag | Type | Description | -|------|------|-------------| -| `--output-dir` | Path | Directory for built binaries (default: `~/.foc-devnet/bin`) | +#### Before Steps -**Why `--output-dir`:** Specify custom location for compiled binaries. +**Pre-start cleanup:** + - Stop any existing cluster + - Generate unique run ID (format: `YYYYMMDDTHHMM_RandomName`) + - Create run directories + - Perform regenesis (delete old run volumes) -### `start` Flags +**Genesis prerequisites (one-time per start):** + - Generate BLS keys for prefunded accounts + - Create pre-sealed sectors + - Build genesis block configuration -| Flag | Type | Description | -|------|------|-------------| -| `--volumes-dir` | Path | Custom docker volumes directory | -| `--run-dir` | Path | Custom run-specific data directory | -| `--parallel` | Boolean | ⚑ **Execute steps in parallel (~40% faster, recommended)** | -| `--notest` | Boolean | Skip end-to-end Synapse tests | +**Port allocation:** + - Validate port range availability (see [Configuration System](#configuration-system) for port range settings) + - Allocate Portainer port + - Initialize port allocator for dynamic assignment -**Why `--parallel` (Recommended):** -- **⚑ Significant speedup:** Reduces startup time from ~10 min to ~6 min -- **Smart parallelization:** Steps that don't depend on each other run concurrently -- **Production-ready:** Thread-safe implementation with proper synchronization -- **Use case:** Default for most workflows, especially development iteration +**Network creation:** + - Create Lotus network + - Create Lotus Miner network + - Create Curio networks (one per SP) -**When NOT to use `--parallel`:** -- Debugging step ordering issues -- Very low-resource machines (< 4GB RAM) -- First-time setup (sequential is easier to follow) +#### Steps -**Parallel execution epochs (with `--parallel`):** +Steps run sequentially by default, or in parallel when using the `--parallel` flag. With `--parallel`, steps are grouped into execution epochs based on dependencies: | Epoch | Steps | Parallelized? | Why | |-------|-------|---------------|-----| @@ -797,146 +796,92 @@ commit = "789012345678..." **Without `--parallel`:** All 8 epochs run sequentially (~5 minutes total). **With `--parallel`:** Epochs 4-5 run concurrently (~3 minutes total). -**Why `--notest`:** Skip time-consuming E2E tests when rapid iteration needed. - -**Why `--volumes-dir` / `--run-dir`:** Use custom paths (e.g., faster SSD, network storage). - ---- - -## Lifecycle Overview - -### Full Lifecycle - -``` -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ init β”‚ Download repos, build images, generate keys -β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ - β”‚ - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ build β”‚ Compile lotus and curio binaries -β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ - β”‚ - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ start β”‚ Launch cluster (see detailed flow below) -β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ - β”‚ - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ [running]β”‚ Cluster active, contracts deployed -β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ - β”‚ - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ stop β”‚ Stop containers, cleanup networks -β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ - β”‚ - β–Ό -β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” -β”‚ start β”‚ Regenesis + restart (fresh blockchain) -β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ -``` - -### Detailed Start Sequence - -**1. Pre-start cleanup:** - - Stop any existing cluster - - Generate unique run ID - - Create run directories - - Perform regenesis (delete old run volumes) - -**2. Genesis prerequisites (one-time per start):** - - Generate BLS keys for prefunded accounts - - Create pre-sealed sectors - - Build genesis block configuration - -**3. Port allocation:** - - Validate port range availability - - Allocate Portainer port - - Initialize port allocator for dynamic assignment - -**4. Network creation:** - - Create Lotus network - - Create Lotus Miner network - - Create Curio networks (one per SP) - -**5. Step execution (sequential or parallel):** - - **a. Lotus Step:** +**Lotus Step:** - Start Lotus daemon container - Wait for API file - Verify RPC connectivity - **b. Lotus Miner Step:** +**Lotus Miner Step:** - Import pre-sealed sectors - Initialize miner - Start mining - **c. ETH Account Funding Step:** +**ETH Account Funding Step:** - Transfer FIL to create FEVM addresses - Fund deployer accounts - Wait for address activation - **d. MockUSDFC Deploy Step:** +**MockUSDFC Deploy Step:** - Deploy ERC-20 test token - Save contract address - **e. USDFC Funding Step:** +**USDFC Funding Step:** - Transfer tokens to test accounts - Fund Curio SPs - **f. Multicall3 Deploy Step:** +**Multicall3 Deploy Step:** - Deploy Multicall3 contract - Save contract address - **g. FOC Deploy Step:** +**FOC Deploy Step:** - Deploy FOC service contracts - Deploy PDPVerifier, ServiceProviderRegistry, etc. - Save all contract addresses - **h. Yugabyte Step:** - - Start Yugabyte database +**Yugabyte Step:** + - Start Yugabyte database (one instance per Curio SP, on the SP's network) - Verify PostgreSQL port - **i. Curio Step:** +**Curio Step:** - Initialize Curio database schemas - Start N Curio SP containers - Configure PDP endpoints - **j. PDP SP Registration Step:** +**PDP SP Registration Step:** - Register each Curio SP in registry - Approve authorized SPs - Save provider IDs - **k. Synapse E2E Test Step:** +**Synapse E2E Test Step:** - Run end-to-end verification - - Test deal flow (unless `--notest`) + - Test deal flow (unless `--notest` flag is used) -**6. Post-start:** +#### Post Start Steps - Save step context - Display summary - Print access URLs +#### running +At this point the cluster is active and already has contracts deployed. It is ready for further interaction. + +#### stop +- Stops containers +- Cleans up networks + +#### (re)start +- Regenesis by following the [start steps](#steps) and creating a new blockchain. + ### Step Implementation Pattern Every step follows this trait: ```rust pub trait Step: Send + Sync { + // Human-readable name for logging and display (e.g., "Lotus", "FOC Deploy") fn name(&self) -> &str; + // Validation phase: check prerequisites before executing + // (e.g., verify Docker images exist, check port availability) fn pre_execute(&self, context: &SetupContext) -> Result<(), Box>; + // Main work: perform the actual step (e.g., start container, deploy contract) fn execute(&self, context: &SetupContext) -> Result<(), Box>; + // Verification phase: confirm step succeeded (e.g., check API is responding, confirm deployment) fn post_execute(&self, context: &SetupContext) -> Result<(), Box>; + // Orchestrates the full step lifecycle: pre_execute β†’ execute β†’ post_execute + // Returns the duration taken for the step fn run(&self, context: &SetupContext) -> Result>; } ``` -**Phases:** -1. **Pre-execute:** Validation (check images, ports, prerequisites) -2. **Execute:** Main work (start container, deploy contract, etc.) -3. **Post-execute:** Verification (check API, confirm deployment) - --- ## Service Provider Examples @@ -1030,14 +975,6 @@ docker exec foc--builder cast call \ # Returns: false ``` -**Testing scenarios:** -```bash -# Send deal to approved SP (should succeed) -# Send deal to unapproved SP (should fail) -# Test failover from SP1 to SP2 -# Test SP3 attempting to accept deal (should reject) -``` - ### Example 3: Maximum SPs (5) **Scenario:** Stress testing, load balancing. @@ -1090,8 +1027,8 @@ docker logs foc--curio-2 # Query provider IDs cat ~/.foc-devnet/state/latest/pdp_sps/*.provider_id.json -# Access Yugabyte (shared by all SPs) -docker exec -it foc--yugabyte ysqlsh -h localhost -p 5433 +# Access Yugabyte (one per SP) +docker exec -it foc--yugabyte-1 ysqlsh -h localhost -p 5433 # Query Lotus for miner info docker exec foc--lotus lotus state miner-info f01000 @@ -1157,10 +1094,23 @@ foc-devnet start --- -## Advanced Topics +## Additional User Actions ### Custom Genesis Block -Edit genesis templates before `start`: + +The genesis template is generated during the genesis prerequisites phase of the `start` command and is located at: + +``` +~/.foc-devnet/docker/volumes/run-specific//genesis/ +``` + +To customize the genesis block, you can: +- Modify the genesis generation code in `src/commands/start/lotus/` +- Edit the template files after they're generated (before the Lotus step completes) +- Pass custom genesis parameters through the configuration system + +**Note:** Customizing genesis requires understanding the Filecoin genesis format. See the [Detailed Start Sequence](#detailed-start-sequence) for when genesis prerequisites run. + ```bash # Modify sector size, block time, etc. # (Advanced - requires understanding Filecoin genesis format) @@ -1206,16 +1156,3 @@ docker run --rm --network host \ --broadcast ``` ---- - -## Reference Links - -- **Lotus Documentation:** https://lotus.filecoin.io/ -- **Curio Documentation:** https://github.com/filecoin-project/curio -- **FEVM Documentation:** https://docs.filecoin.io/smart-contracts/ -- **Foundry Book:** https://book.getfoundry.sh/ -- **Docker Documentation:** https://docs.docker.com/ - ---- - -**Last Updated:** January 2026 diff --git a/src/run_id/mod.rs b/src/run_id/mod.rs index 6109ffe..0648ca9 100644 --- a/src/run_id/mod.rs +++ b/src/run_id/mod.rs @@ -1,7 +1,7 @@ //! Run ID generation and management. //! //! This module handles generating unique run IDs for each cluster start. -//! Format: YYMMDD-HHMM-random-name (e.g., 251203-1246-thirsty-wolf) +//! Format: YYYYMMDDTHHMM-random-name (e.g., 20251203T1246-thirsty-wolf) mod persistence; @@ -28,21 +28,22 @@ pub const NOUNS: &[&str] = &[ /// Generate a unique run ID for this execution. /// -/// Returns a string like "foc_25dec15-2206_blue_ibis_lotus" where: -/// - foc_ is the prefix -/// - 25dec15 is the date (YYmmmDD where mmm is the lowercase abbreviated month name) -/// - 2206 is the time (HHMM) -/// - blue_ibis_lotus is the modified random name with underscores and suffix +/// Returns a string like "20251215T2206_ZanyPip" where: +/// - 20251215 is the date (YYYYMMDD, condensed ISO8601 format) +/// - T is the date/time separator (ISO8601) +/// - 2206 is the time (HHMM, 24-hour format, no colons for Docker compatibility) +/// - ZanyPip is the random name (adjective + noun) +/// +/// Uses condensed ISO8601 format (no dashes or colons) for Docker network name compatibility. /// /// # Example /// ```no_run /// let run_id = generate_run_id(); -/// println!("{}", run_id); // e.g., "25dec15-2206_blue_ibis_lotus" +/// println!("{}", run_id); // e.g., "20251215T2206_ZanyPip" /// ``` pub fn generate_run_id() -> String { let now = Local::now(); - let date = now.format("%y%b%d").to_string().to_lowercase(); - let time = now.format("%H%M"); + let datetime = now.format("%Y%m%dT%H%M"); // Implement our own random name generator to control format let random_name = { @@ -52,7 +53,7 @@ pub fn generate_run_id() -> String { format!("{}{}", adjective, noun) }; - format!("{}-{}_{}", date, time, random_name) + format!("{}_{}", datetime, random_name) } /// Create a symlink to the latest run directory. @@ -90,8 +91,8 @@ mod tests { fn test_run_id_format() { let run_id = generate_run_id(); - // Should match pattern: foc_YYmmmDD-HHMM_word_word_lotus - let pattern = Regex::new(r"^foc_\d{2}[a-z]{3}\d{2}-\d{4}_.+$").unwrap(); + // Should match pattern: YYYYMMDDTHHMM_RandomName (condensed ISO8601, no dashes/colons) + let pattern = Regex::new(r"^\d{8}T\d{4}_.+$").unwrap(); assert!( pattern.is_match(&run_id), "Run ID should match format: {}",