Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,16 @@ build-geth:
build-rbuilder:
cd clients && ./build-rbuilder.sh

.PHONY: build-rollup-boost
build-rollup-boost:
cd clients && ./build-rollup-boost.sh

.PHONY: build-binaries
build-binaries: build-reth build-geth build-rbuilder

.PHONY: build-binaries-flashblocks
build-binaries-flashblocks: build-reth build-geth build-rbuilder build-rollup-boost

.PHONY: build-frontend
build-frontend:
cd report && yarn build
Expand Down
72 changes: 66 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,49 @@ benchmarks:

This configuration runs a `transfer-only` transaction payload against both Geth and Reth clients for 20 blocks.

### Flashblocks Support

The benchmark system supports **flashblocks** - Base's sub-200ms transaction preconfirmation feature via the `rbuilder` client.

**Two Modes:**

1. **Simple mode (standalone)**: Just rbuilder for testing
```yaml
benchmarks:
- variables:
- type: node_type
value: rbuilder
```

2. **Dual-builder mode (production architecture)**: Fallback builder + rbuilder + optional rollup-boost
```bash
./bin/base-bench run \
--config ./configs/examples/flashblocks.yml \
--root-dir ./data-dir \
--reth-bin ./bin/op-reth \
--geth-bin ./bin/geth \
--rbuilder-bin ./bin/op-rbuilder \
--flashblocks-fallback reth \
--rollup-boost-bin ./bin/rollup-boost \
--output-dir ./output
```

**How it works (dual-builder mode):**
- **Fallback builder** (geth/reth): Produces final 2s canonical blocks
- **Rbuilder** (primary): Produces flashblocks every 200ms (10 per 2s block)
- **Rollup-boost** (optional): Coordinates between the two builders

**Configurable flashblock interval:**
```yaml
- type: flashblock_interval
values:
- 100 # 100ms (aggressive)
- 200 # 200ms (Base default)
- 500 # 500ms (conservative)
```

See [configs/examples/flashblocks.yml](configs/examples/flashblocks.yml) for complete examples.

### Test Methodology

Each test executes a standardized workflow:
Expand Down Expand Up @@ -125,21 +168,38 @@ OPTIONS:

# Reth Configuration
--reth-bin value Reth binary path (default: "reth")
--reth-http-port value HTTP port (default: 9545)
--reth-auth-rpc-port value Auth RPC port (default: 9551)
--reth-metrics-port value Metrics port (default: 9080)

# Geth Configuration
--geth-bin value Geth binary path (default: "geth")
--geth-http-port value HTTP port (default: 8545)
--geth-auth-rpc-port value Auth RPC port (default: 8551)
--geth-metrics-port value Metrics port (default: 8080)

# Rbuilder (Flashblocks) Configuration
--rbuilder-bin value Rbuilder binary path (default: "rbuilder")
--flashblocks-fallback value Fallback client for dual-builder mode: geth or reth (default: "reth")
--rollup-boost-bin value Rollup-boost coordinator binary for dual-builder mode

# General Options
--proxy-port value Proxy port (default: 8546)
--help, -h Show help (default: false)
```

**Note**: Without `--flashblocks-fallback`, rbuilder runs in simple standalone mode. Set `--flashblocks-fallback` to enable production dual-builder architecture.

### Client-Specific Arguments

You can customize client behavior using the `client_args` variable in YAML configs:

```yaml
benchmarks:
- variables:
- type: client_args
value:
geth: "--verbosity 4 --txpool.globalslots 20000000"
reth: "--txpool.pending-max-count 200000000 -vvvv"
rbuilder: "--txpool.pending-max-count 200000000"
```

This allows testing different client configurations without code changes.

## 📊 Example Reports

<div align="center">
Expand Down
62 changes: 57 additions & 5 deletions clients/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,21 @@ Builds the op-geth binary from the Ethereum Optimism op-geth repository using ju
Builds the op-rbuilder binary from the op-rbuilder repository using Cargo.

**Default Configuration:**
- Repository: `https://github.com/haardikk21/op-rbuilder`
- Version: `a8bb38693ece585e7fa98d52f51290e7dcececff`
- Repository: `https://github.com/base/op-rbuilder.git`
- Version: `bc7e167a8d11362a78b9c30d59adcd8d2c7f9e84`
- Build tool: `cargo`

### build-rollup-boost.sh
Builds the rollup-boost binary from the Flashbots rollup-boost repository using Cargo.
Required for flashblocks dual-builder mode (production architecture).

**Default Configuration:**
- Repository: `https://github.com/flashbots/rollup-boost.git`
- Version: `08ebd3e75a8f4c7ebc12db13b042dee04e132c05`
- Build tool: `cargo`

**Note:** Rollup-boost is optional. Rbuilder can run in standalone mode without it.

## Usage

### Using Makefile (Recommended)
Expand All @@ -44,6 +55,9 @@ make build-geth

# Build only op-rbuilder
make build-rbuilder

# Build only rollup-boost (for flashblocks dual-builder mode)
make build-rollup-boost
```

### Direct Script Execution
Expand All @@ -58,6 +72,9 @@ cd clients

# Build op-rbuilder with defaults
./build-rbuilder.sh

# Build rollup-boost with defaults (for flashblocks dual-builder mode)
./build-rollup-boost.sh
```

## Version Management
Expand Down Expand Up @@ -107,8 +124,14 @@ RBUILDER_VERSION="main" ./build-rbuilder.sh
- `OUTPUT_DIR`: Directory for built binaries (default: ../bin)

#### For op-rbuilder (build-rbuilder.sh):
- `RBUILDER_REPO`: Git repository URL (default: https://github.com/haardikk21/op-rbuilder)
- `RBUILDER_VERSION`: Git branch, tag, or commit hash (default: a8bb38693ece585e7fa98d52f51290e7dcececff)
- `RBUILDER_REPO`: Git repository URL (default: https://github.com/base/op-rbuilder.git)
- `RBUILDER_VERSION`: Git branch, tag, or commit hash (default: bc7e167a8d11362a78b9c30d59adcd8d2c7f9e84)
- `BUILD_DIR`: Directory for source code (default: ./build)
- `OUTPUT_DIR`: Directory for built binaries (default: ../bin)

#### For rollup-boost (build-rollup-boost.sh):
- `ROLLUP_BOOST_REPO`: Git repository URL (default: https://github.com/flashbots/rollup-boost.git)
- `ROLLUP_BOOST_VERSION`: Git branch, tag, or commit hash (default: 08ebd3e75a8f4c7ebc12db13b042dee04e132c05)
- `BUILD_DIR`: Directory for source code (default: ./build)
- `OUTPUT_DIR`: Directory for built binaries (default: ../bin)

Expand All @@ -126,9 +149,38 @@ RBUILDER_VERSION="main" ./build-rbuilder.sh
- Rust and Cargo installed
- Git

### For rollup-boost:
- Rust and Cargo installed
- Git
- `libssl-dev` and `pkg-config` (on Linux)

## Output

Built binaries will be placed in the `bin/` directory at the project root:
- `bin/reth` - The reth binary
- `bin/geth` - The op-geth binary
- `bin/op-rbuilder` - The op-rbuilder binary
- `bin/op-rbuilder` - The op-rbuilder binary
- `bin/rollup-boost` - The rollup-boost binary (for flashblocks dual-builder mode)

## Flashblocks Dual-Builder Mode

To run benchmarks with the full flashblocks architecture:

1. Build the required binaries:
```bash
make build-rbuilder
make build-rollup-boost
make build-reth # or build-geth for fallback
```

2. Run benchmark with dual-builder flags:
```bash
./bin/base-bench run \
--config ./configs/examples/flashblocks.yml \
--rbuilder-bin ./bin/op-rbuilder \
--flashblocks-fallback reth \
--rollup-boost-bin ./bin/rollup-boost \
--output-dir ./output
```

See [configs/examples/flashblocks.yml](../configs/examples/flashblocks.yml) for complete examples.
2 changes: 1 addition & 1 deletion clients/build-rbuilder.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ if [ -f "versions.env" ]; then
source versions.env
else
# Default values
RBUILDER_REPO="${RBUILDER_REPO:-https://github.com/haardikk21/op-rbuilder}"
RBUILDER_REPO="${RBUILDER_REPO:-https://github.com/base/op-rbuilder.git}"
RBUILDER_VERSION="${RBUILDER_VERSION:-main}"
BUILD_DIR="${BUILD_DIR:-./build}"
OUTPUT_DIR="${OUTPUT_DIR:-../bin}"
Expand Down
68 changes: 68 additions & 0 deletions clients/build-rollup-boost.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
#!/bin/bash

set -e

# Source versions if available, otherwise use defaults
if [ -f "versions.env" ]; then
source versions.env
else
# Default values
ROLLUP_BOOST_REPO="${ROLLUP_BOOST_REPO:-https://github.com/flashbots/rollup-boost.git}"
ROLLUP_BOOST_VERSION="${ROLLUP_BOOST_VERSION:-main}"
BUILD_DIR="${BUILD_DIR:-./build}"
OUTPUT_DIR="${OUTPUT_DIR:-../bin}"
fi

echo "Building rollup-boost binary..."
echo "Repository: $ROLLUP_BOOST_REPO"
echo "Version/Commit: $ROLLUP_BOOST_VERSION"
echo "Build directory: $BUILD_DIR"
echo "Output directory: $OUTPUT_DIR"

# Create build directory if it doesn't exist
mkdir -p "$BUILD_DIR"
cd "$BUILD_DIR"

# Clone or update repository
if [ -d "rollup-boost" ]; then
echo "Updating existing rollup-boost repository..."
cd rollup-boost
git fetch origin
else
echo "Cloning rollup-boost repository..."
git clone "$ROLLUP_BOOST_REPO" rollup-boost
cd rollup-boost
fi

# Checkout specified version/commit
echo "Checking out version: $ROLLUP_BOOST_VERSION"
git checkout "$ROLLUP_BOOST_VERSION"

# Build the binary using cargo
echo "Building rollup-boost with cargo..."
cargo build --bin rollup-boost --release

# Copy binary to output directory
echo "Copying binary to output directory..."
mkdir -p "../../$OUTPUT_DIR"

# Find the built binary and copy it
if [ -f "target/release/rollup-boost" ]; then
cp target/release/rollup-boost "../../$OUTPUT_DIR/"
echo "rollup-boost binary built successfully and placed in $OUTPUT_DIR/rollup-boost"
else
echo "Error: rollup-boost binary not found in target/release/"
exit 1
fi

echo ""
echo "✓ Rollup-boost built successfully!"
echo ""
echo "To use with dual-builder flashblocks mode:"
echo " ./bin/base-bench run \\"
echo " --config ./configs/examples/flashblocks.yml \\"
echo " --rbuilder-bin ./bin/op-rbuilder \\"
echo " --flashblocks-fallback reth \\"
echo " --rollup-boost-bin ./bin/rollup-boost \\"
echo " --output-dir ./output"

12 changes: 8 additions & 4 deletions clients/versions.env
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,19 @@

# Reth Configuration
RETH_REPO="https://github.com/paradigmxyz/reth/"
RETH_VERSION="v1.8.1"
RETH_VERSION="v1.8.2"

# Op-Geth Configuration
GETH_REPO="https://github.com/ethereum-optimism/op-geth/"
GETH_VERSION="v1.101511.0"
GETH_VERSION="v1.101603.1"

# Op-Rbuilder Configuration
RBUILDER_REPO="https://github.com/haardikk21/op-rbuilder"
RBUILDER_VERSION="a8bb38693ece585e7fa98d52f51290e7dcececff"
RBUILDER_REPO="https://github.com/base/op-rbuilder.git"
RBUILDER_VERSION="bc7e167a8d11362a78b9c30d59adcd8d2c7f9e84"

# Rollup-Boost Configuration (for flashblocks dual-builder mode)
ROLLUP_BOOST_REPO="https://github.com/flashbots/rollup-boost.git"
ROLLUP_BOOST_VERSION="08ebd3e75a8f4c7ebc12db13b042dee04e132c05"

# Build Configuration
BUILD_DIR="./build"
Expand Down
64 changes: 63 additions & 1 deletion configs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ This directory contains benchmark configurations for testing various aspects of
| [📄 simulator.yml](./examples/simulator.yml) | Simulation | Comprehensive workload with mixed operations | 90M |
| [📄 snapshot.yml](./examples/snapshot.yml) | Infrastructure | Tests snapshot creation and loading | 15M-90M |
| [📄 tx-fuzz-geth.yml](./examples/tx-fuzz-geth.yml) | Stress Test | Randomized transaction pattern testing | Default |
| [📄 flashblocks.yml](./examples/flashblocks.yml) | Flashblocks | Tests flashblocks (sub-200ms preconfirmations) | 30M-90M |
| [📄 client-args-demo.yml](./examples/client-args-demo.yml) | Configuration | Demonstrates custom client argument usage | 30M-60M |

## 📁 Public Configurations

Expand Down Expand Up @@ -77,11 +79,71 @@ benchmarks:
- name: "Benchmark Name"
description: "What this benchmark tests"
variables:
- type: payload|node_type|num_blocks|gas_limit
- type: payload|node_type|num_blocks|gas_limit|client_args|flashblock_interval
value: single-value
values: [array, of, values] # for matrix testing
```

### Client Arguments

You can customize client behavior using the `client_args` variable:

```yaml
# Per-client argument mapping
- type: client_args
value:
geth: "--verbosity 4 --txpool.globalslots 20000000"
reth: "--txpool.pending-max-count 200000000 -vvvv"
rbuilder: "--txpool.pending-max-count 200000000"

# Or test multiple configurations
- type: client_args
values:
- geth: "--verbosity 3"
- geth: "--verbosity 4"
- geth: "--verbosity 5"
```

### Flashblocks Support

Flashblocks provides sub-200ms transaction preconfirmations using the `rbuilder` client in two modes:

**Simple mode (standalone):**
```yaml
- type: node_type
value: rbuilder # Just rbuilder for testing
```

**Dual-builder mode (production):**
```bash
# Use CLI flags to enable dual-builder architecture
./bin/base-bench run \
--config ./configs/examples/flashblocks.yml \
--rbuilder-bin ./bin/op-rbuilder \
--flashblocks-fallback reth \
--rollup-boost-bin ./bin/rollup-boost \
--output-dir ./output
```

In dual-builder mode:
- **Fallback builder** (geth/reth): Produces final 2s canonical blocks
- **Rbuilder** (primary): Produces flashblocks every 200ms (10 per 2s block)
- **Rollup-boost** (optional): Coordinates between the two builders

**Configurable Flashblock Interval:**

You can customize the flashblock interval (default 200ms) using the `flashblock_interval` variable:

```yaml
- type: flashblock_interval
values:
- 100 # 100ms (20 flashblocks per 2s block)
- 200 # 200ms (10 flashblocks per 2s block) - Base default
- 500 # 500ms (4 flashblocks per 2s block)
```

This only applies to the `rbuilder` node type.

## 🎯 Choosing the Right Configuration

- **Development/Testing**: Use `examples/` configurations for focused testing
Expand Down
Loading
Loading