Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions docker/builder/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,18 +18,23 @@ RUN apt-get update && apt-get install -y \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*

# Install Go 1.24.7
RUN wget -c https://golang.org/dl/go1.24.7.linux-amd64.tar.gz -O - | tar -xz -C /usr/local
# Install Go 1.24.7 (auto-detect architecture)
ARG TARGETARCH
RUN GOARCH=${TARGETARCH:-amd64} && \
wget -c https://golang.org/dl/go1.24.7.linux-${GOARCH}.tar.gz -O - | tar -xz -C /usr/local
ENV PATH=$PATH:/usr/local/go/bin

# Create foc-user and foc-group with matching host UID/GID when it is run. The 1002 below is just a placeholder which will be replaced during build process. See `docker.rs`
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user && \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user && \
mkdir -p /home/foc-user/go/pkg && \
chown -R foc-user:foc-group /home/foc-user
chown -R foc-user:$GROUP_NAME /home/foc-user

# Install Rust as foc-user
USER foc-user
Expand All @@ -51,7 +56,7 @@ RUN curl -fsSL https://deb.nodesource.com/setup_24.x | bash - && \
WORKDIR /workspace

# Define volumes for external access
VOLUME ["/home/foc-user/.cargo", "/home/foc-user/.rustup", "/home/foc-user/go", "/var/tmp/filecoin-proof-parameters"]
Copy link
Collaborator

@redpanda-f redpanda-f Jan 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed? This is good for caching.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only works when there is data present. The first time you run the mount, it will clear these directories, causing the toolchain to be lost. I added the critical mappings in a subsequent commit.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is true, but it cleans up only the first time. The caches are for when the curio or lotus codebase is actively being changed and rebuilt frequently. In those instances, re-init is not needed, and these caches would be useful.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

The cargo bin directory was overwritten, so this binary will never exist unless it is manually downloaded on the host machine.

There are two commits here: one temporarily disables everything, and the second maps according to what's actually usable. The -v parameter plays a crucial role. Therefore, I've removed it here because I think it's meaningless.

However, I must admit that some Rust binaries are not cached. The rest, such as go pkg and rust registry, are cached.

VOLUME ["/var/tmp/filecoin-proof-parameters"]

# Default command
CMD ["/bin/bash"]
23 changes: 18 additions & 5 deletions docker/builder/volumes_map.toml
Original file line number Diff line number Diff line change
@@ -1,9 +1,22 @@
# Volume mappings for builder Docker image
# Format: host_subdirectory = "container_path"
# Host paths will be relative to ~/.foc-devnet/artifacts/docker/volumes/builder/
# Host paths will be relative to ~/.foc-devnet/docker/volumes/cache/foc-c-builder/
#
# Strategy: Mount only dependency cache directories, not tool directories
# This allows caching of downloaded dependencies while keeping tools functional.

[volumes]
cargo = "/home/foc-user/.cargo"
rustup = "/home/foc-user/.rustup"
go = "/home/foc-user/go"
foundry = "/home/foc-user/.foundry"
# Cargo dependency caches (NOT the cargo binary itself)
cargo-registry = "/home/foc-user/.cargo/registry" # crates.io downloads
cargo-git = "/home/foc-user/.cargo/git" # git dependencies

# Go module cache (NOT the go toolchain)
go-pkg = "/home/foc-user/go/pkg" # go modules cache

# Note: We do NOT mount:
# - /home/foc-user/.cargo/bin (contains cargo/rustc binaries)
# - /home/foc-user/.rustup/toolchains (contains rust toolchain)
# - /home/foc-user/.rustup/downloads (rustup deletes files after use)
# - /home/foc-user/.foundry (contains forge/cast binaries)
# - /usr/local/go (Go toolchain itself)

14 changes: 9 additions & 5 deletions docker/curio/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,20 @@ RUN apt-get update && apt-get install -y \
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Create necessary directories with proper ownership
RUN mkdir -p /var/tmp/filecoin-proof-parameters \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
mkdir -p /var/tmp/filecoin-proof-parameters \
&& mkdir -p /home/foc-user/.curio \
&& mkdir -p /home/foc-user/curio/fast-storage \
&& mkdir -p /home/foc-user/curio/long-term-storage \
&& chown -R foc-user:foc-group /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:foc-group /home/foc-user
&& chown -R foc-user:$GROUP_NAME /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:$GROUP_NAME /home/foc-user

# Switch to foc-user
USER foc-user
Expand Down
14 changes: 9 additions & 5 deletions docker/lotus-miner/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -19,16 +19,20 @@ RUN apt-get update && apt-get install -y \
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Create necessary directories with proper ownership
RUN mkdir -p /var/tmp/filecoin-proof-parameters \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
mkdir -p /var/tmp/filecoin-proof-parameters \
&& mkdir -p /home/foc-user/.lotus-miner-local-net \
&& mkdir -p /home/foc-user/.lotus-local-net \
&& mkdir -p /home/foc-user/.genesis-sectors \
&& chown -R foc-user:foc-group /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:foc-group /home/foc-user
&& chown -R foc-user:$GROUP_NAME /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:$GROUP_NAME /home/foc-user

# Switch to foc-user
USER foc-user
Expand Down
14 changes: 9 additions & 5 deletions docker/lotus/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,19 @@ RUN apt-get update && apt-get install -y \
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Create necessary directories with proper ownership
RUN mkdir -p /var/tmp/filecoin-proof-parameters \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
mkdir -p /var/tmp/filecoin-proof-parameters \
&& mkdir -p /home/foc-user/.lotus-local-net \
&& mkdir -p /home/foc-user/.genesis \
&& chown -R foc-user:foc-group /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:foc-group /home/foc-user
&& chown -R foc-user:$GROUP_NAME /var/tmp/filecoin-proof-parameters \
&& chown -R foc-user:$GROUP_NAME /home/foc-user

# Switch to foc-user
USER foc-user
Expand Down
12 changes: 8 additions & 4 deletions docker/yugabyte/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,11 @@ ENV LC_ALL=en_US.UTF-8
ARG USER_ID=1002
ARG GROUP_ID=1002
RUN userdel ubuntu || true
RUN groupadd -g ${GROUP_ID} foc-group && \
useradd -l -u ${USER_ID} -g foc-group -m -s /bin/bash foc-user
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1 || echo "") && \
if [ -z "$GROUP_NAME" ]; then \
groupadd -g ${GROUP_ID} foc-group && GROUP_NAME=foc-group; \
fi && \
useradd -l -u ${USER_ID} -g $GROUP_NAME -m -s /bin/bash foc-user

# Set working directory
WORKDIR /yugabyte
Expand All @@ -30,9 +33,10 @@ ARG YUGABYTE_SOURCE=yugabyte
COPY ${YUGABYTE_SOURCE} /yugabyte

# Run post-install script and set ownership
RUN /yugabyte/bin/post_install.sh && \
RUN GROUP_NAME=$(getent group ${GROUP_ID} | cut -d: -f1) && \
/yugabyte/bin/post_install.sh && \
mkdir -p /yugabyte/data && \
chown -R foc-user:foc-group /yugabyte
chown -R foc-user:$GROUP_NAME /yugabyte

# Switch to foc-user
USER foc-user
Expand Down
18 changes: 14 additions & 4 deletions scripts/cache-artifacts.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,21 @@ echo "✅ Created artifacts directory: $FOC_ARTIFACTS"
# Cache Yugabyte tarball
echo
echo "📦 Caching Yugabyte tarball..."
if [ -f "$STASH_DIR/yugabyte/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz" ]; then
cp "$STASH_DIR/yugabyte/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz" "$FOC_ARTIFACTS/"
echo "✅ Yugabyte cached (~422 MB)"

# Detect architecture for YugabyteDB
ARCH=$(uname -m)
if [ "$ARCH" = "aarch64" ] || [ "$ARCH" = "arm64" ]; then
YUGABYTE_ARCH_SUFFIX="el8-aarch64"
else
YUGABYTE_ARCH_SUFFIX="linux-x86_64"
fi
YUGABYTE_TARBALL="yugabyte-2.25.1.0-b381-${YUGABYTE_ARCH_SUFFIX}.tar.gz"

if [ -f "$STASH_DIR/yugabyte/$YUGABYTE_TARBALL" ]; then
cp "$STASH_DIR/yugabyte/$YUGABYTE_TARBALL" "$FOC_ARTIFACTS/"
echo "✅ Yugabyte cached ($YUGABYTE_ARCH_SUFFIX, ~422 MB)"
else
echo "⚠️ Warning: Yugabyte tarball not found in $STASH_DIR/yugabyte/"
echo "⚠️ Warning: Yugabyte tarball not found: $STASH_DIR/yugabyte/$YUGABYTE_TARBALL"
echo " This will be downloaded during init (~422 MB)"
fi

Expand Down
3 changes: 1 addition & 2 deletions src/commands/build/docker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,7 @@ pub fn setup_docker_run_args(

let mut docker_run_args = vec![
"run".to_string(),
"-u".to_string(),
"foc-user".to_string(),
"--rm".to_string(),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unsure if the intent really is:

  • "do not use foc-user for building"
  • "remove" the container, lose logs

I don't disagree with them, but would like to understand safety argument for removing -u foc-user.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --rm option is used because if cargo run --build lotus fails, subsequent runs will save the changes, requiring manual cleanup. Error logs will be output upon exit, so there's no need to worry.

The -u foc-user was removed because you added -u gid:uid repeatedly later, so this redundant part was removed.

"--name".to_string(),
container_name,
"-e".to_string(),
Expand Down
38 changes: 28 additions & 10 deletions src/commands/start/curio/verification.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@

use super::super::step::SetupContext;
use super::constants::TEST_FILE_SIZE_BYTES;
use crate::paths::foc_devnet_bin;
use rand::Rng;
use std::error::Error;
use std::fs;
Expand Down Expand Up @@ -101,38 +100,54 @@ fn create_random_test_file(temp_dir: &TempDir) -> Result<PathBuf, Box<dyn Error>
}

/// Upload test file using pdptool.
///
/// Runs pdptool inside foc-builder container (which uses --network host)
/// to test via external port, simulating real external client access.
fn upload_test_file(
context: &SetupContext,
file_path: &Path,
sp_index: usize,
) -> Result<String, Box<dyn Error>> {
// Get dynamically allocated PDP port from context
// Get dynamically allocated PDP port from context (external port)
let port: u16 = context
.get(&format!("curio_sp_{}_pdp_port", sp_index))
.ok_or("Curio PDP port not found in context")?
.parse()?;

// Use external port via host network for stricter testing
let service_url = format!("http://localhost:{}", port);

let file_path_str = file_path
.to_str()
.ok_or("Invalid file path: contains non-UTF8 characters")?;
// Mount the test file directory into foc-builder and use it
let file_dir = file_path.parent().ok_or("Invalid file path")?;
let file_name = file_path.file_name().ok_or("Invalid file name")?;
let container_file_path = format!("/tmp/test-data/{}", file_name.to_string_lossy());

// Mount bin directory where pdptool is located
let bin_dir = crate::paths::foc_devnet_bin();

let args = [
"run",
"--rm",
"--network",
"host",
"-v",
&format!("{}:/tmp/test-data", file_dir.display()),
"-v",
&format!("{}:/usr/local/bin/lotus-bins", bin_dir.display()),
crate::constants::BUILDER_DOCKER_IMAGE,
"/usr/local/bin/lotus-bins/pdptool",
"upload-piece",
"--service-url",
&service_url,
"--service-name",
"public",
"--hash-type",
"commp",
file_path_str,
&container_file_path,
"--verbose",
];

let output = Command::new(foc_devnet_bin().join("pdptool"))
.args(args)
.output()?;
let output = Command::new("docker").args(args).output()?;

if !output.status.success() {
return Err(format!(
Expand All @@ -142,7 +157,10 @@ fn upload_test_file(
.into());
}

info!("File uploaded via pdptool");
info!(
"File uploaded via pdptool (foc-builder, external port {})",
port
);

// Extract piece CID from output
let stdout = String::from_utf8_lossy(&output.stdout);
Expand Down
27 changes: 25 additions & 2 deletions src/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,9 @@ pub struct Config {
/// URL to download Yugabyte database tarball.
///
/// This is the direct link to the Yugabyte tarball required for running curio.
/// Default: https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz
/// The default URL is automatically selected based on system architecture:
/// - ARM64 (aarch64): yugabyte-2.25.1.0-b381-el8-aarch64.tar.gz
/// - x86_64: yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz
pub yugabyte_download_url: String,

/// Number of approved PDP service providers.
Expand Down Expand Up @@ -227,14 +229,35 @@ impl Default for Config {
url: "https://github.com/FilOzone/synapse-sdk.git".to_string(),
commit: "773551bf1e9cf4cdc49aeb63a47a81f8dc5cb9e1".to_string(),
},
yugabyte_download_url: "https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz".to_string(),
yugabyte_download_url: Self::get_default_yugabyte_url(),
approved_pdp_sp_count: 1,
active_pdp_sp_count: 1,
}
}
}

impl Config {
/// Get the default YugabyteDB download URL based on system architecture.
///
/// Returns the appropriate YugabyteDB tarball URL for the current platform:
/// - el8-aarch64 for ARM64 systems (Apple Silicon, AWS Graviton, etc.)
/// - linux-x86_64 for x86_64 systems
fn get_default_yugabyte_url() -> String {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This addition is a "feature update". The S3 fallback has to be changed accordingly in the code as well. Please check that.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you mean by S3 fallback here? But I do see a place I forgot to update.

# Cache Yugabyte tarball

const YUGABYTE_VERSION: &str = "2.25.1.0";
const YUGABYTE_BUILD: &str = "b381";

let arch_suffix = if std::env::consts::ARCH == "aarch64" {
"el8-aarch64"
} else {
"linux-x86_64"
};

format!(
"https://software.yugabyte.com/releases/{}/yugabyte-{}-{}-{}.tar.gz",
YUGABYTE_VERSION, YUGABYTE_VERSION, YUGABYTE_BUILD, arch_suffix
)
}

/// Validate configuration values.
///
/// Ensures that:
Expand Down