feat: program commitment (bytecode + program image)

[quangvdao]

Overview

This PR introduces Committed Program Mode, extending the bytecode commitment work to also commit to the initial RAM (program image). This enables fully succinct verification independent of both bytecode size and program image size.

This PR supersedes #1213 (bytecode commitment only) by:

1. Adding program image commitment alongside bytecode commitment
2. Unifying BytecodePreprocessing and ProgramImagePreprocessing into a single ProgramPreprocessing type
3. Creating unified TrustedProgramCommitments for both bytecode and program image commitments
4. Removing verifier's need to materialize initial RAM state

---

Key Additions (Beyond Bytecode Commitment)

1. Program Image Commitment

New file: jolt-core/src/zkvm/claim_reductions/program_image.rs

Claim reduction sumcheck that binds the program-image contribution to Val_init(r_address) claims from Stage 4 to a trusted commitment:

• ProgramImageClaimReductionParams: Stores gamma, opening points, padded length
• ProgramImageClaimReductionProver/Verifier: Runs the claim reduction sumcheck
• Caches a CommittedPolynomial::ProgramImageInit opening for Stage 8 batching

RAM module additions (jolt-core/src/zkvm/ram/mod.rs):

• prover_accumulate_program_image(): Computes and stages program-image contribution claims
• verifier_accumulate_program_image(): Stages opening points without materializing RAM

2. Unified Program Preprocessing

New file: jolt-core/src/zkvm/program.rs

Consolidated types for all static program data:

pub struct ProgramPreprocessing {
    // Bytecode (instructions)
    pub instructions: Vec<Instruction>,
    pub pc_map: BytecodePCMapper,
    // Program image (RAM init)
    pub min_bytecode_address: u64,
    pub program_image_words: Vec<u64>,
}

pub struct ProgramMetadata {
    pub min_bytecode_address: u64,
    pub program_image_len_words: usize,
    pub bytecode_len: usize,
}

pub struct TrustedProgramCommitments<PCS> {
    // Bytecode commitments
    pub bytecode_commitments: Vec<PCS::Commitment>,
    pub bytecode_num_columns: usize,
    pub log_k_chunk: u8,
    pub bytecode_len: usize,
    // Program image commitment
    pub program_image_commitment: PCS::Commitment,
    pub program_image_num_columns: usize,
    pub program_image_num_words: usize,
}

pub enum VerifierProgram<PCS> {
    Full(Arc<ProgramPreprocessing>),
    Committed(TrustedProgramCommitments<PCS>),
}

Removed file: jolt-core/src/zkvm/program_image.rs (folded into program.rs)

3. Verifier No Longer Materializes Initial RAM

In committed mode, the verifier:

• Does NOT call build_initial_ram_state() (avoids O(program_size) work)
• Uses verifier_accumulate_program_image() to stage claims
• Runs ProgramImageClaimReductionVerifier to bind claims to commitment
• Verifies batched Stage 8 opening against program_image_commitment

4. Stage 8 Program Image Integration

File: jolt-core/src/poly/rlc_polynomial.rs

• Added ProgramImageInit handling in StreamingRLCContext
• Program image polynomial embedded with trace-dense layout (missing lane variables)
• Supports both CycleMajor and AddressMajor layouts

---

Preprocessing API Changes

Before (separate types)

let bytecode = BytecodePreprocessing::preprocess(instructions);
let program_image = ProgramImagePreprocessing::preprocess(memory_init);
let shared = JoltSharedPreprocessing::new(&bytecode, layout, program_image.meta(), trace_len);
let prover = JoltProverPreprocessing::new(shared, bytecode, program_image);

After (unified type)

let program = Arc::new(ProgramPreprocessing::preprocess(instructions, memory_init));
let shared = JoltSharedPreprocessing::new(program.meta(), layout, trace_len);
let prover = JoltProverPreprocessing::new(shared, program);
// or for committed mode:
let prover = JoltProverPreprocessing::new_committed(shared, program);

---

Key Difficulties & Solutions

1. Program Image Padding for Main Sigma Compatibility

Problem: Small program images may have fewer words than Main's num_columns, making the matrix degenerate.

Solution: In committed mode, pad program image to at least main_num_columns:

let padded_len_words = unpadded_len_words
    .next_power_of_two()
    .max(main_num_columns);

2. Trace-Dense Embedding for Program Image

Problem: Program image lacks lane variables, so it must be embedded specially in Stage 8.

Solution: Use contiguous block embedding (CycleMajor) or stride-by-K embedding (AddressMajor), mirroring advice polynomial handling.

3. Stage 4 Claim Staging

Problem: Stage 4's Val_init(r_address) computation requires O(program_size) work for the program-image contribution.

Solution: In committed mode, prover stages scalar claims via prover_accumulate_program_image(), which are later bound to the commitment in Stage 6b via claim reduction.

---

Files Changed (Summary)

New Files

• jolt-core/src/zkvm/program.rs - Unified program preprocessing
• jolt-core/src/zkvm/claim_reductions/program_image.rs - Program image claim reduction

Deleted Files

• jolt-core/src/zkvm/program_image.rs - Superseded by program.rs

Major Changes

• jolt-core/src/zkvm/prover.rs - Unified preprocessing, program image accumulation
• jolt-core/src/zkvm/verifier.rs - Program image verification, claim reduction
• jolt-core/src/zkvm/ram/mod.rs - prover/verifier_accumulate_program_image functions
• jolt-core/src/poly/rlc_polynomial.rs - Stage 8 program image handling
• jolt-sdk/macros/src/lib.rs - Updated preprocessing API

64 files changed, 6832 insertions(+), 1627 deletions(-)

---

Testing

All existing tests pass, including:

• fib_e2e_committed_bytecode (CycleMajor + AddressMajor)
• bytecode_mode_detection_full/committed
• All 20 zkvm e2e tests

---

Usage

// Full mode (default) - verifier has all program data
let prover_prep = JoltProverPreprocessing::new(shared, program);

// Committed mode - verifier only has commitments
let prover_prep = JoltProverPreprocessing::new_committed(shared, program);

// CLI flag
cargo run --release --example fibonacci -- --committed-bytecode