Compression integration (draft) [JOLT-191] #1134

JuI3s · 2025-11-26T23:46:12Z

No description provided.

graphite-app · 2025-11-27T05:12:22Z

jolt-core/src/poly/commitment/dory/commitment_scheme.rs

+    fn commit_compressed(
+        poly: &MultilinearPolynomial<ark_bn254::Fr>,
+        setup: &Self::ProverSetup,
+    ) -> (Self::CompressedCommitment, Self::OpeningProofHint) {
+        let _span = trace_span!("DoryCommitmentScheme::commit").entered();
+
+        let num_cols = DoryGlobals::get_num_columns();
+        let num_rows = DoryGlobals::get_max_num_rows();
+        let sigma = num_cols.log_2();
+        let nu = num_rows.log_2();
+
+        todo!()
+        // let (tier_2, row_commitments) = <MultilinearPolynomial<ark_bn254::Fr> as Polynomial<
+        //     ArkFr,
+        // >>::commit_compressed::<ArkBn254, JoltG1Routines>(
+        //     poly, nu, sigma, setup
+        // )
+        // .expect("commitment should succeed");
+
+        // (tier_2, row_commitments)
+    }


The commit_compressed method currently contains a todo!() statement that will panic at runtime if called. Before this PR can be merged, this method needs to be fully implemented. Consider uncommenting and completing the code below the todo!() statement, which appears to be the intended implementation path. This is particularly important since the compressed commitment functionality is a core part of this integration.

Suggested change

fn commit_compressed(

poly: &MultilinearPolynomial<ark_bn254::Fr>,

setup: &Self::ProverSetup,

) -> (Self::CompressedCommitment, Self::OpeningProofHint) {

let _span = trace_span!("DoryCommitmentScheme::commit").entered();

let num_cols = DoryGlobals::get_num_columns();

let num_rows = DoryGlobals::get_max_num_rows();

let sigma = num_cols.log_2();

let nu = num_rows.log_2();

todo!()

// let (tier_2, row_commitments) = <MultilinearPolynomial<ark_bn254::Fr> as Polynomial<

// ArkFr,

// >>::commit_compressed::<ArkBn254, JoltG1Routines>(

// poly, nu, sigma, setup

// )

// .expect("commitment should succeed");

// (tier_2, row_commitments)

}

fn commit_compressed(

poly: &MultilinearPolynomial<ark_bn254::Fr>,

setup: &Self::ProverSetup,

) -> (Self::CompressedCommitment, Self::OpeningProofHint) {

let _span = trace_span!("DoryCommitmentScheme::commit").entered();

let num_cols = DoryGlobals::get_num_columns();

let num_rows = DoryGlobals::get_max_num_rows();

let sigma = num_cols.log_2();

let nu = num_rows.log_2();

let (tier_2, row_commitments) = <MultilinearPolynomial<ark_bn254::Fr> as Polynomial<

ArkFr,

>>::commit_compressed::<ArkBn254, JoltG1Routines>(poly, nu, sigma, setup)

.expect("commitment should succeed");

(tier_2, row_commitments)

}

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-27T05:17:06Z

Cargo.toml

 ark-ec = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout" }
 ark-serialize = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout" }
 allocative = { git = "https://github.com/facebookexperimental/allocative", rev = "85b773d85d526d068ce94724ff7a7b81203fc95e" }
+dory-pcs = { git = "https://github.com/a16z/dory", branch = "dev/twist-shout" }


Inconsistency between Cargo.toml and Cargo.lock for dory-pcs dependency. Cargo.toml adds a patch pointing to a16z/dory branch dev/twist-shout, but Cargo.lock shows the dependency is resolved from JuI3s/dory branch compression-integration. This mismatch will cause build issues and dependency resolution failures.

# Cargo.toml line 130 adds: dory-pcs = { git = "https://github.com/a16z/dory", branch = "dev/twist-shout" } # But Cargo.lock shows: source = "git+https://github.com/JuI3s/dory?branch=compression-integration#..."

Update Cargo.toml to match the actual dependency being used:

dory-pcs = { git = "https://github.com/JuI3s/dory", branch = "compression-integration" }

Suggested change

dory-pcs = { git = "https://github.com/a16z/dory", branch = "dev/twist-shout" }

dory-pcs = { git = "https://github.com/JuI3s/dory", branch = "compression-integration" }

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

graphite-app · 2025-11-27T05:17:07Z

Cargo.toml

 # Cryptography and Math
 ark-bn254 = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }
-ark-grumpkin = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }
+ark-grumpkin = { git = "https://github.com/JuI3s/arkworks-algebra", branch = "compression", default-features = false }
 ark-ec = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }
 ark-ff = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }


Inconsistent arkworks-algebra sources will cause version conflicts. Most dependencies use a16z/arkworks-algebra branch dev/twist-shout (lines 136, 138-140) while ark-grumpkin uses JuI3s/arkworks-algebra branch compression (line 137). Since these packages are tightly coupled, mixing sources will lead to type incompatibilities and compilation failures.

# Line 136: a16z repo ark-bn254 = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", ... } # Line 137: JuI3s repo (different!) ark-grumpkin = { git = "https://github.com/JuI3s/arkworks-algebra", branch = "compression", ... } # Lines 138-140: back to a16z repo ark-ec = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", ... }

All arkworks dependencies should use the same repository and branch to ensure compatibility.

Suggested change

# Cryptography and Math

ark-bn254 = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

ark-grumpkin = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

ark-grumpkin = { git = "https://github.com/JuI3s/arkworks-algebra", branch = "compression", default-features = false }

ark-ec = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

ark-ff = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

# Cryptography and Math

ark-bn254 = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

ark-grumpkin = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

ark-ec = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

ark-ff = { git = "https://github.com/a16z/arkworks-algebra", branch = "dev/twist-shout", default-features = false }

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

* change default guest opt level to 3. Add env var for specifying opt level * make clippy happy

``` feat(zkvm): add proof size tracking and testing infrastructure Add comprehensive JoltProofFieldSizeTracker for measuring serialized sizes of proof components. Replace unimplemented serialization stub with no-op for testing. Minor code style improvements. ```

graphite-app · 2025-12-02T21:03:57Z

jolt-core/src/zkvm/verifier.rs

+    pub fn uncompressed_commitments(mut self) -> Self {
+        match self.proof.commitments {
+            JoltProofCommitments::Compressed(commitments) => {
+                let uncompressed_commitments = commitments
+                    .into_iter()
+                    .map(|commitment| commitment.into())
+                    .collect();
+                self.proof.commitments =
+                    JoltProofCommitments::Uncompressed(uncompressed_commitments);
+                self
+            }
+            JoltProofCommitments::Uncompressed(_) => self,
+        }
+    }
+
    #[tracing::instrument(skip_all)]
    pub fn verify(mut self) -> Result<(), anyhow::Error> {
+        // Uncompress commitments if they are compressed
+        self = self.uncompressed_commitments();
+


Critical transcript mismatch bug: The verifier converts compressed commitments to uncompressed before appending them to the transcript (lines 157-158), but the prover appends the original compressed commitments to the transcript (lines 476-479 in prover.rs). This causes a transcript mismatch that will always fail verification.

Fix: The transcript must append the same commitment representation that the prover used:

pub fn verify(mut self) -> Result<(), anyhow::Error> { let _pprof_verify = pprof_scope!("verify"); // Append commitments to transcript BEFORE uncompressing match &self.proof.commitments { JoltProofCommitments::Compressed(commitments) => { for commitment in commitments { self.transcript.append_serializable(commitment); } } JoltProofCommitments::Uncompressed(commitments) => { for commitment in commitments { self.transcript.append_serializable(commitment); } } }; // NOW uncompress for verification self = self.uncompressed_commitments(); // ... rest of verification }

Remove the duplicate transcript appending at lines 179-188 since commitments are now appended above.

Suggested change

pub fn uncompressed_commitments(mut self) -> Self {

match self.proof.commitments {

JoltProofCommitments::Compressed(commitments) => {

let uncompressed_commitments = commitments

.into_iter()

.map(|commitment| commitment.into())

.collect();

self.proof.commitments =

JoltProofCommitments::Uncompressed(uncompressed_commitments);

self

}

JoltProofCommitments::Uncompressed(_) => self,

}

}

#[tracing::instrument(skip_all)]

pub fn verify(mut self) -> Result<(), anyhow::Error> {

// Uncompress commitments if they are compressed

self = self.uncompressed_commitments();

pub fn uncompressed_commitments(mut self) -> Self {

match self.proof.commitments {

JoltProofCommitments::Compressed(commitments) => {

let uncompressed_commitments = commitments

.into_iter()

.map(|commitment| commitment.into())

.collect();

self.proof.commitments =

JoltProofCommitments::Uncompressed(uncompressed_commitments);

self

}

JoltProofCommitments::Uncompressed(_) => self,

}

}

#[tracing::instrument(skip_all)]

pub fn verify(mut self) -> Result<(), anyhow::Error> {

let _pprof_verify = pprof_scope!("verify");

// Append commitments to transcript BEFORE uncompressing

match &self.proof.commitments {

JoltProofCommitments::Compressed(commitments) => {

for commitment in commitments {

self.transcript.append_serializable(commitment);

}

}

JoltProofCommitments::Uncompressed(commitments) => {

for commitment in commitments {

self.transcript.append_serializable(commitment);

}

}

};

// NOW uncompress for verification

self = self.uncompressed_commitments();

Spotted by Graphite Agent

Is this helpful? React 👍 or 👎 to let us know.

Implement torus-based compression for GT elements in Dory PCS, reducing proof and commitment sizes. NOTE: More changes are pending. Changes: - Add CompressedArkDoryProof with custom CanonicalSerialize/Deserialize that uses ArkGTCompressed instead of ArkGT for GT elements - Add ArkGTCompressed wrapper around CompressedFq12 with DorySerialize/ DoryDeserialize and Valid trait implementations - Add CompressedCommitmentScheme trait to CommitmentScheme for schemes that support compressed proofs - Implement CompressedPairingCurve for JoltBn254 with compressed multi- pairing - Fix Valid::check() for ArkGTCompressed by checking inner Fq2 elements since CompressedFq12 doesn't implement ark_serialize::Valid This enables smaller proof sizes by using torus compression for GT elements in the BN254 pairing, which compresses 12 field elements down to 4 field elements.

JuI3s added 5 commits November 26, 2025 18:45

Update commitment scheme trait to include compressed target field

5717045

Compressed commitment implementation

545301e

temp

1965545

aggregate_chunks_compressed

0b9169a

Commit compressed in progress

a49ebfa

graphite-app bot reviewed Nov 27, 2025

View reviewed changes

Fix Cargo.toml

3e632f8

graphite-app bot reviewed Nov 27, 2025

View reviewed changes

JuI3s added 2 commits November 27, 2025 12:52

Dory Prover/Verifier setup

985e303

update toml

2eb13d3

JuI3s force-pushed the compression-integration branch from 6489b2a to d1b6d0c Compare November 27, 2025 17:59

fix toml

b31cb94

JuI3s force-pushed the compression-integration branch from d1b6d0c to b31cb94 Compare November 27, 2025 18:00

JuI3s added 2 commits November 27, 2025 13:04

fmt

3fd5217

Disable public visibility of DoryBn254

0da7b59

0xAndoroid changed the title ~~Compression integration (draft)~~ Compression integration (draft) [JOLT-191] Dec 1, 2025

markosg04 self-requested a review December 1, 2025 16:30

mathmasterzach and others added 6 commits December 1, 2025 22:27

feat: customizable guest optimization level (a16z#1135)

37f52ca

* change default guest opt level to 3. Add env var for specifying opt level * make clippy happy

Patch for dev

148b6f8

expect test

3e65b6a

temp

fac5c10

integration in progress

b83d918

graphite-app bot reviewed Dec 2, 2025

View reviewed changes

JuI3s added 6 commits December 2, 2025 20:04

Fix compressed prover/verifier transcript bug

a485f03

Implement DoryElement for compressed pairing GT

62bdecd

Before deleting Jolt commitment

e5a21c7

temp

01be1c4

Temp

dbee230

JuI3s added 3 commits December 4, 2025 15:40

Refactor ProverOpeningAccumulator::reduce_and_prove_sumcheck_helper

2496a42

temp

0b3ab94

prove_compressed

c61027e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Compression integration (draft) [JOLT-191] #1134

Compression integration (draft) [JOLT-191] #1134

Uh oh!

JuI3s commented Nov 26, 2025

Uh oh!

graphite-app bot Nov 27, 2025

Uh oh!

graphite-app bot Nov 27, 2025

Uh oh!

graphite-app bot Nov 27, 2025

Uh oh!

graphite-app bot Dec 2, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	dory-pcs = { git = "https://github.com/a16z/dory", branch = "dev/twist-shout" }
	dory-pcs = { git = "https://github.com/JuI3s/dory", branch = "compression-integration" }

Compression integration (draft) [JOLT-191] #1134

Are you sure you want to change the base?

Compression integration (draft) [JOLT-191] #1134

Uh oh!

Conversation

JuI3s commented Nov 26, 2025

Uh oh!

graphite-app bot Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

graphite-app bot Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

graphite-app bot Nov 27, 2025

Choose a reason for hiding this comment

Uh oh!

graphite-app bot Dec 2, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants