Skip to content

Perf/gkr/solvable var #1471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 70 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
7a27f88
build: update gnark-crypto dep
Tabaie Feb 25, 2025
be478af
fix: use registry in testing.go
Tabaie Feb 25, 2025
54a8fcc
refactor: adapt gkr package to gnark-crypto changes
Tabaie Mar 4, 2025
d66a1b9
refactor: linearVar -> solvableVar
Tabaie Mar 5, 2025
c19628b
fix: poseidon2 Compress feed fwd
Tabaie Mar 5, 2025
fc62f36
style: gofmt
Tabaie Mar 5, 2025
d1ff5f2
Merge branch 'master' into chore/gkr-gateregistry
Tabaie Mar 5, 2025
4cd2932
fix: GKR testing
Tabaie Mar 6, 2025
7f13b98
build: gnark-crypto checksum
Tabaie Mar 6, 2025
71b3101
build: update gnark-crypto dep
Tabaie Mar 27, 2025
8455981
Merge branch 'master' into chore/gkr-gateregistry
Tabaie Mar 27, 2025
9d5b88e
build: go generate
Tabaie Mar 27, 2025
218128a
refactor: to match GKR Gates@gnark-crypto
Tabaie Mar 27, 2025
948f95c
chore: gkr.GateName
Tabaie Mar 27, 2025
25e482d
chore: address copilot PR feedback
Tabaie Mar 27, 2025
a12e759
nitpick: "identity" -> Identity
Tabaie Mar 27, 2025
cbaf5fd
checkpoint: up to line 418
Tabaie Mar 23, 2025
509942b
checkpoint: line 422
Tabaie Mar 23, 2025
7d4e063
docs: complete test for DoubleAssign with GKR
Tabaie Mar 23, 2025
e030c50
fix gate name
Tabaie Mar 23, 2025
da5c15f
test: works under test engine, proof fails
Tabaie Mar 24, 2025
f9f576b
test with const hash
Tabaie Mar 24, 2025
b07cba3
revert: use mimc hash
Tabaie Mar 25, 2025
5e222e4
replicating error with const hash
Tabaie Mar 25, 2025
f5e7410
checkpoint
Tabaie Mar 27, 2025
a2fc86f
chore: example works
Tabaie Mar 27, 2025
f9df20e
chore: remove prints
Tabaie Mar 27, 2025
ca23cc9
fix: gofmt
Tabaie Mar 27, 2025
7257006
remove hash reporter
Tabaie Mar 27, 2025
6997e71
gofmt
Tabaie Mar 27, 2025
f4852d9
remove checks for S
Tabaie Mar 27, 2025
235f125
build use nogkr from gnark-crypto
Tabaie Mar 27, 2025
a308811
generate code for sumcheck
Tabaie Apr 1, 2025
cb17ed9
small-rational
Tabaie Apr 1, 2025
a02d4b3
feat: generate sumcheck test vecs
Tabaie Apr 1, 2025
7edcf03
fix: codegen error
Tabaie Apr 2, 2025
f01b8b6
perf: run test vec gen in same process
Tabaie Apr 2, 2025
16b20bc
generate gkr
Tabaie Apr 2, 2025
43ea601
generate gkr test vec generator
Tabaie Apr 2, 2025
67d9c58
Merge branch 'master' into allgkr
Tabaie Apr 3, 2025
bc53066
feat nomimc option
Tabaie Apr 4, 2025
4168046
fix: no tests for small rational gkr
Tabaie Apr 4, 2025
c2b83fc
fix: gkr vec gen imports
Tabaie Apr 4, 2025
efbed6a
fix: gkr test vec gen
Tabaie Apr 4, 2025
6b4b26c
fix: gkr test vector tests
Tabaie Apr 4, 2025
338df83
fix gkr imports in constraint
Tabaie Apr 4, 2025
4bca288
fix: remove wrongly generated files
Tabaie Apr 4, 2025
b2d06ba
feat: fr side of gkr-poseidon2
Tabaie Apr 4, 2025
877856f
fix small codegen issues
Tabaie Apr 4, 2025
ecd5b35
Merge branch 'master' into allgkr
Tabaie Apr 7, 2025
4b2b183
docs: better comments for gkr
Tabaie Apr 7, 2025
188a49d
docs: generify comments
Tabaie Apr 7, 2025
e1f6f76
refactor: remove unnecessary set
Tabaie Apr 7, 2025
979dadb
Merge branch 'master' into perf/gkr/solvable-var
Tabaie Apr 7, 2025
66c7304
docs computeGJ
Tabaie Apr 7, 2025
f4c074a
docs: next, proveFinalEval
Tabaie Apr 7, 2025
c3083e2
Merge branch 'allgkr' into docs/gkr
Tabaie Apr 7, 2025
c703ae6
refactor: finalEvalProof as fr slice
Tabaie Apr 8, 2025
8733799
fix: inputevaluations
Tabaie Apr 8, 2025
f8715b1
chore: generify changes
Tabaie Apr 8, 2025
05716d7
refactor: remove casting to []fr.Element
Tabaie Apr 8, 2025
694a3de
docs remove panicky todo comment
Tabaie Apr 10, 2025
0648dbc
feat: solvable vars for bn254
Tabaie Apr 10, 2025
88850d2
fix: don't count on a slice being nil
Tabaie Apr 10, 2025
f1b2b47
chore: generify
Tabaie Apr 10, 2025
99775e6
fix: elemIndex
Tabaie Apr 10, 2025
bb611ff
Merge branch 'master' into perf/gkr/solvable-var
Tabaie Apr 10, 2025
3d3a69c
docs: comments for gkr snark verifier
Tabaie Apr 10, 2025
b3692de
Merge branch 'docs/gkr' into perf/gkr/solvable-var
Tabaie Apr 10, 2025
cf46fc3
fix: getBaseChallenge in snark verifier
Tabaie Apr 10, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions constraint/bls12-377/gkr.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions constraint/bls12-381/gkr.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions constraint/bls24-315/gkr.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions constraint/bls24-317/gkr.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions constraint/bn254/gkr.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions constraint/bw6-633/gkr.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 3 additions & 4 deletions constraint/bw6-761/gkr.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ require (
github.com/blang/semver/v4 v4.0.0
github.com/consensys/bavard v0.1.31-0.20250314194434-b30d4344e6d4
github.com/consensys/compress v0.2.5
github.com/consensys/gnark-crypto v0.17.1-0.20250326164229-5fd6610ac2a1
github.com/consensys/gnark-crypto v0.17.1-0.20250331132656-820ac1d108bd
github.com/fxamacker/cbor/v2 v2.7.0
github.com/google/go-cmp v0.6.0
github.com/google/pprof v0.0.0-20240727154555-813a5fbdbec8
Expand Down
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,10 @@ github.com/consensys/compress v0.2.5 h1:gJr1hKzbOD36JFsF1AN8lfXz1yevnJi1YolffY19
github.com/consensys/compress v0.2.5/go.mod h1:pyM+ZXiNUh7/0+AUjUf9RKUM6vSH7T/fsn5LLS0j1Tk=
github.com/consensys/gnark-crypto v0.17.1-0.20250326164229-5fd6610ac2a1 h1:6cK71BoMAjWHNl+EpvBh2PDDa0PIeoz1KFJ/6R16DjQ=
github.com/consensys/gnark-crypto v0.17.1-0.20250326164229-5fd6610ac2a1/go.mod h1:uV1HwfBwGRj50DGK3LbDLeCvq0RX/vFXST3CRSAu0Fs=
github.com/consensys/gnark-crypto v0.17.1-0.20250327163404-2fc9f58298e2 h1:vWEj2nIXK3dG2oyNlqLXVOFRGi1E3BQn+YVkwqf7GnM=
github.com/consensys/gnark-crypto v0.17.1-0.20250327163404-2fc9f58298e2/go.mod h1:uV1HwfBwGRj50DGK3LbDLeCvq0RX/vFXST3CRSAu0Fs=
github.com/consensys/gnark-crypto v0.17.1-0.20250331132656-820ac1d108bd h1:og4X8KhBpFv37u0PuXLz7yOLz9vwOcpTX4pKfbbwtgM=
github.com/consensys/gnark-crypto v0.17.1-0.20250331132656-820ac1d108bd/go.mod h1:uV1HwfBwGRj50DGK3LbDLeCvq0RX/vFXST3CRSAu0Fs=
github.com/coreos/go-semver v0.3.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd/v22 v22.3.2/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
Expand Down
121 changes: 115 additions & 6 deletions internal/generator/backend/main.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
package main

import (
"fmt"
"github.com/consensys/gnark-crypto/field/generator/config"
"os"
"os/exec"
"path/filepath"
Expand All @@ -9,7 +11,6 @@ import (

"github.com/consensys/bavard"
"github.com/consensys/gnark-crypto/field/generator"
"github.com/consensys/gnark-crypto/field/generator/config"
)

const copyrightHolder = "Consensys Software Inc."
Expand Down Expand Up @@ -79,7 +80,7 @@ func main() {
panic(err)
}

datas := []templateData{
data := []templateData{
bls12_377,
bls12_381,
bn254,
Expand All @@ -91,10 +92,9 @@ func main() {
}

const importCurve = "../imports.go.tmpl"

var wg sync.WaitGroup

for _, d := range datas {
for _, d := range data {

wg.Add(1)

Expand Down Expand Up @@ -129,10 +129,24 @@ func main() {

// gkr backend
if d.Curve != "tinyfield" {
// solver and proof delegator TODO merge with "backend" below
entries = []bavard.Entry{{File: filepath.Join(csDir, "gkr.go"), Templates: []string{"gkr.go.tmpl", importCurve}}}
if err := bgen.Generate(d, "cs", "./template/representations/", entries...); err != nil {
panic(err)
err := bgen.Generate(d, "cs", "./template/representations/", entries...)
assertNoError(err)

curvePackageName := strings.ToLower(d.Curve)

cfg := gkrConfig{
FieldDependency: config.FieldDependency{
ElementType: "fr.Element",
FieldPackageName: "fr",
FieldPackagePath: "github.com/consensys/gnark-crypto/ecc/" + curvePackageName + "/fr",
},
GkrPackageRelativePath: "internal/gkr/" + curvePackageName,
CanUseFFT: true,
}

assertNoError(generateGkrBackend(cfg))
}

entries = []bavard.Entry{
Expand Down Expand Up @@ -203,6 +217,41 @@ func main() {

}

wg.Add(1)
// GKR test vectors
go func() {
// generate gkr and sumcheck for small-rational
cfg := gkrConfig{
FieldDependency: config.FieldDependency{
ElementType: "small_rational.SmallRational",
FieldPackagePath: "github.com/consensys/gnark/internal/small_rational",
FieldPackageName: "small_rational",
},
GkrPackageRelativePath: "internal/gkr/small_rational",
CanUseFFT: false,
NoGkrTests: true,
}
assertNoError(generateGkrBackend(cfg))

// generate gkr test vector generator
cfg.GenerateTestVectors = true
cfg.OutsideGkrPackage = true

assertNoError(bgen.Generate(cfg, "gkr", "./template/gkr/",
bavard.Entry{
File: "../../gkr/test_vectors/gkr/gkr-gen-vectors.go",
Templates: []string{"gkr.test.vectors.gen.go.tmpl", "gkr.test.vectors.go.tmpl"},
},
))

fmt.Println("generating test vectors for gkr and sumcheck")
cmd := exec.Command("go", "run", "../../gkr/test_vectors")
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
assertNoError(cmd.Run())
wg.Done()
}()

wg.Wait()

// run go fmt on whole directory
Expand All @@ -223,3 +272,63 @@ type templateData struct {
noBackend bool
NoGKR bool
}

func generateGkrBackend(cfg gkrConfig) error {
const repoRoot = "../../../"
packageOutPath := filepath.Join(repoRoot, cfg.GkrPackageRelativePath)

// test vector utils
packageDir := filepath.Join(packageOutPath, "test_vector_utils")
entries := []bavard.Entry{
{File: filepath.Join(packageDir, "test_vector_utils.go"), Templates: []string{"test_vector_utils.go.tmpl"}},
}

if err := bgen.Generate(cfg, "test_vector_utils", "./template/gkr/", entries...); err != nil {
return err
}

// sumcheck backend
packageDir = filepath.Join(packageOutPath, "sumcheck")
entries = []bavard.Entry{
{File: filepath.Join(packageDir, "sumcheck.go"), Templates: []string{"sumcheck.go.tmpl"}},
{File: filepath.Join(packageDir, "sumcheck_test.go"), Templates: []string{"sumcheck.test.go.tmpl"}},
}

if err := bgen.Generate(cfg, "sumcheck", "./template/gkr/", entries...); err != nil {
return err
}

// gkr backend
packageDir = packageOutPath
entries = []bavard.Entry{
{File: filepath.Join(packageDir, "gkr.go"), Templates: []string{"gkr.go.tmpl"}},
{File: filepath.Join(packageDir, "registry.go"), Templates: []string{"registry.go.tmpl"}},
}

if !cfg.NoGkrTests {
entries = append(entries, bavard.Entry{
File: filepath.Join(packageDir, "gkr_test.go"), Templates: []string{"gkr.test.go.tmpl", "gkr.test.vectors.go.tmpl"},
})
}

if err := bgen.Generate(cfg, "gkr", "./template/gkr/", entries...); err != nil {
return err
}

return nil
}

type gkrConfig struct {
config.FieldDependency
GkrPackageRelativePath string // the GKR package, relative to the repo root
CanUseFFT bool
OutsideGkrPackage bool
GenerateTestVectors bool
NoGkrTests bool
}

func assertNoError(err error) {
if err != nil {
panic(err)
}
}
Loading