Skip to content

Commit 1b78e56

Browse files
committed
Switch to SIMD-optimized SHA256 implementation
The SHA256 implementation provided by the github.com/minio/sha256-simd package is measurably faster than the standard library's crypto/sha256 package: ``` name old speed new speed delta StreamWriteTo/tar/1.0MiB-4 394MB/s ± 6% 1024MB/s ± 6% +160.10% (p=0.000 n=9+10) StreamWriteTo/tar/8.0MiB-4 452MB/s ± 2% 1444MB/s ± 4% +219.48% (p=0.000 n=9+9) StreamWriteTo/tar/16.0MiB-4 462MB/s ± 1% 1470MB/s ± 1% +218.07% (p=0.000 n=10+9) StreamWriteTo/tar/128.0MiB-4 464MB/s ± 0% 1444MB/s ± 3% +211.36% (p=0.000 n=8+10) StreamWriteTo/tgz/1.0MiB-4 310MB/s ± 9% 594MB/s ±13% +91.77% (p=0.000 n=10+8) StreamWriteTo/tgz/8.0MiB-4 403MB/s ± 2% 1012MB/s ± 4% +150.86% (p=0.000 n=10+9) StreamWriteTo/tgz/16.0MiB-4 408MB/s ± 1% 1069MB/s ± 4% +161.80% (p=0.000 n=9+10) StreamWriteTo/tgz/128.0MiB-4 418MB/s ± 1% 1101MB/s ± 6% +163.57% (p=0.000 n=10+10) ``` Benchmark command: ``` go test -cpu=4 -benchtime=1s -count=10 -bench=. \ ./internal/snapshotstream ``` Results compared using golang.org/x/perf/cmd/benchstat.
1 parent 1c0f258 commit 1b78e56

File tree

3 files changed

+7
-1
lines changed

3 files changed

+7
-1
lines changed

go.mod

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ require (
88
github.com/gorilla/handlers v1.5.1
99
github.com/gorilla/mux v1.8.0
1010
github.com/klauspost/compress v1.15.12
11+
github.com/minio/sha256-simd v1.0.0
1112
github.com/prometheus/client_golang v1.14.0
1213
github.com/prometheus/common v0.37.0
1314
go.uber.org/multierr v1.8.0
@@ -19,6 +20,7 @@ require (
1920
github.com/felixge/httpsnoop v1.0.3 // indirect
2021
github.com/golang/protobuf v1.5.2 // indirect
2122
github.com/json-iterator/go v1.1.12 // indirect
23+
github.com/klauspost/cpuid/v2 v2.0.4 // indirect
2224
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
2325
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
2426
github.com/modern-go/reflect2 v1.0.2 // indirect

go.sum

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,8 @@ github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8
151151
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
152152
github.com/klauspost/compress v1.15.12 h1:YClS/PImqYbn+UILDnqxQCZ3RehC9N318SU3kElDUEM=
153153
github.com/klauspost/compress v1.15.12/go.mod h1:QPwzmACJjUTFsnSHH934V6woptycfrDDJnH7hvFVbGM=
154+
github.com/klauspost/cpuid/v2 v2.0.4 h1:g0I61F2K2DjRHz1cnxlkNSBIaePVoJIjjnHui8QHbiw=
155+
github.com/klauspost/cpuid/v2 v2.0.4/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
154156
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
155157
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
156158
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
@@ -160,6 +162,8 @@ github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
160162
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
161163
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
162164
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
165+
github.com/minio/sha256-simd v1.0.0 h1:v1ta+49hkWZyvaKwrQB8elexRqm6Y0aMLjCNsrYxo6g=
166+
github.com/minio/sha256-simd v1.0.0/go.mod h1:OuYzVNI5vcoYIAmbIvHPl3N3jUzVedXbKy5RFepssQM=
163167
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
164168
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
165169
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=

internal/snapshotstream/stream.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
package snapshotstream
22

33
import (
4-
"crypto/sha256"
54
"encoding/hex"
65
"errors"
76
"fmt"
@@ -15,6 +14,7 @@ import (
1514

1615
"github.com/hansmi/prombackup/api"
1716
"github.com/klauspost/compress/gzip"
17+
"github.com/minio/sha256-simd"
1818
"go.uber.org/multierr"
1919
)
2020

0 commit comments

Comments
 (0)