Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for zstd multi-threading #17

Open
berlincount opened this issue Jun 5, 2017 · 3 comments
Open

Add support for zstd multi-threading #17

berlincount opened this issue Jun 5, 2017 · 3 comments

Comments

@berlincount
Copy link

zstd improved the multi-threading code present in pzstd in 1.1.0 and moved it up to the main library. I'd love to be able to use multiple threads - at least when reading zstd compressed streams.

Thanks in advance!

What version of Go are you using (go version)?

go version go1.8.1 darwin/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/count"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.8.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.8.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/qx/x2xsgzgx5y90lmwwkhs3w91r0000gn/T/go-build691925268=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

@jcftang
Copy link

jcftang commented Sep 18, 2017

Actually having thread reads and writes would be nice, I'm easily hitting the limits of a single thread where my datasets are in the 10's of gigabytes in size

@Viq111 Viq111 changed the title Please add support for zstd 1.2.0 including multi-threading Add support for zstd multi-threading Oct 1, 2018
@bsergean
Copy link
Contributor

bsergean commented Jun 6, 2022

Head up that is being looked into in #117

@bsergean
Copy link
Contributor

bsergean commented Jun 6, 2022

Just for the writer side of thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants