Closed
Description
Here is my code, I am trying to save some sst files and read with rocksdb in python:
from rocksdict import Rdict, Options, SstFileWriter
import random
import rocksdb
# generate some rand bytes
rand_bytes1 = [random.randbytes(200) for _ in range(100000)]
rand_bytes1.sort()
# write to file1.sst
writer = SstFileWriter(options=Options(raw_mode=True))
writer.open("file1.sst")
for k, v in zip(rand_bytes1, rand_bytes1):
writer[k] = v
writer.finish()
# bulid Rdict
d = Rdict("tmp", options=Options(raw_mode=True))
d.ingest_external_file(["file1.sst"])
d.close()
# rocksdb read
db = rocksdb.DB("tmp", read_only=True, opts=rocksdb.Options())And I am getting the error as below:
---------------------------------------------------------------------------
Corruption Traceback (most recent call last)
remote%2B10.97.150.93/opt/huawei/data1/z00811450/code/test.ipynb#Y211sdnNjb2RlLXJlbW90ZQ%3D%3D?line=22) db = rocksdb.DB("tmp", read_only=True, opts=rocksdb.Options())
File /opt/huawei/wisebrain/anaconda3/lib/python3.9/site-packages/rocksdb/_rocksdb.pyx:1636, in rocksdb._rocksdb.DB.__cinit__()
File /opt/huawei/wisebrain/anaconda3/lib/python3.9/site-packages/rocksdb/_rocksdb.pyx:76, in rocksdb._rocksdb.check_status()
Corruption: b'Corruption: unknown checksum type 4 from footer of tmp/000019.sst, while checking block at offset 22672231 size 35'Does this mean that rocksdict is not compatible with rocksdb in python? Looking forward to anyone's help!
Metadata
Metadata
Assignees
Labels
No labels
Activity
Congyuwang commentedon Apr 18, 2023
rocksdict uses rocksdb 8.0 for the moment. So it maybe incompatible to rocksdb 6 which is used by python-rocksdb
Congyuwang commentedon Apr 18, 2023
But it should be compatible with other rocksdb clients, e.g., java, c++, rust