One writer, many readers #110

Activity

1. Congyuwang commented on Feb 28, 2024

   

   Congyuwang

   on Feb 28, 2024

   Collaborator

   More actions

   Hi, I found the following from rocksdb doc:

   RocksDB database can be opened in read-write mode (aka. Primary Instance) or can be opened in read-only mode. RocksDB supports two variations of read-only mode:

   Read-only Instance - Opens the database in read-only mode. When the Read-only instance is created, it gets a static read-only view of the Primary Instance’s database contents
   Secondary Instance – Opens the database in read-only mode. Supports extra ability to dynamically catch-up with the Primary instance (through a manual call by the user – based on their delay/frequency requirements)

   https://github.com/facebook/rocksdb/wiki/Read-only-and-Secondary-instances

   So, the read only mode gets a static view. You should use the secondary instance.
2. Congyuwang commented on Feb 28, 2024

   

   Congyuwang

   on Feb 28, 2024

   Collaborator

   More actions

   Example from rocksdb doc:

   const std::string kDbPath = "/tmp/rocksdbtest";
   ...
   // Assume we have already opened a regular RocksDB instance db_primary
   // whose database directory is kDbPath.
   assert(db_primary);
   
   Options options;
   options.max_open_files = -1;
   
   // Secondary instance needs its own directory to store info logs (LOG)
   const std::string kSecondaryPath = "/tmp/rocksdb_secondary/";
   DB* db_secondary = nullptr;
   
   Status s = DB::OpenAsSecondary(options, kDbPath, kSecondaryPath, &db_secondary);
   assert(!s.ok() || db_secondary);
   
   // Let secondary **try** to catch up with primary
   s = db_secondary->TryCatchUpWithPrimary();
   assert(s.ok());
   
   // Read operations
   std::string value;
   s = db_secondary->Get(ReadOptions(), "foo", &value);
   ...

   for Python code use

   opt = Options(raw_mode=True)
   opt.set_max_open_files(-1)
   db = Rdict(db_path, options=opt, access_type=AccessType.Secondary(some_path_for_secondary_log))
   
   # need to catch up at some frequency manually by calling:
   db.try_catch_up_with_primary()
3. WhattaSkilL commented on Feb 28, 2024

   

   WhattaSkilL

   on Feb 28, 2024

   Last edited by WhattaSkilL

   Author

   More actions

   Yah, thanks you!
   I saw that but I didn't understand by

   If the writes on Primary Instance does not have WAL enabled (WriteOptions.disableWAL == true), the Read-only/Secondary
    Instances will not have visibility of data residing in Primary’s memtables – resulting in partial view of the database.
   

   and try to enable wal, or try to undestand why not working good
   try_catch_up_with_primary is 7 times better than re-open for cpu perfomance, but thats not free too use any way.
   When i use reopen, i check rocks directory modify time. Maybe better way to check changes in primary?
4. Congyuwang commented on Feb 29, 2024

   

   Congyuwang

   on Feb 29, 2024

   Last edited by Congyuwang

   Collaborator

   More actions

   So, for rocksdb, part of the data is written to those SST files, and part of them are not flushed (stored in memtable), but those data are in the WAL log. So, since the secondary or the readonly instance is another process, it does not have access to the memtable of the primary instance (which is in memory). So, it loads its view from SST files + WAL files. If WAL is not enabled, the view can be more partial than if it is enabled.
5. rocksdict

   locked and limited conversation to collaborators on Feb 29, 2024
6. Congyuwang

   converted this issue into a discussion #111 on Feb 29, 2024