Tkrzw
|
Sharding database manager adapter. More...
#include <tkrzw_dbm_shard.h>
Classes | |
class | Iterator |
Iterator for each record. More... | |
Public Member Functions | |
ShardDBM () | |
Default constructor. More... | |
virtual | ~ShardDBM () |
Destructor. More... | |
ShardDBM (const ShardDBM &rhs)=delete | |
Copy and assignment are disabled. More... | |
ShardDBM & | operator= (const ShardDBM &rhs)=delete |
Status | Open (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT) override |
Opens a database file. More... | |
Status | OpenAdvanced (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT, const std::map< std::string, std::string > ¶ms={}) override |
Opens a database file, in an advanced way. More... | |
Status | Close () override |
Closes the database file. More... | |
Status | Process (std::string_view key, RecordProcessor *proc, bool writable) override |
Processes a record with a processor. More... | |
Status | Get (std::string_view key, std::string *value=nullptr) override |
Gets the value of a record of a key. More... | |
Status | Set (std::string_view key, std::string_view value, bool overwrite=true, std::string *old_value=nullptr) override |
Sets a record of a key and a value. More... | |
Status | Remove (std::string_view key, std::string *old_value=nullptr) override |
Removes a record of a key. More... | |
Status | Append (std::string_view key, std::string_view value, std::string_view delim="") override |
Appends data at the end of a record of a key. More... | |
Status | ProcessFirst (RecordProcessor *proc, bool writable) override |
Processes the first record with a processor. More... | |
Status | ProcessMulti (const std::vector< std::pair< std::string_view, DBM::RecordProcessor * >> &key_proc_pairs, bool writable) override |
Processes multiple records with processors. More... | |
Status | CompareExchangeMulti (const std::vector< std::pair< std::string_view, std::string_view >> &expected, const std::vector< std::pair< std::string_view, std::string_view >> &desired) override |
Compares the values of records and exchanges if the condition meets. More... | |
Status | ProcessEach (RecordProcessor *proc, bool writable) override |
Processes each and every record in the database with a processor. More... | |
Status | Count (int64_t *count) override |
Gets the number of records. More... | |
Status | GetFileSize (int64_t *size) override |
Gets the current file size of the database. More... | |
Status | GetFilePath (std::string *path) override |
Gets the path of the database file. More... | |
Status | GetTimestamp (double *timestamp) override |
Gets the timestamp in seconds of the last modified time. More... | |
Status | Clear () override |
Removes all records. More... | |
Status | Rebuild () override |
Rebuilds the entire database. More... | |
Status | RebuildAdvanced (const std::map< std::string, std::string > ¶ms={}) override |
Rebuilds the entire database, in an advanced way. More... | |
Status | ShouldBeRebuilt (bool *tobe) override |
Checks whether the database should be rebuilt. More... | |
Status | Synchronize (bool hard, FileProcessor *proc=nullptr) override |
Synchronizes the content of the database to the file system. More... | |
Status | SynchronizeAdvanced (bool hard, FileProcessor *proc=nullptr, const std::map< std::string, std::string > ¶ms={}) override |
Synchronizes the content of the database to the file system, in an advanced way. More... | |
Status | CopyFileData (const std::string &dest_path, bool sync_hard=false) override |
Copies the content of the database files to other files. More... | |
std::vector< std::pair< std::string, std::string > > | Inspect () override |
Inspects the database. More... | |
bool | IsOpen () const override |
Checks whether the database is open. More... | |
bool | IsWritable () const override |
Checks whether the database is writable. More... | |
bool | IsHealthy () const override |
Checks whether the database condition is healthy. More... | |
bool | IsOrdered () const override |
Checks whether ordered operations are supported. More... | |
std::unique_ptr< DBM::Iterator > | MakeIterator () override |
Makes an iterator for each record. More... | |
std::unique_ptr< DBM > | MakeDBM () const override |
Makes a new DBM object of the same concrete class. More... | |
UpdateLogger * | GetUpdateLogger () const override |
Gets the logger to write all update operations. More... | |
void | SetUpdateLogger (UpdateLogger *update_logger) override |
Sets the logger to write all update operations. More... | |
DBM * | GetInternalDBM () const |
Gets the pointer to the first internal database object. More... | |
![]() | |
virtual | ~ParamDBM ()=default |
Destructor. More... | |
![]() | |
virtual | ~DBM ()=default |
Destructor. More... | |
virtual Status | Process (std::string_view key, RecordLambdaType rec_lambda, bool writable) |
Processes a record with a lambda function. More... | |
virtual std::string | GetSimple (std::string_view key, std::string_view default_value="") |
Gets the value of a record of a key, in a simple way. More... | |
virtual Status | GetMulti (const std::vector< std::string_view > &keys, std::map< std::string, std::string > *records) |
Gets the values of multiple records of keys, with a string view vector. More... | |
virtual Status | GetMulti (const std::initializer_list< std::string_view > &keys, std::map< std::string, std::string > *records) |
Gets the values of multiple records of keys, with an initializer list. More... | |
virtual Status | GetMulti (const std::vector< std::string > &keys, std::map< std::string, std::string > *records) |
Gets the values of multiple records of keys, with a string vector. More... | |
virtual Status | SetMulti (const std::map< std::string_view, std::string_view > &records, bool overwrite=true) |
Sets multiple records, with a map of string views. More... | |
virtual Status | SetMulti (const std::initializer_list< std::pair< std::string_view, std::string_view >> &records, bool overwrite=true) |
Sets multiple records, with an initializer list. More... | |
virtual Status | SetMulti (const std::map< std::string, std::string > &records, bool overwrite=true) |
Sets multiple records, with a map of strings. More... | |
virtual Status | RemoveMulti (const std::vector< std::string_view > &keys) |
Removes records of keys, with a string view vector. More... | |
virtual Status | RemoveMulti (const std::initializer_list< std::string_view > &keys) |
Removes records of keys, with an initializer list. More... | |
virtual Status | RemoveMulti (const std::vector< std::string > &keys) |
Removes records of keys, with a string vector. More... | |
virtual Status | AppendMulti (const std::map< std::string_view, std::string_view > &records, std::string_view delim="") |
Appends data to multiple records, with a map of string views. More... | |
virtual Status | AppendMulti (const std::initializer_list< std::pair< std::string_view, std::string_view >> &records, std::string_view delim="") |
Appends data to multiple records, with an initializer list. More... | |
virtual Status | AppendMulti (const std::map< std::string, std::string > &records, std::string_view delim="") |
Appends data to multiple records, with a map of strings. More... | |
virtual Status | CompareExchange (std::string_view key, std::string_view expected, std::string_view desired, std::string *actual=nullptr, bool *found=nullptr) |
Compares the value of a record and exchanges if the condition meets. More... | |
virtual Status | Increment (std::string_view key, int64_t increment=1, int64_t *current=nullptr, int64_t initial=0) |
Increments the numeric value of a record. More... | |
virtual int64_t | IncrementSimple (std::string_view key, int64_t increment=1, int64_t initial=0) |
Increments the numeric value of a record, in a simple way. More... | |
virtual Status | ProcessMulti (const std::vector< std::pair< std::string_view, RecordProcessor * >> &key_proc_pairs, bool writable)=0 |
Processes multiple records with processors. More... | |
virtual Status | ProcessMulti (const std::vector< std::pair< std::string_view, RecordLambdaType >> &key_lambda_pairs, bool writable) |
Processes multiple records with lambda functions. More... | |
virtual Status | Rekey (std::string_view old_key, std::string_view new_key, bool overwrite=true, bool copying=false, std::string *value=nullptr) |
Changes the key of a record. More... | |
virtual Status | ProcessFirst (RecordLambdaType rec_lambda, bool writable) |
Processes the first record with a lambda function. More... | |
virtual Status | PopFirst (std::string *key=nullptr, std::string *value=nullptr) |
Gets the first record and removes it. More... | |
virtual Status | PushLast (std::string_view value, double wtime=-1, std::string *key=nullptr) |
Adds a record with a key of the current timestamp. More... | |
virtual Status | ProcessEach (RecordLambdaType rec_lambda, bool writable) |
Processes each and every record in the database with a lambda function. More... | |
virtual int64_t | CountSimple () |
Gets the number of records, in a simple way. More... | |
virtual int64_t | GetFileSizeSimple () |
Gets the current file size of the database, in a simple way. More... | |
virtual std::string | GetFilePathSimple () |
Gets the path of the database file, in a simple way. More... | |
virtual double | GetTimestampSimple () |
Gets the timestamp of the last modified time, in a simple way. More... | |
virtual bool | ShouldBeRebuiltSimple () |
Checks whether the database should be rebuilt, in a simple way. More... | |
virtual Status | Export (DBM *dest_dbm) |
Exports all records to another database. More... | |
const std::type_info & | GetType () const |
Gets the type information of the actual class. More... | |
Static Public Member Functions | |
static Status | GetNumberOfShards (const std::string &path, int32_t *num_shards) |
Gets the number of shards of a database. More... | |
static Status | RestoreDatabase (const std::string &old_file_path, const std::string &new_file_path, const std::string &class_name="", int64_t end_offset=-1, std::string_view cipher_key="") |
Restores a broken database as a new healthy database. More... | |
static Status | RenameDatabase (const std::string &old_file_path, const std::string &new_file_path) |
Renames all files of a database to ones of another name. More... | |
Additional Inherited Members | |
![]() | |
typedef std::function< std::string_view(std::string_view, std::string_view)> | RecordLambdaType |
Lambda function type to process a record. More... | |
![]() | |
static const std::string_view | ANY_DATA |
The special string_view value to represent any data. More... | |
Sharding database manager adapter.
All operations except for Open and Close are thread-safe; Multiple threads can access the same database concurrently. Every opened database must be closed explicitly to avoid data corruption.
This class is a wrapper of PolyDBM for sharding the database into multiple instances.
|
explicit |
Default constructor.
|
virtual |
Destructor.
|
explicitdelete |
Copy and assignment are disabled.
|
overridevirtual |
Opens a database file.
path | A path of the file. |
writable | If true, the file is writable. If false, it is read-only. |
options | Bit-sum options of File::OpenOption enums for opening the file. |
Implements tkrzw::DBM.
|
overridevirtual |
Opens a database file, in an advanced way.
path | A path of the file. |
writable | If true, the file is writable. If false, it is read-only. |
options | Bit-sum options for opening the file. |
params | Optional parameters. All parameters for PolyDBM::OpenAdvanced are supported. Moreover, the parameter "num_shards" specifies the number of shards. Each shard file has a suffix like "-00003-of-00015". If the number of shards is not specified and existing files match the path, it is implicitly specified. If there are no matching files, 1 is implicitly set. |
Implements tkrzw::ParamDBM.
|
overridevirtual |
|
overridevirtual |
Processes a record with a processor.
key | The key of the record. |
proc | The pointer to the processor object. |
writable | True if the processor can edit the record. |
If the specified record exists, the ProcessFull of the processor is called. Otherwise, the ProcessEmpty of the processor is called.
Implements tkrzw::DBM.
|
overridevirtual |
Gets the value of a record of a key.
key | The key of the record. |
value | The pointer to a string object to contain the result value. If it is nullptr, the value data is ignored. |
Reimplemented from tkrzw::DBM.
|
overridevirtual |
Sets a record of a key and a value.
key | The key of the record. |
value | The value of the record. |
overwrite | Whether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned. |
old_value | The pointer to a string object to contain the old value. Assignment is done even on the duplication error. If it is nullptr, it is ignored. |
Reimplemented from tkrzw::DBM.
|
overridevirtual |
Removes a record of a key.
key | The key of the record. |
old_value | The pointer to a string object to contain the old value. If it is nullptr, it is ignored. |
Reimplemented from tkrzw::DBM.
|
overridevirtual |
Appends data at the end of a record of a key.
key | The key of the record. |
value | The value to append. |
delim | The delimiter to put after the existing record. |
If there's no existing record, the value is set without the delimiter.
Reimplemented from tkrzw::DBM.
|
overridevirtual |
Processes the first record with a processor.
proc | The pointer to the processor object. |
writable | True if the processor can edit the record. |
If the first record exists, the ProcessFull of the processor is called. Otherwise, this method fails and no method of the processor is called. Whereas ordered databases have efficient implementations of this method, unordered databases have inefficient implementations.
Implements tkrzw::DBM.
|
override |
Processes multiple records with processors.
key_proc_pairs | Pairs of the keys and their processor objects. |
writable | True if the processors can edit the records. |
Precondition: The database is opened. The writable parameter should be consistent to the open mode.
If the specified record exists, the ProcessFull of the processor is called. Otherwise, the ProcessEmpty of the processor is called.
ShardDBM has a special implementation to assure isolation of transactions in multi-threading environments. The given keys are locked in the space of the first shard so that isolation is assured among ProcessMulti even if actual keys belongs to different shards. In other words, other methods such as Get, Set, and Remove are executed without exclusive control.
|
overridevirtual |
Compares the values of records and exchanges if the condition meets.
expected | The record keys and their expected values. If the value is nullptr, no existing record is expected. If the value is ANY_DATA, an existing record with any value is expacted. |
desired | The record keys and their desired values. If the value is nullptr, the record is to be removed. |
ShardDBM has a special implementation to assure isolation of transactions in multi-threading environments. In contrast to the ProcessMulti method, isolation of CompareExchangeMulti is assured among all kinds of methods.
Reimplemented from tkrzw::DBM.
|
overridevirtual |
Processes each and every record in the database with a processor.
proc | The pointer to the processor object. |
writable | True if the processor can edit the record. |
The ProcessFull of the processor is called repeatedly for each record. The ProcessEmpty of the processor is called once before the iteration and once after the iteration.
Implements tkrzw::DBM.
|
overridevirtual |
Gets the number of records.
count | The pointer to an integer object to contain the result count. |
Implements tkrzw::DBM.
|
overridevirtual |
Gets the current file size of the database.
size | The pointer to an integer object to contain the result size. |
Implements tkrzw::DBM.
|
overridevirtual |
Gets the path of the database file.
path | The pointer to a string object to contain the result path. |
Implements tkrzw::DBM.
|
overridevirtual |
Gets the timestamp in seconds of the last modified time.
timestamp | The pointer to a double object to contain the timestamp. |
Implements tkrzw::DBM.
|
overridevirtual |
|
overridevirtual |
|
overridevirtual |
Rebuilds the entire database, in an advanced way.
params | Optional parameters. |
The parameters work in the same way as with PolyDBM::RebuildAdvanced.
Implements tkrzw::ParamDBM.
|
overridevirtual |
Checks whether the database should be rebuilt.
tobe | The pointer to a boolean object to contain the result decision. |
Implements tkrzw::DBM.
|
overridevirtual |
Synchronizes the content of the database to the file system.
hard | True to do physical synchronization with the hardware or false to do only logical synchronization with the file system. |
proc | The pointer to the file processor object, whose Process method is called while the content of the file is synchronized. If it is nullptr, it is ignored. |
Implements tkrzw::DBM.
|
overridevirtual |
Synchronizes the content of the database to the file system, in an advanced way.
hard | True to do physical synchronization with the hardware or false to do only logical synchronization with the file system. |
proc | The pointer to the file processor object, whose Process method is called while the content of the file is synchronized. If it is nullptr, it is ignored. |
params | Optional parameters. |
The parameters work in the same way as with PolyDBM::OpenAdvanced.
Implements tkrzw::ParamDBM.
|
overridevirtual |
Copies the content of the database files to other files.
dest_path | A path prefix to the destination files. |
sync_hard | True to do physical synchronization with the hardware. |
Copying is done while the content is synchronized and stable. So, this method is suitable for making a backup file while running a database service. Each shard file is copied and the destination file also has the same suffix.
Reimplemented from tkrzw::DBM.
|
overridevirtual |
Inspects the database.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether the database is open.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether the database is writable.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether the database condition is healthy.
Implements tkrzw::DBM.
|
overridevirtual |
Checks whether ordered operations are supported.
Implements tkrzw::DBM.
|
overridevirtual |
|
overridevirtual |
Makes a new DBM object of the same concrete class.
Implements tkrzw::DBM.
|
overridevirtual |
Gets the logger to write all update operations.
Implements tkrzw::DBM.
|
overridevirtual |
Sets the logger to write all update operations.
update_logger | The pointer to the update logger object. Ownership is not taken. If it is nullptr, no logger is used. |
Implements tkrzw::DBM.
DBM* tkrzw::ShardDBM::GetInternalDBM | ( | ) | const |
Gets the pointer to the first internal database object.
|
static |
Gets the number of shards of a database.
path | The database path. |
num_shards | The pointer to a variable to store the result. |
If there are multiple candidates, this function report a failure.
|
static |
Restores a broken database as a new healthy database.
old_file_path | The path of the broken database. |
new_file_path | The path of the new database to be created. |
class_name | The name of the database class. If it is empty, the class is guessed from the file extension. |
end_offset | The exclusive end offset of records to read. Negative means unlimited. 0 means the size when the database is synched or closed properly. Using a positive value is not meaningful if the number of shards is more than one. |
cipher_key | The encryption key for cipher compressors. |
|
static |
Renames all files of a database to ones of another name.
old_file_path | The path of the database. |
new_file_path | The path of the new database. |