Tkrzw-RPC
Classes | Public Member Functions | List of all members
tkrzw::RemoteDBM Class Referencefinal

RPC interface to access the database service via gRPC protocol. More...

#include <tkrzw_dbm_remote.h>

Classes

class  Iterator
 Iterator for each record. More...
 
struct  ReplicateLog
 Wrapper of an update log for replication. More...
 
class  Replicator
 Reader for update logs for asynchronous replicatoin. More...
 
class  Stream
 Stream for better performance of intensive operations. More...
 

Public Member Functions

 RemoteDBM ()
 Constructor. More...
 
 ~RemoteDBM ()
 Destructor. More...
 
 RemoteDBM (const RemoteDBM &rhs)=delete
 Copy and assignment are disabled. More...
 
RemoteDBMoperator= (const RemoteDBM &rhs)=delete
 
void InjectStub (void *stub)
 Injects a stub for testing. More...
 
Status Connect (const std::string &address, double timeout=-1, const std::string &auth_config="")
 Connects to the server. More...
 
Status Disconnect ()
 Disconnects the connection to the server. More...
 
Status SetDBMIndex (int32_t dbm_index)
 Sets the index of the DBM to access. More...
 
Status Echo (std::string_view message, std::string *echo)
 Sends a message and gets back the echo message. More...
 
Status Inspect (std::vector< std::pair< std::string, std::string >> *records)
 Inspects a database. More...
 
Status Get (std::string_view key, std::string *value=nullptr)
 Gets the value of a record of a key. More...
 
std::string GetSimple (std::string_view key, std::string_view default_value="")
 Gets the value of a record of a key, in a simple way. More...
 
Status GetMulti (const std::vector< std::string_view > &keys, std::map< std::string, std::string > *records)
 Gets the values of multiple records of keys, with a string view vector. More...
 
Status GetMulti (const std::initializer_list< std::string_view > &keys, std::map< std::string, std::string > *records)
 Gets the values of multiple records of keys, with an initializer list. More...
 
Status GetMulti (const std::vector< std::string > &keys, std::map< std::string, std::string > *records)
 Gets the values of multiple records of keys, with a string vector. More...
 
Status Set (std::string_view key, std::string_view value, bool overwrite=true)
 Sets a record of a key and a value. More...
 
Status SetMulti (const std::map< std::string_view, std::string_view > &records, bool overwrite=true)
 Sets multiple records, with a map of string views. More...
 
Status SetMulti (const std::initializer_list< std::pair< std::string_view, std::string_view >> &records, bool overwrite=true)
 Sets multiple records, with an initializer list. More...
 
Status SetMulti (const std::map< std::string, std::string > &records, bool overwrite=true)
 Sets multiple records, with a map of strings. More...
 
Status Remove (std::string_view key)
 Removes a record of a key. More...
 
Status RemoveMulti (const std::vector< std::string_view > &keys)
 Removes records of keys, with a string view vector. More...
 
Status RemoveMulti (const std::initializer_list< std::string_view > &keys)
 Removes records of keys, with an initializer list. More...
 
Status RemoveMulti (const std::vector< std::string > &keys)
 Removes records of keys, with a string vector. More...
 
Status Append (std::string_view key, std::string_view value, std::string_view delim="")
 Appends data at the end of a record of a key. More...
 
Status AppendMulti (const std::map< std::string_view, std::string_view > &records, std::string_view delim="")
 Appends data to multiple records, with a map of string views. More...
 
Status AppendMulti (const std::initializer_list< std::pair< std::string_view, std::string_view >> &records, std::string_view delim="")
 Appends data to multiple records, with an initializer list. More...
 
Status AppendMulti (const std::map< std::string, std::string > &records, std::string_view delim="")
 Appends data to multiple records, with a map of strings. More...
 
Status CompareExchange (std::string_view key, std::string_view expected, std::string_view desired, std::string *actual=nullptr, bool *found=nullptr, double retry_wait=0, bool notify=false)
 Compares the value of a record and exchanges if the condition meets. More...
 
Status Increment (std::string_view key, int64_t increment=1, int64_t *current=nullptr, int64_t initial=0)
 Increments the numeric value of a record. More...
 
int64_t IncrementSimple (std::string_view key, int64_t increment=1, int64_t initial=0)
 Increments the numeric value of a record, in a simple way. More...
 
Status CompareExchangeMulti (const std::vector< std::pair< std::string_view, std::string_view >> &expected, const std::vector< std::pair< std::string_view, std::string_view >> &desired)
 Compares the values of records and exchanges if the condition meets. More...
 
Status Rekey (std::string_view old_key, std::string_view new_key, bool overwrite=true, bool copying=false)
 Changes the key of a record. More...
 
Status PopFirst (std::string *key=nullptr, std::string *value=nullptr, double retry_wait=0)
 Gets the first record and removes it. More...
 
Status PushLast (std::string_view value, double wtime=-1, bool notify=false)
 Adds a record with a key of the current timestamp. More...
 
Status Count (int64_t *count)
 Gets the number of records. More...
 
int64_t CountSimple ()
 Gets the number of records, in a simple way. More...
 
Status GetFileSize (int64_t *size)
 Gets the current file size of the database. More...
 
int64_t GetFileSizeSimple ()
 Gets the current file size of the database, in a simple way. More...
 
Status Clear ()
 Removes all records. More...
 
Status Rebuild (const std::map< std::string, std::string > &params={})
 Rebuilds the entire database. More...
 
Status ShouldBeRebuilt (bool *tobe)
 Checks whether the database should be rebuilt. More...
 
Status Synchronize (bool hard, const std::map< std::string, std::string > &params={})
 Synchronizes the content of the database to the file system. More...
 
Status Search (std::string_view mode, std::string_view pattern, std::vector< std::string > *matched, size_t capacity=0)
 Searches a database and get keys which match a pattern, according to a mode expression. More...
 
Status ChangeMaster (std::string_view master, double timestamp_skew=0)
 Changes the master server of the replication. More...
 
std::unique_ptr< StreamMakeStream ()
 Makes a stream for intensive operations. More...
 
std::unique_ptr< IteratorMakeIterator ()
 Makes an iterator for each record. More...
 
std::unique_ptr< ReplicatorMakeReplicator ()
 Makes a replicator for asynchronous replication. More...
 

Detailed Description

RPC interface to access the database service via gRPC protocol.

All operations are thread-safe; Multiple threads can access the same connection concurrently. The SetDBMIndex affects all threads so it should be called before the object is shared.

Constructor & Destructor Documentation

◆ RemoteDBM() [1/2]

tkrzw::RemoteDBM::RemoteDBM ( )

Constructor.

◆ ~RemoteDBM()

tkrzw::RemoteDBM::~RemoteDBM ( )

Destructor.

◆ RemoteDBM() [2/2]

tkrzw::RemoteDBM::RemoteDBM ( const RemoteDBM rhs)
explicitdelete

Copy and assignment are disabled.

Member Function Documentation

◆ InjectStub()

void tkrzw::RemoteDBM::InjectStub ( void *  stub)

Injects a stub for testing.

Parameters
stubThe pointer to the DBMService::StubInterface object. The ownership is taken.

This method is used instead of the Connect method. With this, you can inject mock stubs for testing and any kind of stubs with arbitrary auth/network settings.

◆ Connect()

Status tkrzw::RemoteDBM::Connect ( const std::string &  address,
double  timeout = -1,
const std::string &  auth_config = "" 
)

Connects to the server.

Parameters
addressThe address or the host name of the server and its port number. For IPv4 address, it's like "127.0.0.1:1978". For IPv6, it's like "[::1]:1978". For UNIX domain sockets, it's like "unix:/path/to/file".
timeoutThe timeout in seconds for connection and each operation. Negative means unlimited.
auth_configThe authentication configuration. It it is empty, no authentication is done. If it begins with "ssl:", the SSL authentication is done. Key-value parameters in "key=value,key=value,..." format comes next. For SSL, "key", "cert", and "root" parameters specify the paths of the client private key file, the client certificate file, and the root CA certificate file respectively.
Returns
The result status.

◆ Disconnect()

Status tkrzw::RemoteDBM::Disconnect ( )

Disconnects the connection to the server.

Returns
The result status.

◆ SetDBMIndex()

Status tkrzw::RemoteDBM::SetDBMIndex ( int32_t  dbm_index)

Sets the index of the DBM to access.

Parameters
dbm_indexThe index of the DBM to access.
Returns
The result status.

◆ Echo()

Status tkrzw::RemoteDBM::Echo ( std::string_view  message,
std::string *  echo 
)

Sends a message and gets back the echo message.

Parameters
messageThe message to send.
echoThe pointer to a string object to contain the echo message.
Returns
The result status.

◆ Inspect()

Status tkrzw::RemoteDBM::Inspect ( std::vector< std::pair< std::string, std::string >> *  records)

Inspects a database.

Parameters
recordsThe pointer to a map to store retrieved records.
Returns
The result status.

If the DBM index is negative, basic metadata of all DBMs are obtained.

◆ Get()

Status tkrzw::RemoteDBM::Get ( std::string_view  key,
std::string *  value = nullptr 
)

Gets the value of a record of a key.

Parameters
keyThe key of the record.
valueThe pointer to a string object to contain the result value. If it is nullptr, the value data is ignored.
Returns
The result status. If there's no matching record, NOT_FOUND_ERROR is returned.

◆ GetSimple()

std::string tkrzw::RemoteDBM::GetSimple ( std::string_view  key,
std::string_view  default_value = "" 
)

Gets the value of a record of a key, in a simple way.

Parameters
keyThe key of the record.
default_valueThe value to be returned on failure.
Returns
The value of the matching record on success, or the default value on failure.

◆ GetMulti() [1/3]

Status tkrzw::RemoteDBM::GetMulti ( const std::vector< std::string_view > &  keys,
std::map< std::string, std::string > *  records 
)

Gets the values of multiple records of keys, with a string view vector.

Parameters
keysThe keys of records to retrieve.
recordsThe pointer to a map to store retrieved records. Keys which don't match existing records are ignored.
Returns
The result status. If all records of the given keys are found, SUCCESS is returned. If one or more records are missing, NOT_FOUND_ERROR is returned. Thus, even with an error code, the result map can have elements.

◆ GetMulti() [2/3]

Status tkrzw::RemoteDBM::GetMulti ( const std::initializer_list< std::string_view > &  keys,
std::map< std::string, std::string > *  records 
)

Gets the values of multiple records of keys, with an initializer list.

Parameters
keysThe keys of records to retrieve.
recordsThe pointer to a map to store retrieved records. Keys which don't match existing records are ignored.
Returns
The result status. If all records of the given keys are found, SUCCESS is returned. If one or more records are missing, NOT_FOUND_ERROR is returned. Thus, even with an error code, the result map can have elements.

◆ GetMulti() [3/3]

Status tkrzw::RemoteDBM::GetMulti ( const std::vector< std::string > &  keys,
std::map< std::string, std::string > *  records 
)

Gets the values of multiple records of keys, with a string vector.

Parameters
keysThe keys of records to retrieve.
recordsThe pointer to a map to store retrieved records. Keys which don't match existing records are ignored.
Returns
The result status. If all records of the given keys are found, SUCCESS is returned. If one or more records are missing, NOT_FOUND_ERROR is returned. Thus, even with an error code, the result map can have elements.

◆ Set()

Status tkrzw::RemoteDBM::Set ( std::string_view  key,
std::string_view  value,
bool  overwrite = true 
)

Sets a record of a key and a value.

Parameters
keyThe key of the record.
valueThe value of the record.
overwriteWhether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
Returns
The result status. If overwriting is abandoned, DUPLICATION_ERROR is returned.

◆ SetMulti() [1/3]

Status tkrzw::RemoteDBM::SetMulti ( const std::map< std::string_view, std::string_view > &  records,
bool  overwrite = true 
)

Sets multiple records, with a map of string views.

Parameters
recordsThe records to store.
overwriteWhether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
Returns
The result status. If there are records avoiding overwriting, DUPLICATION_ERROR is returned.

◆ SetMulti() [2/3]

Status tkrzw::RemoteDBM::SetMulti ( const std::initializer_list< std::pair< std::string_view, std::string_view >> &  records,
bool  overwrite = true 
)

Sets multiple records, with an initializer list.

Parameters
recordsThe records to store.
overwriteWhether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
Returns
The result status. If there are records avoiding overwriting, DUPLICATION_ERROR is returned.

◆ SetMulti() [3/3]

Status tkrzw::RemoteDBM::SetMulti ( const std::map< std::string, std::string > &  records,
bool  overwrite = true 
)

Sets multiple records, with a map of strings.

Parameters
recordsThe records to store.
overwriteWhether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
Returns
The result status. If there are records avoiding overwriting, DUPLICATION_ERROR is returned.

◆ Remove()

Status tkrzw::RemoteDBM::Remove ( std::string_view  key)

Removes a record of a key.

Parameters
keyThe key of the record.
Returns
The result status. If there's no matching record, NOT_FOUND_ERROR is returned.

◆ RemoveMulti() [1/3]

Status tkrzw::RemoteDBM::RemoveMulti ( const std::vector< std::string_view > &  keys)

Removes records of keys, with a string view vector.

Parameters
keysThe keys of records to remove.
Returns
The result status. If there are missing records, NOT_FOUND_ERROR is returned.

◆ RemoveMulti() [2/3]

Status tkrzw::RemoteDBM::RemoveMulti ( const std::initializer_list< std::string_view > &  keys)

Removes records of keys, with an initializer list.

Parameters
keysThe keys of records to remove.
Returns
The result status.

◆ RemoveMulti() [3/3]

Status tkrzw::RemoteDBM::RemoveMulti ( const std::vector< std::string > &  keys)

Removes records of keys, with a string vector.

Parameters
keysThe keys of records to remove.
Returns
The result status. If there are missing records, NOT_FOUND_ERROR is returned.

◆ Append()

Status tkrzw::RemoteDBM::Append ( std::string_view  key,
std::string_view  value,
std::string_view  delim = "" 
)

Appends data at the end of a record of a key.

Parameters
keyThe key of the record.
valueThe value to append.
delimThe delimiter to put after the existing record.
Returns
The result status.

If there's no existing record, the value is set without the delimiter.

◆ AppendMulti() [1/3]

Status tkrzw::RemoteDBM::AppendMulti ( const std::map< std::string_view, std::string_view > &  records,
std::string_view  delim = "" 
)

Appends data to multiple records, with a map of string views.

Parameters
recordsThe records to append.
delimThe delimiter to put after the existing record.
Returns
The result status.

If there's no existing record, the value is set without the delimiter.

◆ AppendMulti() [2/3]

Status tkrzw::RemoteDBM::AppendMulti ( const std::initializer_list< std::pair< std::string_view, std::string_view >> &  records,
std::string_view  delim = "" 
)

Appends data to multiple records, with an initializer list.

Parameters
recordsThe records to store.
delimThe delimiter to put after the existing record.
Returns
The result status.

If there's no existing record, the value is set without the delimiter.

◆ AppendMulti() [3/3]

Status tkrzw::RemoteDBM::AppendMulti ( const std::map< std::string, std::string > &  records,
std::string_view  delim = "" 
)

Appends data to multiple records, with a map of strings.

Parameters
recordsThe records to append.
delimThe delimiter to put after the existing record.
Returns
The result status.

If there's no existing record, the value is set without the delimiter.

◆ CompareExchange()

Status tkrzw::RemoteDBM::CompareExchange ( std::string_view  key,
std::string_view  expected,
std::string_view  desired,
std::string *  actual = nullptr,
bool *  found = nullptr,
double  retry_wait = 0,
bool  notify = false 
)

Compares the value of a record and exchanges if the condition meets.

Parameters
keyThe key of the record.
expectedThe expected value. If the data is nullptr, no existing record is expected. If it is DBM::ANY_DATA, an existing record with any value is expacted.
desiredThe desired value. If the data is nullptr, the record is to be removed. If it is DBM::ANY_DATA, no update is done.
actualThe pointer to a string object to contain the actual value of the existing record. If it is nullptr, it is ignored.
foundThe pointer to a variable to contain whether there is an existing record. If it is nullptr, it is ignored.
retry_waitThe maximum wait time in seconds before retrying. If it is zero, no retry is done. If it is positive, retry is done after waiting for the notifications of the next update for the time at most.
notifyIf true, a notification signal is sent to wake up retrying threads.
Returns
The result status. If the condition doesn't meet, INFEASIBLE_ERROR is returned.

◆ Increment()

Status tkrzw::RemoteDBM::Increment ( std::string_view  key,
int64_t  increment = 1,
int64_t *  current = nullptr,
int64_t  initial = 0 
)

Increments the numeric value of a record.

Parameters
keyThe key of the record.
incrementThe incremental value. If it is INT64MIN, the current value is not changed and a new record is not created.
currentThe pointer to an integer to contain the current value. If it is nullptr, it is ignored.
initialThe initial value.
Returns
The result status.

The record value is stored as an 8-byte big-endian integer. Negative is also supported.

◆ IncrementSimple()

int64_t tkrzw::RemoteDBM::IncrementSimple ( std::string_view  key,
int64_t  increment = 1,
int64_t  initial = 0 
)

Increments the numeric value of a record, in a simple way.

Parameters
keyThe key of the record.
incrementThe incremental value.
initialThe initial value.
Returns
The current value or INT64MIN on failure.

The record value is treated as a decimal integer. Negative is also supported.

◆ CompareExchangeMulti()

Status tkrzw::RemoteDBM::CompareExchangeMulti ( const std::vector< std::pair< std::string_view, std::string_view >> &  expected,
const std::vector< std::pair< std::string_view, std::string_view >> &  desired 
)

Compares the values of records and exchanges if the condition meets.

Parameters
expectedThe record keys and their expected values. If the value is nullptr, no existing record is expected. If the value is DBM::ANY_DATA, an existing record with any value is expacted.
desiredThe record keys and their desired values. If the value is nullptr, the record is to be removed.
Returns
The result status. If the condition doesn't meet, INFEASIBLE_ERROR is returned.

◆ Rekey()

Status tkrzw::RemoteDBM::Rekey ( std::string_view  old_key,
std::string_view  new_key,
bool  overwrite = true,
bool  copying = false 
)

Changes the key of a record.

Parameters
old_keyThe old key of the record.
new_keyThe new key of the record.
overwriteWhether to overwrite the existing record of the new key.
copyingWhether to retain the record of the old key.
Returns
The result status. If there's no matching record to the old key, NOT_FOUND_ERROR is returned. If the overwrite flag is false and there is an existing record of the new key, DUPLICATION ERROR is returned.

◆ PopFirst()

Status tkrzw::RemoteDBM::PopFirst ( std::string *  key = nullptr,
std::string *  value = nullptr,
double  retry_wait = 0 
)

Gets the first record and removes it.

Parameters
keyThe pointer to a string object to contain the key of the first record. If it is nullptr, it is ignored.
valueThe pointer to a string object to contain the value of the first record. If it is nullptr, it is ignored.
retry_waitThe maximum wait time in seconds before retrying. If it is zero, no retry is done. If it is positive, retry is done after waiting for the notifications of the next update for the time at most.
Returns
The result status.

◆ PushLast()

Status tkrzw::RemoteDBM::PushLast ( std::string_view  value,
double  wtime = -1,
bool  notify = false 
)

Adds a record with a key of the current timestamp.

Parameters
valueThe value of the record.
wtimeThe current wall time used to generate the key. If it is negative, the system clock is used.
notifyIf true, notification signal is sent.
Returns
The result status.

The key is generated as an 8-bite big-endian binary string of the timestamp. If there is an existing record matching the generated key, the key is regenerated and the attempt is repeated until it succeeds.

◆ Count()

Status tkrzw::RemoteDBM::Count ( int64_t *  count)

Gets the number of records.

Parameters
countThe pointer to an integer object to contain the result count.
Returns
The result status.

◆ CountSimple()

int64_t tkrzw::RemoteDBM::CountSimple ( )

Gets the number of records, in a simple way.

Returns
The number of records on success, or -1 on failure.

◆ GetFileSize()

Status tkrzw::RemoteDBM::GetFileSize ( int64_t *  size)

Gets the current file size of the database.

Parameters
sizeThe pointer to an integer object to contain the result size.
Returns
The result status.

◆ GetFileSizeSimple()

int64_t tkrzw::RemoteDBM::GetFileSizeSimple ( )

Gets the current file size of the database, in a simple way.

Returns
The current file size of the database, or -1 on failure.

◆ Clear()

Status tkrzw::RemoteDBM::Clear ( )

Removes all records.

Returns
The result status.

◆ Rebuild()

Status tkrzw::RemoteDBM::Rebuild ( const std::map< std::string, std::string > &  params = {})

Rebuilds the entire database.

Parameters
paramsOptional parameters.
Returns
The result status.

Tuning options can be given by the optional parameters, as with the Open method of the PolyDBM class and the database configurations of the server command. A unset parameter means that the current setting is succeeded or calculated implicitly.

In addition, HashDBM, TreeDBM, and SkipDBM supports the following parameters.

  • skip_broken_records (bool): If true, the operation continues even if there are broken records which can be skipped.
  • sync_hard (bool): If true, physical synchronization with the hardware is done before finishing the rebuilt file.

◆ ShouldBeRebuilt()

Status tkrzw::RemoteDBM::ShouldBeRebuilt ( bool *  tobe)

Checks whether the database should be rebuilt.

Parameters
tobeThe pointer to a boolean object to contain the result decision.
Returns
The result status.

◆ Synchronize()

Status tkrzw::RemoteDBM::Synchronize ( bool  hard,
const std::map< std::string, std::string > &  params = {} 
)

Synchronizes the content of the database to the file system.

Parameters
hardTrue to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
paramsOptional parameters.
Returns
The result status.

The "reducer" parameter specifies the reducer for SkipDBM. "ReduceToFirst", "ReduceToSecond", "ReduceToLast", etc are supported. If the parameter "make_backup" exists, a backup file is created in the same directory as the database file. The backup file name has a date suffix in GMT, like ".backup.20210831213749". If the value of "make_backup" not empty, it is the value is used as the suffix.

◆ Search()

Status tkrzw::RemoteDBM::Search ( std::string_view  mode,
std::string_view  pattern,
std::vector< std::string > *  matched,
size_t  capacity = 0 
)

Searches a database and get keys which match a pattern, according to a mode expression.

Parameters
modeThe search mode. "contain" extracts keys containing the pattern. "begin" extracts keys beginning with the pattern. "end" extracts keys ending with the pattern. "regex" extracts keys partially matches the pattern of a regular expression. "edit" extracts keys whose edit distance to the UTF-8 pattern is the least. "editbin" extracts keys whose edit distance to the binary pattern is the least. Ordered databases support "upper" and "lower" which extract keys whose positions are upper/lower than the pattern. "upperinc" and "lowerinc" are their inclusive versions.
patternThe pattern for matching.
matchedA vector to contain the result.
capacityThe maximum records to obtain. 0 means unlimited.
Returns
The result status.

◆ ChangeMaster()

Status tkrzw::RemoteDBM::ChangeMaster ( std::string_view  master,
double  timestamp_skew = 0 
)

Changes the master server of the replication.

Parameters
masterThe address of the master server. If it is empty, replication stops.
timestamp_skewThe timestamp skew in milliseconds. Negative makes the timestamp back to the past.

◆ MakeStream()

std::unique_ptr<Stream> tkrzw::RemoteDBM::MakeStream ( )

Makes a stream for intensive operations.

Returns
The stream for intensive operations.

◆ MakeIterator()

std::unique_ptr<Iterator> tkrzw::RemoteDBM::MakeIterator ( )

Makes an iterator for each record.

Returns
The iterator for each record.

◆ MakeReplicator()

std::unique_ptr<Replicator> tkrzw::RemoteDBM::MakeReplicator ( )

Makes a replicator for asynchronous replication.

Returns
The replicator for asynchronous replication.