tkrzw_rpc package¶

Submodules¶

tkrzw_rpc.tkrzw_rpc_pb2 module¶

tkrzw_rpc.tkrzw_rpc_pb2_grpc module¶

class tkrzw_rpc.tkrzw_rpc_pb2_grpc.DBMServiceStub(channel)¶

Bases: object

Definition of the database service.

__init__(channel)¶

Constructor.

Args:: channel: A grpc.Channel.

class tkrzw_rpc.tkrzw_rpc_pb2_grpc.DBMServiceServicer¶

Bases: object

Definition of the database service.

Echo(request, context)¶

Inspect(request, context)¶

Get(request, context)¶

GetMulti(request, context)¶

Set(request, context)¶

SetMulti(request, context)¶

Remove(request, context)¶

RemoveMulti(request, context)¶

Append(request, context)¶

AppendMulti(request, context)¶

CompareExchange(request, context)¶

Increment(request, context)¶

CompareExchangeMulti(request, context)¶

Rekey(request, context)¶

PopFirst(request, context)¶

PushLast(request, context)¶

Count(request, context)¶

GetFileSize(request, context)¶

Clear(request, context)¶

Rebuild(request, context)¶

ShouldBeRebuilt(request, context)¶

Synchronize(request, context)¶

Search(request, context)¶

Stream(request_iterator, context)¶

Iterate(request_iterator, context)¶

Replicate(request, context)¶

ChangeMaster(request, context)¶

tkrzw_rpc.tkrzw_rpc_pb2_grpc.add_DBMServiceServicer_to_server(servicer, server)¶

Module contents¶

class tkrzw_rpc.Status(code=0, message='')¶

Bases: object

Status of operations.

SUCCESS = 0¶: Success.

UNKNOWN_ERROR = 1¶: Generic error whose cause is unknown.

SYSTEM_ERROR = 2¶: Generic error from underlying systems.

NOT_IMPLEMENTED_ERROR = 3¶: Error that the feature is not implemented.

PRECONDITION_ERROR = 4¶: Error that a precondition is not met.

INVALID_ARGUMENT_ERROR = 5¶: Error that a given argument is invalid.

CANCELED_ERROR = 6¶: Error that the operation is canceled.

NOT_FOUND_ERROR = 7¶: Error that a specific resource is not found.

PERMISSION_ERROR = 8¶: Error that the operation is not permitted.

INFEASIBLE_ERROR = 9¶: Error that the operation is infeasible.

DUPLICATION_ERROR = 10¶: Error that a specific resource is duplicated.

BROKEN_DATA_ERROR = 11¶: Error that internal data are broken.

NETWORK_ERROR = 12¶: Error caused by networking failure.

APPLICATION_ERROR = 13¶: Generic error caused by the application logic.

__init__(code=0, message='')¶

Sets the code and the message.

Parameters

code – The status code. This can be omitted and then SUCCESS is set.
message – An arbitrary status message. This can be omitted and the an empty string is set.

__repr__()¶

Returns a string representation of the object.

Returns: The string representation of the object.

__str__()¶

Returns a string representation of the content.

Returns: The string representation of the content.

__eq__(rhs)¶

Returns true if the given object is equivalent to this object.

Returns: True if the given object is equivalent to this object.

This supports comparison between a status object and a status code number.

Set(code=0, message='')¶

Sets the code and the message.

Parameters

code – The status code. This can be omitted and then SUCCESS is set.
message – An arbitrary status message. This can be omitted and the an empty string is set.

Join(rht)¶

Assigns the internal state from another status object only if the current state is success.

Parameters: rhs – The status object.

GetCode()¶

Gets the status code.

Returns: The status code.

GetMessage()¶

Gets the status message.

Returns: The status message.

IsOK()¶

Returns true if the status is success.

Returns: True if the status is success, or False on failure.

OrDie()¶

Raises an exception if the status is not success.

Raises: StatusException – An exception containing the status object.

classmethod CodeName(code)¶

Gets the string name of a status code.

Param: code The status code.
Returns: The name of the status code.

__hash__ = None¶

exception tkrzw_rpc.StatusException(status)¶

Bases: RuntimeError

Exception to convey the status of operations.

__init__(status)¶

Sets the status.

Parameters: status – The status object.

__repr__()¶

Returns A string representation of the object.

Returns: The string representation of the object.

__str__()¶

Returns A string representation of the content.

Returns: The string representation of the content.

GetStatus()¶

Gets the status object

Returns: The status object.

class tkrzw_rpc.RemoteDBM¶

Bases: object

Remote database manager.

All operations are thread-safe; Multiple threads can access the same database concurrently. The SetDBMIndex affects all threads so it should be called before the object is shared. This class implements the iterable protocol so an instance is usable with “for-in” loop.

ANY_DATA = b'\x00[ANY]\x00'¶: The special bytes value for no-operation or any data.

__init__()¶: Does nothing especially.

__repr__()¶

Returns A string representation of the object.

Returns: The string representation of the object.

__str__()¶

Returns A string representation of the content.

Returns: The string representation of the content.

__len__()¶

Gets the number of records, to enable the len operator.

Returns: The number of records on success, or 0 on failure.

__contains__(key)¶

Checks if a record exists or not, to enable the in operator.

Parameters: key – The key of the record.
Returns: True if the record exists, or False if not. No exception is raised for missing records.

__getitem__(key)¶

Gets the value of a record, to enable the [] operator.

Parameters: key – The key of the record.
Returns: The value of the matching record. An exception is raised for missing records. If the given key is a string, the returned value is also a string. Otherwise, the return value is bytes.
Raises: StatusException – An exception containing the status object.

__setitem__(key, value)¶

Sets a record of a key and a value, to enable the []= operator.

Parameters

key – The key of the record.
value – The value of the record.

Raises

StatusException – An exception containing the status object.

__delitem__(key)¶

Removes a record of a key, to enable the del [] operator.

Parameters: key – The key of the record.
Raises: StatusException – An exception containing the status object.

__iter__()¶

Makes an iterator and initialize it, to comply to the iterator protocol.

Returns: The iterator for each record.

Connect(address, timeout=None, auth_config=None)¶

Connects to the server.

Parameters

address – The address or the host name of the server and its port number. For IPv4 address, it’s like “127.0.0.1:1978”. For IPv6, it’s like “[::1]:1978”. For UNIX domain sockets, it’s like “unix:/path/to/file”.
timeout – The timeout in seconds for connection and each operation. Negative means unlimited.
auth_config – The authentication configuration. It it is empty or None, no authentication is done. If it begins with “ssl:”, the SSL authentication is done. Key-value parameters in “key=value,key=value,…” format comes next. For SSL, “key”, “cert”, and “root” parameters specify the paths of the client private key file, the client certificate file, and the root CA certificate file respectively.

Returns

The result status.

Disconnect()¶

Disconnects the connection to the server.

:return The result status.

SetDBMIndex(dbm_index)¶

Sets the index of the DBM to access.

Parameters: dbm_index – The index of the DBM to access.
Returns: The result status.

Echo(message, status=None)¶

Sends a message and gets back the echo message.

Param: message The message to send.
Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: The string value of the echoed message or None on failure.

Inspect()¶

Inspects the database.

Returns: A map of property names and their values.

If the DBM index is negative, basic metadata of all DBMs are obtained.

Get(key, status=None)¶

Gets the value of a record of a key.

Parameters

key – The key of the record.
status – A status object to which the result status is assigned. It can be omitted.

Returns

The bytes value of the matching record or None on failure.

GetStr(key, status=None)¶

Gets the value of a record of a key, as a string.

Parameters

key – The key of the record.
status – A status object to which the result status is assigned. It can be omitted.

Returns

The string value of the matching record or None on failure.

GetMulti(*keys)¶

Gets the values of multiple records of keys.

Parameters: keys – The keys of records to retrieve.
Returns: A map of retrieved records. Keys which don’t match existing records are ignored.

GetMultiStr(*keys)¶

Gets the values of multiple records of keys, as strings.

Parameters: keys – The keys of records to retrieve.
Returns: A map of retrieved records. Keys which don’t match existing records are ignored.

Set(key, value, overwrite=True)¶

Sets a record of a key and a value.

Parameters

key – The key of the record.
value – The value of the record.
overwrite – Whether to overwrite the existing value. It can be omitted and then false is set.

Returns

The result status. If overwriting is abandoned, DUPLICATION_ERROR is returned.

SetMulti(overwrite=True, **records)¶

Sets multiple records of the keyword arguments.

Parameters

overwrite – Whether to overwrite the existing value if there’s a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
records – Records to store, specified as keyword parameters.

Returns

The result status. If there are records avoiding overwriting, DUPLICATION_ERROR is returned.

Remove(key)¶

Removes a record of a key.

Parameters: key – The key of the record.
Returns: The result status. If there’s no matching record, NOT_FOUND_ERROR is returned.

RemoveMulti(*keys)¶

Removes records of keys.

Parameters: key – The keys of the records.
Returns: The result status. If there are missing records, NOT_FOUND_ERROR is returned.

Append(key, value, delim='')¶

Appends data at the end of a record of a key.

Parameters

key – The key of the record.
value – The value to append.
delim – The delimiter to put after the existing record.

Returns

The result status.

If there’s no existing record, the value is set without the delimiter.

AppendMulti(delim='', **records)¶

Appends data to multiple records of the keyword arguments.

Parameters

delim – The delimiter to put after the existing record.
records – Records to append. Existing records with the same keys are overwritten.

Returns

The result status.

If there’s no existing record, the value is set without the delimiter.

CompareExchange(key, expected, desired)¶

Compares the value of a record and exchanges if the condition meets.

Parameters

key – The key of the record.
expected – The expected value. If it is None, no existing record is expected. If it is ANY_DATA, an existing record with any value is expacted.
desired – The desired value. If it is None, the record is to be removed. If it is ANY_DATA, no update is done.

Returns

The result status. If the condition doesn’t meet, INFEASIBLE_ERROR is returned.

CompareExchangeAdvanced(key, expected, desired, retry_wait=None, notify=False)¶

Does compare-and-exchange and/or gets the old value of the record.

Parameters

key – The key of the record.
expected – The expected value. If it is None, no existing record is expected. If it is ANY_DATA, an existing record with any value is expacted.
desired – The desired value. If it is None, the record is to be removed. If it is ANY_DATA, no update is done.
retry_wait – The maximum wait time in seconds before retrying. If it is None, no retry is done. If it is positive, retry is done after waiting for the notifications of the next update for the time at most.
notify – If true, a notification signal is sent to wake up retrying threads.

Returns

A pair of the result status and the.old value of the record. If the condition doesn’t meet, the state is INFEASIBLE_ERROR. If there’s no existing record, the value is None. If not None, the type of the returned old value is the same as the expected or desired value.

Increment(key, inc=1, init=0, status=None)¶

Increments the numeric value of a record.

Parameters

key – The key of the record.
inc – The incremental value. If it is Utility.INT64MIN, the current value is not changed and a new record is not created.
init – The initial value.
status – A status object to which the result status is assigned. It can be omitted.

Returns

The current value, or None on failure.

The record value is stored as an 8-byte big-endian integer. Negative is also supported.

CompareExchangeMulti(expected, desired)¶

Compares the values of records and exchanges if the condition meets.

Parameters

expected – A sequence of pairs of the record keys and their expected values. If the value is None, no existing record is expected. If the value is ANY_DATA, an existing record with any value is expacted.
desired – A sequence of pairs of the record keys and their desired values. If the value is None, the record is to be removed.

Returns

The result status. If the condition doesn’t meet, INFEASIBLE_ERROR is returned.

Rekey(new_key, overwrite=True, copying=False)¶

Changes the key of a record.

Parameters

old_key – The old key of the record.
new_key – The new key of the record.
overwrite – Whether to overwrite the existing record of the new key.
copying – Whether to retain the record of the old key.

Returns

The result status. If there’s no matching record to the old key, NOT_FOUND_ERROR is returned. If the overwrite flag is false and there is an existing record of the new key, DUPLICATION ERROR is returned.

This method is done atomically. The other threads observe that the record has either the old key or the new key. No intermediate states are observed.

PopFirst(retry_wait=None, status=None)¶

Gets the first record and removes it.

Parameters

retry_wait – The maximum wait time in seconds before retrying. If it is None, no retry is done. If it is positive, retry is done after waiting for the notifications of the next update for the time at most.
status – A status object to which the result status is assigned. It can be omitted.

Returns

A tuple of the bytes key and the bytes value of the first record. On failure, None is returned.

PopFirstStr(retry_wait=None, status=None)¶

Gets the first record as strings and removes it.

Parameters

retry_wait – The maximum wait time in seconds before retrying. If it is None, no retry is done. If it is positive, retry is done after waiting for the notifications of the next update for the time at most.
status – A status object to which the result status is assigned. It can be omitted.

Returns

A tuple of the string key and the string value of the first record. On failure, None is returned.

PushLast(value, wtime=None, notify=False)¶

Adds a record with a key of the current timestamp.

Parameters

value – The value of the record.
wtime – The current wall time used to generate the key. If it is None, the system clock is used.
notify – If true, notification signal is sent.

Returns

The result status.

The key is generated as an 8-bite big-endian binary string of the timestamp. If there is an existing record matching the generated key, the key is regenerated and the attempt is repeated until it succeeds.

Count()¶

Gets the number of records.

Returns: The number of records on success, or None on failure.

GetFileSize()¶

Gets the current file size of the database.

Returns: The current file size of the database, or None on failure.

Clear()¶

Removes all records.

Returns: The result status.

Rebuild(**params)¶

Rebuilds the entire database.

Parameters: params – Optional keyword parameters.
Returns: The result status.

The optional parameters are the same as the Open method of the local DBM class and the database configurations of the server command. Omitted tuning parameters are kept the same or implicitly optimized.

In addition, HashDBM, TreeDBM, and SkipDBM supports the following parameters.

skip_broken_records (bool): If true, the operation continues even if there are broken records which can be skipped.
sync_hard (bool): If true, physical synchronization with the hardware is done before finishing the rebuilt file.

ShouldBeRebuilt()¶

Checks whether the database should be rebuilt.

Returns: True to be optimized or false with no necessity.

Synchronize(hard, **params)¶

Synchronizes the content of the database to the file system.

Parameters

hard – True to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
params – Optional keyword parameters.

Returns

The result status.

The “reducer” parameter specifies the reducer for SkipDBM. “ReduceToFirst”, “ReduceToSecond”, “ReduceToLast”, etc are supported. If the parameter “make_backup” exists, a backup file is created in the same directory as the database file. The backup file name has a date suffix in GMT, like “.backup.20210831213749”. If the value of “make_backup” not empty, it is the value is used as the suffix.

Search(mode, pattern, capacity=0)¶

Searches the database and get keys which match a pattern.

Parameters

mode – The search mode. “contain” extracts keys containing the pattern. “begin” extracts keys beginning with the pattern. “end” extracts keys ending with the pattern. “regex” extracts keys partially matches the pattern of a regular expression. “edit” extracts keys whose edit distance to the UTF-8 pattern is the least. “editbin” extracts keys whose edit distance to the binary pattern is the least.
pattern – The pattern for matching.
capacity – The maximum records to obtain. 0 means unlimited.

Returns

A list of string keys matching the condition.

MakeIterator()¶

Makes an iterator for each record.

Returns: The iterator for each record.

class tkrzw_rpc.Iterator(dbm)¶

Bases: object

Iterator for each record.

__init__(dbm)¶

Initializes the iterator.

Parameters: dbm – The database to scan.

__del__()¶: Destructs the iterator.

__repr__()¶

Returns A string representation of the object.

Returns: The string representation of the object.

__str__()¶

Returns A string representation of the content.

Returns: The string representation of the content.

__next__()¶

Moves the iterator to the next record, to comply to the iterator protocol.

Returns: A tuple of The key and the value of the current record.

First()¶

Initializes the iterator to indicate the first record.

Returns: The result status.

Even if there’s no record, the operation doesn’t fail.

Last()¶

Initializes the iterator to indicate the last record.

Returns: The result status.

Even if there’s no record, the operation doesn’t fail. This method is suppoerted only by ordered databases.

Jump(key)¶

Initializes the iterator to indicate a specific record.

Parameters: key – The key of the record to look for.
Returns: The result status.

Ordered databases can support “lower bound” jump; If there’s no record with the same key, the iterator refers to the first record whose key is greater than the given key. The operation fails with unordered databases if there’s no record with the same key.

JumpLower(key, inclusive=False)¶

Initializes the iterator to indicate the last record whose key is lower than a given key.

Parameters

key – The key to compare with.
inclusive – If true, the considtion is inclusive: equal to or lower than the key.

Returns

The result status.

Even if there’s no matching record, the operation doesn’t fail. This method is suppoerted only by ordered databases.

JumpUpper(key, inclusive=False)¶

Initializes the iterator to indicate the first record whose key is upper than a given key.

Parameters

key – The key to compare with.
inclusive – If true, the considtion is inclusive: equal to or upper than the key.

Returns

The result status.

Even if there’s no matching record, the operation doesn’t fail. This method is suppoerted only by ordered databases.

Next()¶

Moves the iterator to the next record.

Returns: The result status.

If the current record is missing, the operation fails. Even if there’s no next record, the operation doesn’t fail.

Previous()¶

Moves the iterator to the previous record.

Returns: The result status.

If the current record is missing, the operation fails. Even if there’s no previous record, the operation doesn’t fail. This method is suppoerted only by ordered databases.

Get(status=None)¶

Gets the key and the value of the current record of the iterator.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: A tuple of the bytes key and the bytes value of the current record. On failure, None is returned.

GetStr(status=None)¶

Gets the key and the value of the current record of the iterator, as strings.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: A tuple of the string key and the string value of the current record. On failure, None is returned.

GetKey(status=None)¶

Gets the key of the current record.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: The bytes key of the current record or None on failure.

GetKeyStr(status=None)¶

Gets the key of the current record, as a string.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: The string key of the current record or None on failure.

GetValue(status=None)¶

Gets the value of the current record.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: The bytes value of the current record or None on failure.

GetValueStr(status=None)¶

Gets the value of the current record, as a string.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: The string value of the current record or None on failure.

Set(value)¶

Sets the value of the current record.

Parameters: value – The value of the record.
Returns: The result status.

Remove()¶

Removes the current record.

Returns: The result status.

Step(status=None)¶

Gets the current record and moves the iterator to the next record.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: A tuple of the bytes key and the bytes value of the current record. On failure, None is returned.

StepStr(status=None)¶

Gets the current record and moves the iterator to the next record, as strings.

Parameters: status – A status object to which the result status is assigned. It can be omitted.
Returns: A tuple of the string key and the string value of the current record. On failure, None is returned.