class Tkrzw::DBM
Polymorphic database manager. All operations except for “open” and “close” are thread-safe; Multiple threads can access the same database concurrently. You can specify a data structure when you call the “open” method. Every opened database must be closed explicitly by the “close” method to avoid data corruption.
Constants
- ANY_DATA
The special bytes value for no-operation or any data.
Public Class Methods
Does nothing especially.
# File tkrzw.rb, line 293 def initialize() # (native code) end
Restores a broken database as a new healthy database.
-
@param old_file_path The path of the broken database.
-
@param new_file_path The path of the new database to be created.
-
@param class_name The name of the database class. If it is nil or empty, the class is guessed from the file extension.
-
@param end_offset The exclusive end offset of records to read. Negative means unlimited. 0 means the size when the database is synched or closed properly. Using a positive value is not meaningful if the number of shards is more than one.
-
@param cipher_key The encryption key for cipher compressors. If it is nil, an empty key is used.
-
@return The result status.
# File tkrzw.rb, line 705 def self.restore_database(old_file_path, new_file_path, class_name="", end_offset=-1, cipher_key=nil) # (native code) end
Public Instance Methods
Gets the value of a record, to enable the [] operator.
-
@param key The key of the record.
-
@return The value of the matching record or nil on failure.
# File tkrzw.rb, line 731 def [](key) # (native code) end
Sets a record of a key and a value, to enable the []= operator.
-
@param key The key of the record.
-
@param value The value of the record.
-
@return The new value of the record or nil on failure.
# File tkrzw.rb, line 739 def []=(key, value) # (native code) end
Appends data at the end of a record of a key.
-
@param key The key of the record.
-
@param value The value to append.
-
@param delim The delimiter to put after the existing record.
-
@return The result status.
If there's no existing record, the value is set without the delimiter.
# File tkrzw.rb, line 464 def append(key, value, delim="") # (native code) end
Appends data to multiple records of the keyword arguments.
-
@param delim The delimiter to put after the existing record.
-
@param records Records to append, specified as keyword parameters.
-
@return The result status.
# File tkrzw.rb, line 472 def append_multi(delim="", **records) # (native code) end
Removes all records.
-
@return The result status.
# File tkrzw.rb, line 584 def clear() # (native code) end
Closes the database file.
-
@return The result status.
# File tkrzw.rb, line 375 def close() # (native code) end
Compares the value of a record and exchanges if the condition meets.
-
@param key The key of the record.
-
@param expected The expected value. If it is nil, no existing record is expected. If it is
ANY_DATA
, an existing record with any value is expacted. -
@param desired The desired value. If it is nil, the record is to be removed. If it is
ANY_DATA
, no update is done. -
@return The result status. If the condition doesn't meet, INFEASIBLE_ERROR is returned.
# File tkrzw.rb, line 481 def compare_exchange(key, expected, desired) # (native code) end
Does compare-and-exchange and/or gets the old value of the record.
-
@param key The key of the record.
-
@param expected The expected value. If it is nil, no existing record is expected. If it is
ANY_DATA
, an existing record with any value is expacted. -
@param desired The desired value. If it is nil, the record is to be removed. If it is
ANY_DATA
, no update is done. -
@return A pair of the result status and the.old value of the record. If the condition doesn't meet, the state is INFEASIBLE_ERROR. If there's no existing record, the value is nil.
# File tkrzw.rb, line 490 def compare_exchange_and_get(key, expected, desired) # (native code) end
Compares the values of records and exchanges if the condition meets.
-
@param expected An array of pairs of the record keys and their expected values. If the value is nil, no existing record is expected. If the value is
ANY_DATA
, an existing record with any value is expacted. -
@param desired An array of pairs of the record keys and their desired values. If the value is nil, the record is to be removed.
-
@return The result status. If the condition doesn't meet, INFEASIBLE_ERROR is returned.
# File tkrzw.rb, line 518 def compare_exchange_multi(expected, desired) # (native code) end
Copies the content of the database file to another file.
-
@param dest_path A path to the destination file.
-
@param sync_hard True to do physical synchronization with the hardware.
-
@return The result status.
# File tkrzw.rb, line 618 def copy_file_data(dest_path, sync_hard=false) # (native code) end
Gets the number of records.
-
@return The number of records on success, or nil on failure.
# File tkrzw.rb, line 560 def count() # (native code) end
Removes a record of a key.
-
@param key The key of the record.
-
@return The old value of the record or nil on failure.
# File tkrzw.rb, line 746 def delete(key) # (native code) end
Releases the resource explicitly. The database is closed implicitly if it has not been closed. As long as you close the database explicitly, you don't have to call this method.
# File tkrzw.rb, line 299 def destruct() # (native code) end
Calls the given block with the key and the value of each record
# File tkrzw.rb, line 751 def each(&block) # (native code) end
Exports all records to another database.
-
@param dest_dbm The destination database.
-
@return The result status.
# File tkrzw.rb, line 625 def export(dest_dbm) # (native code) end
Exports the keys of all records as lines to a text file.
-
@param dest_file The file object to write keys in.
-
@return The result status.
As the exported text file is smaller than the database file, scanning the text file by the search method is often faster than scanning the whole database.
# File tkrzw.rb, line 648 def export_keys_as_lines(dest_file) # (native code) end
Exports all records of a database to a flat record file.
-
@param dest_file The file object to write records in.
-
@return The result status.
A flat record file contains a sequence of binary records without any high level structure so it is useful as a intermediate file for data migration.
# File tkrzw.rb, line 633 def export_to_flat_records(dest_file) # (native code) end
Gets the path of the database file.
-
@return The file path of the database, or nil on failure.
# File tkrzw.rb, line 572 def file_path() # (native code) end
Gets the current file size of the database.
-
@return The current file size of the database, or nil on failure.
# File tkrzw.rb, line 566 def file_size() # (native code) end
Gets the value of a record of a key.
-
@param key The key of the record.
-
@param status A status object to which the result status is assigned. It can be omitted.
-
@return The value of the matching record or nil on failure.
# File tkrzw.rb, line 400 def get(key, status=nil) # (native code) end
Gets the values of multiple records of keys.
-
@param keys The keys of records to retrieve.
-
@return A map of retrieved records. Keys which don't match existing records are ignored.
# File tkrzw.rb, line 407 def get_multi(*keys) # (native code) end
Checks whether the database condition is healthy.
-
@return True if the database condition is healthy, or false if not.
# File tkrzw.rb, line 672 def healthy?() # (native code) end
Imports records to a database from a flat record file.
-
@param src_file The file object to read records from.
-
@return The result status.
# File tkrzw.rb, line 640 def import_from_flat_records(src_file) # (native code) end
Checks if a record exists or not.
-
@param key The key of the record.
-
@return True if the record exists, or false if not.
# File tkrzw.rb, line 392 def include?(key) # (native code) end
Increments the numeric value of a record.
-
@param key The key of the record.
-
@param inc The incremental value. If it is Utility::INT64MIN, the current value is not changed and a new record is not created.
-
@param init The initial value.
-
@param status A status object to which the result status is assigned. It can be omitted.
-
@return The current value, or nil on failure.
The record value is stored as an 8-byte big-endian integer. Negative is also supported.
# File tkrzw.rb, line 501 def increment(key, inc=1, init=0, status=nil) # (native code) end
Returns a string representation of the object.
-
@return The string representation of the object.
# File tkrzw.rb, line 724 def inspect() # (native code) end
Inspects the database.
-
@return A hash of property names and their values.
# File tkrzw.rb, line 654 def inspect_details() # (native code) end
Makes an iterator for each record.
-
@return The iterator for each record.
Every iterator should be destructed explicitly by the “destruct” method.
# File tkrzw.rb, line 694 def make_iterator() # (native code) end
Opens a database file.
-
@param path A path of the file.
-
@param writable If true, the file is writable. If false, it is read-only.
-
@param params Optional keyword parameters.
-
@return The result status.
The extension of the path indicates the type of the database.
-
.tkh :
File
hash database (HashDBM) -
.tkt :
File
tree database (TreeDBM) -
.tks :
File
skip database (SkipDBM) -
.tkmt : On-memory hash database (TinyDBM)
-
.tkmb : On-memory tree database (BabyDBM)
-
.tkmc : On-memory cache database (CacheDBM)
-
.tksh : On-memory STL hash database (StdHashDBM)
-
.tkst : On-memory STL tree database (StdTreeDBM)
The optional parameters can include an option for the concurrency tuning. By default, database operatins are done under the GVL (Global Virtual-machine Lock), which means that database operations are not done concurrently even if you use multiple threads. If the “concurrent” parameter is true, database operations are done outside the GVL, which means that database operations can be done concurrently if you use multiple threads. However, the downside is that swapping thread data is costly so the actual throughput is often worse in the concurrent mode than in the normal mode. Therefore, the concurrent mode should be used only if the database is huge and it can cause blocking of threads in multi-thread usage.
By default, the encoding of retrieved record data by the “get” method is implicitly set as “ASCII-8BIT”. If you want to change the implicit encoding to “UTF-8” or others, set the encoding name as the value of the “encoding” parameter. The optional parameters can include options for the file opening operation.
-
truncate (bool): True to truncate the file.
-
no_create (bool): True to omit file creation.
-
no_wait (bool): True to fail if the file is locked by another process.
-
no_lock (bool): True to omit file locking.
-
sync_hard (bool): True to do physical synchronization when closing.
The optional parameter “dbm” supercedes the decision of the database type by the extension. The value is the type name: “HashDBM”, “TreeDBM”, “SkipDBM”, “TinyDBM”, “BabyDBM”, “CacheDBM”, “StdHashDBM”, “StdTreeDBM”.
The optional parameter “file” specifies the internal file implementation class. The default file class is “MemoryMapAtomicFile”. The other supported classes are “StdFile”, “MemoryMapAtomicFile”, “PositionalParallelFile”, and “PositionalAtomicFile”.
For HashDBM, these optional parameters are supported.
-
update_mode (string): How to update the database file: “UPDATE_IN_PLACE” for the in-palce or “UPDATE_APPENDING” for the appending mode.
-
record_crc_mode (string): How to add the CRC data to the record: “RECORD_CRC_NONE” to add no CRC to each record, “RECORD_CRC_8” to add CRC-8 to each record, “RECORD_CRC_16” to add CRC-16 to each record, or “RECORD_CRC_32” to add CRC-32 to each record.
-
record_comp_mode (string): How to compress the record data: “RECORD_COMP_NONE” to do no compression, “RECORD_COMP_ZLIB” to compress with ZLib, “RECORD_COMP_ZSTD” to compress with ZStd, “RECORD_COMP_LZ4” to compress with LZ4, “RECORD_COMP_LZMA” to compress with LZMA, “RECORD_COMP_RC4” to cipher with RC4, “RECORD_COMP_AES” to cipher with AES.
-
offset_width (int): The width to represent the offset of records.
-
align_pow (int): The power to align records.
-
num_buckets (int): The number of buckets for hashing.
-
restore_mode (string): How to restore the database file: “RESTORE_SYNC” to restore to the last synchronized state, “RESTORE_READ_ONLY” to make the database read-only, or “RESTORE_NOOP” to do nothing. By default, as many records as possible are restored.
-
fbp_capacity (int): The capacity of the free block pool.
-
min_read_size (int): The minimum reading size to read a record.
-
cache_buckets (bool): True to cache the hash buckets on memory.
-
cipher_key (string): The encryption key for cipher compressors.
For TreeDBM, all optional parameters for HashDBM are available. In addition, these optional parameters are supported.
-
max_page_size (int): The maximum size of a page.
-
max_branches (int): The maximum number of branches each inner node can have.
-
max_cached_pages (int): The maximum number of cached pages.
-
page_update_mode (string): What to do when each page is updated: “PAGE_UPDATE_NONE” is to do no operation or “PAGE_UPDATE_WRITE” is to write immediately.
-
key_comparator (string): The comparator of record keys: “LexicalKeyComparator” for the lexical order, “LexicalCaseKeyComparator” for the lexical order ignoring case, “DecimalKeyComparator” for the order of decimal integer numeric expressions, “HexadecimalKeyComparator” for the order of hexadecimal integer numeric expressions, “RealNumberKeyComparator” for the order of decimal real number expressions, “SignedBigEndianKeyComparator” for the order of binary signed integer expressions, and “FloatBigEndianKeyComparator” for the order of binary float-number expressions.
For SkipDBM, these optional parameters are supported.
-
offset_width (int): The width to represent the offset of records.
-
step_unit (int): The step unit of the skip list.
-
max_level (int): The maximum level of the skip list.
-
restore_mode (string): How to restore the database file: “RESTORE_SYNC” to restore to the last synchronized state or “RESTORE_NOOP” to do nothing make the database read-only. By default, as many records as possible are restored.
-
sort_mem_size (int): The memory size used for sorting to build the database in the at-random mode.
-
insert_in_order (bool): If true, records are assumed to be inserted in ascending order of the key.
-
max_cached_records (int): The maximum number of cached records.
For TinyDBM, these optional parameters are supported.
-
num_buckets (int): The number of buckets for hashing.
For BabyDBM, these optional parameters are supported.
-
key_comparator (string): The comparator of record keys. The same ones as TreeDBM.
For CacheDBM, these optional parameters are supported.
-
cap_rec_num (int): The maximum number of records.
-
cap_mem_size (int): The total memory size to use.
All databases support taking update logs into files. It is enabled by setting the prefix of update log files.
-
ulog_prefix (str): The prefix of the update log files.
-
ulog_max_file_size (num): The maximum file size of each update log file. By default, it is 1GiB.
-
ulog_server_id (num): The server ID attached to each log. By default, it is 0.
-
ulog_dbm_index (num): The
DBM
index attached to each log. By default, it is 0.
For the file “PositionalParallelFile” and “PositionalAtomicFile”, these optional parameters are supported.
-
block_size (int): The block size to which all blocks should be aligned.
-
access_options (str): Values separated by colon. “direct” for direct I/O. “sync” for synchrnizing I/O, “padding” for file size alignment by padding, “pagecache” for the mini page cache in the process.
If the optional parameter “num_shards” is set, the database is sharded into multiple shard files. Each file has a suffix like “-00003-of-00015”. If the value is 0, the number of shards is set by patterns of the existing files, or 1 if they doesn't exist.
# File tkrzw.rb, line 369 def open(path, writable, **params) # (native code) end
Checks whether the database is open.
-
@return True if the database is open, or false if not.
# File tkrzw.rb, line 660 def open?() # (native code) end
Checks whether ordered operations are supported.
-
@return True if ordered operations are supported, or false if not.
# File tkrzw.rb, line 678 def ordered?() # (native code) end
Gets the first record and removes it.
-
@param status A status object to which the result status is assigned. It can be omitted.
-
@return A tuple of The key and the value of the first record. On failure, nil is returned.
# File tkrzw.rb, line 536 def pop_first(status=nil) # (native code) end
Processes a record with an arbitrary block.
-
@param key The key of the record.
-
@param writable True if the processor can edit the record.
-
@block The block to process a record. The first parameter is the key of the record. The second parameter is the value of the existing record, or nil if it the record doesn't exist. The return value is a string or bytes to update the record value. If the return value is nil, the record is not modified. If the return value is false (not a false value but the false object), the record is removed.
-
@return The result status.
This method is not available in the concurrent mode because the function cannot be invoked outside the GIL.
# File tkrzw.rb, line 385 def process(key, writable) # (native code) end
Processes each and every record in the database with an arbitrary block.
-
@block The block to process a record. The first parameter is the key of the record. The second parameter is the value of the existing record, or nil if it the record doesn't exist. The return value is a string or bytes to update the record value. If the return value is nil, the record is not modified. If the return value is false (not a false value but the false object), the record is removed.
-
@param writable True if the processor can edit the record.
-
@return The result status.
The block function is called repeatedly for each record. It is also called once before the iteration and once after the iteration with both the key and the value being nil. This method is not available in the concurrent mode because the function cannot be invoked outside the GIL.
# File tkrzw.rb, line 554 def process_each(writable) # (native code) end
Processes multiple records with an arbitrary block.
-
@param keys A list of record keys.
-
@block The block to process a record. The first parameter is the key of the record. The second parameter is the value of the existing record, or nil if it the record doesn't exist. The return value is a string or bytes to update the record value. If the return value is nil, the record is not modified. If the return value is false (not a false value but the false object), the record is removed.
-
@return The result status.
This method is not available in the concurrent mode because the function cannot be invoked outside the GIL.
# File tkrzw.rb, line 510 def process_multi(keys, writable) # (native code) end
Adds a record with a key of the current timestamp.
-
@param value The value of the record.
-
@param wtime The current wall time used to generate the key. If it is nil, the system clock is used.
-
@return The result status.
The key is generated as an 8-bite big-endian binary string of the timestamp. If there is an existing record matching the generated key, the key is regenerated and the attempt is repeated until it succeeds.
# File tkrzw.rb, line 545 def push_last(value, wtime=nil) # (native code) end
Rebuilds the entire database.
-
@param params Optional keyword parameters.
-
@return The result status.
The optional parameters are the same as the “open” method. Omitted tuning parameters are kept the same or implicitly optimized.
In addition, HashDBM, TreeDBM, and SkipDBM supports the following parameters.
-
skip_broken_records (bool): If true, the operation continues even if there are broken records which can be skipped.
-
sync_hard (bool): If true, physical synchronization with the hardware is done before finishing the rebuilt file.
# File tkrzw.rb, line 595 def rebuild(**params) # (native code) end
Changes the key of a record.
-
@param old_key The old key of the record.
-
@param new_key The new key of the record.
-
@param overwrite Whether to overwrite the existing record of the new key.
-
@param copying Whether to retain the record of the old key.
-
@return The result status. If there's no matching record to the old key, NOT_FOUND_ERROR is returned. If the overwrite flag is false and there is an existing record of the new key, DUPLICATION ERROR is returned.
This method is done atomically. The other threads observe that the record has either the old key or the new key. No intermediate states are observed.
# File tkrzw.rb, line 529 def rekey(old_key, new_key, overwrite=true, copying=false) # (native code) end
Removes a record of a key.
-
@param key The key of the record.
-
@return The result status. If there's no matching record, NOT_FOUND_ERROR is returned.
# File tkrzw.rb, line 440 def remove(key) # (native code) end
Removes a record and get the value.
-
@param key The key of the record.
-
@return A pair of the result status and the record value. If the record does not exist, nil is assigned as the value.
# File tkrzw.rb, line 454 def remove_and_get(key) # (native code) end
Removes records of keys.
-
@param keys The keys of the records.
-
@return The result status. If there are missing records, NOT_FOUND_ERROR is returned.
# File tkrzw.rb, line 447 def remove_multi(*keys) # (native code) end
Searches the database and get keys which match a pattern.
-
@param mode The search mode. “contain” extracts keys containing the pattern. “begin” extracts keys beginning with the pattern. “end” extracts keys ending with the pattern. “regex” extracts keys partially matches the pattern of a regular expression. “edit” extracts keys whose edit distance to the UTF-8 pattern is the least. “editbin” extracts keys whose edit distance to the binary pattern is the least. “containcase”, “containword”, and “containcaseword” extract keys considering case and word boundary. Ordered databases support “upper” and “lower” which extract keys whose positions are upper/lower than the pattern. “upperinc” and “lowerinc” are their inclusive versions.
-
@param pattern The pattern for matching.
-
@param capacity The maximum records to obtain. 0 means unlimited.
-
@return A list of keys matching the condition.
# File tkrzw.rb, line 687 def search(mode, pattern, capacity=0) # (native code) end
Sets a record of a key and a value.
-
@param key The key of the record.
-
@param value The value of the record.
-
@param overwrite Whether to overwrite the existing value. It can be omitted and then false is set.
-
@return The result status. If overwriting is abandoned, DUPLICATION_ERROR is returned.
# File tkrzw.rb, line 416 def set(key, value, overwrite=true) # (native code) end
Sets a record and get the old value.
-
@param key The key of the record.
-
@param value The value of the record.
-
@param overwrite Whether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
-
@return A pair of the result status and the old value. If the record has not existed when inserting the new record, nil is assigned as the value.
# File tkrzw.rb, line 433 def set_and_get(key, value, overwrite=true) # (native code) end
Sets multiple records of the keyword arguments.
-
@param overwrite Whether to overwrite the existing value if there's a record with the same key. If true, the existing value is overwritten by the new value. If false, the operation is given up and an error status is returned.
-
@param records Records to store, specified as keyword parameters.
-
@return The result status. If there are records avoiding overwriting, DUPLICATION_ERROR is returned.
# File tkrzw.rb, line 424 def set_multi(overwrite=true, **records) # (native code) end
Checks whether the database should be rebuilt.
-
@return True to be optimized or False with no necessity.
# File tkrzw.rb, line 601 def should_be_rebuilt?() # (native code) end
Synchronizes the content of the database to the file system.
-
@param hard True to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
-
@param params Optional keyword parameters.
-
@return The result status.
Only SkipDBM uses the optional parameters. The “merge” parameter specifies paths of databases to merge, separated by colon. The “reducer” parameter specifies the reducer to apply to records of the same key. “ReduceToFirst”, “ReduceToSecond”, “ReduceToLast”, etc are supported.
# File tkrzw.rb, line 610 def synchronize(hard, **params) # (native code) end
Gets the timestamp in seconds of the last modified time.
-
@return The timestamp of the last modified time, or nil on failure.
# File tkrzw.rb, line 578 def timestamp() # (native code) end
Gets the number of records.
-
@return The number of records on success, or -1 on failure.
# File tkrzw.rb, line 718 def to_i() # (native code) end
Returns a string representation of the content.
-
@return The string representation of the content.
# File tkrzw.rb, line 712 def to_s() # (native code) end
Checks whether the database is writable.
-
@return True if the database is writable, or false if not.
# File tkrzw.rb, line 666 def writable?() # (native code) end