Tkrzw
Classes | Public Types | Public Member Functions | Static Public Member Functions | Static Public Attributes | List of all members
tkrzw::SkipDBM Class Referencefinal

File database manager implementation based on skip list. More...

#include <tkrzw_dbm_skip.h>

Classes

class  Iterator
 Iterator for each record. More...
 
struct  TuningParameters
 Tuning parameters for the database. More...
 

Public Types

enum  RestoreMode : int32_t {
  RESTORE_DEFAULT = 0 , RESTORE_SYNC = 1 , RESTORE_READ_ONLY = 2 , RESTORE_NOOP = 3 ,
  RESTORE_NO_SHORTCUTS = 1 << 16 , RESTORE_WITH_HARDSYNC = 1 << 17
}
 Enumeration for restore modes. More...
 
typedef std::vector< std::string >(* ReducerType) (const std::string &, const std::vector< std::string > &)
 Interface of a function to reduce records of the same key in the at-random build mode. More...
 
- Public Types inherited from tkrzw::DBM
typedef std::function< std::string_view(std::string_view, std::string_view)> RecordLambdaType
 Lambda function type to process a record. More...
 

Public Member Functions

 SkipDBM ()
 Default constructor. More...
 
 SkipDBM (std::unique_ptr< File > file)
 Constructor with a file object. More...
 
 ~SkipDBM ()
 Destructor. More...
 
 SkipDBM (const SkipDBM &rhs)=delete
 Copy and assignment are disabled. More...
 
SkipDBMoperator= (const SkipDBM &rhs)=delete
 
Status Open (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT) override
 Opens a database file. More...
 
Status OpenAdvanced (const std::string &path, bool writable, int32_t options=File::OPEN_DEFAULT, const TuningParameters &tuning_params=TuningParameters())
 Opens a database file, in an advanced way. More...
 
Status Close () override
 Closes the database file. More...
 
Status Process (std::string_view key, RecordProcessor *proc, bool writable) override
 Processes a record with a processor. More...
 
Status Set (std::string_view key, std::string_view value, bool overwrite=true, std::string *old_value=nullptr) override
 Sets a record of a key and a value. More...
 
Status Remove (std::string_view key, std::string *old_value=nullptr) override
 Removes a record of a key. More...
 
Status GetByIndex (int64_t index, std::string *key=nullptr, std::string *value=nullptr)
 Gets the key and the value of the record of an index. More...
 
Status ProcessMulti (const std::vector< std::pair< std::string_view, DBM::RecordProcessor * >> &key_proc_pairs, bool writable) override
 Processes multiple records with processors. More...
 
Status ProcessFirst (RecordProcessor *proc, bool writable) override
 Processes the first record with a processor. More...
 
Status ProcessEach (RecordProcessor *proc, bool writable) override
 Processes each and every record in the database with a processor. More...
 
Status Count (int64_t *count) override
 Gets the number of records. More...
 
Status GetFileSize (int64_t *size) override
 Gets the current file size of the database. More...
 
Status GetFilePath (std::string *path) override
 Gets the path of the database file. More...
 
Status GetTimestamp (double *timestamp) override
 Gets the timestamp in seconds of the last modified time. More...
 
Status Clear () override
 Removes all records. More...
 
Status Rebuild () override
 Rebuilds the entire database. More...
 
Status RebuildAdvanced (const TuningParameters &tuning_params=TuningParameters(), bool skip_broken_records=false, bool sync_hard=false)
 Rebuilds the entire database, in an advanced way. More...
 
Status ShouldBeRebuilt (bool *tobe) override
 Checks whether the database should be rebuilt. More...
 
Status Synchronize (bool hard, FileProcessor *proc=nullptr) override
 Synchronizes the content of the database to the file system. More...
 
Status SynchronizeAdvanced (bool hard, FileProcessor *proc=nullptr, ReducerType reducer=nullptr)
 Synchronizes the content of the database to the file system. More...
 
std::vector< std::pair< std::string, std::string > > Inspect () override
 Inspects the database. More...
 
bool IsOpen () const override
 Checks whether the database is open. More...
 
bool IsWritable () const override
 Checks whether the database is writable. More...
 
bool IsHealthy () const override
 Checks whether the database condition is healthy. More...
 
bool IsAutoRestored () const
 Checks whether the database has been restored automatically. More...
 
bool IsOrdered () const override
 Checks whether ordered operations are supported. More...
 
std::unique_ptr< DBM::IteratorMakeIterator () override
 Makes an iterator for each record. More...
 
std::unique_ptr< DBMMakeDBM () const override
 Makes a new DBM object of the same concrete class. More...
 
UpdateLoggerGetUpdateLogger () const override
 Gets the logger to write all update operations. More...
 
void SetUpdateLogger (UpdateLogger *update_logger) override
 Sets the logger to write all update operations. More...
 
FileGetInternalFile () const
 Gets the pointer to the internal file object. More...
 
int64_t GetEffectiveDataSize ()
 Gets the effective data size. More...
 
int32_t GetDatabaseType ()
 Gets the database type. More...
 
Status SetDatabaseType (uint32_t db_type)
 Sets the database type. More...
 
std::string GetOpaqueMetadata ()
 Gets the opaque metadata. More...
 
Status SetOpaqueMetadata (const std::string &opaque)
 Sets the opaque metadata. More...
 
Status Revert ()
 Reverts all updates. More...
 
bool IsUpdated ()
 Checks whether the database has been updated after being opened. More...
 
Status MergeSkipDatabase (const std::string &src_path)
 Merges the contents of another skip database file. More...
 
Status ValidateRecords ()
 Validates records. More...
 
- Public Member Functions inherited from tkrzw::DBM
virtual ~DBM ()=default
 Destructor. More...
 
virtual Status Process (std::string_view key, RecordLambdaType rec_lambda, bool writable)
 Processes a record with a lambda function. More...
 
virtual Status Get (std::string_view key, std::string *value=nullptr)
 Gets the value of a record of a key. More...
 
virtual std::string GetSimple (std::string_view key, std::string_view default_value="")
 Gets the value of a record of a key, in a simple way. More...
 
virtual Status GetMulti (const std::vector< std::string_view > &keys, std::map< std::string, std::string > *records)
 Gets the values of multiple records of keys, with a string view vector. More...
 
virtual Status GetMulti (const std::initializer_list< std::string_view > &keys, std::map< std::string, std::string > *records)
 Gets the values of multiple records of keys, with an initializer list. More...
 
virtual Status GetMulti (const std::vector< std::string > &keys, std::map< std::string, std::string > *records)
 Gets the values of multiple records of keys, with a string vector. More...
 
virtual Status SetMulti (const std::map< std::string_view, std::string_view > &records, bool overwrite=true)
 Sets multiple records, with a map of string views. More...
 
virtual Status SetMulti (const std::initializer_list< std::pair< std::string_view, std::string_view >> &records, bool overwrite=true)
 Sets multiple records, with an initializer list. More...
 
virtual Status SetMulti (const std::map< std::string, std::string > &records, bool overwrite=true)
 Sets multiple records, with a map of strings. More...
 
virtual Status RemoveMulti (const std::vector< std::string_view > &keys)
 Removes records of keys, with a string view vector. More...
 
virtual Status RemoveMulti (const std::initializer_list< std::string_view > &keys)
 Removes records of keys, with an initializer list. More...
 
virtual Status RemoveMulti (const std::vector< std::string > &keys)
 Removes records of keys, with a string vector. More...
 
virtual Status Append (std::string_view key, std::string_view value, std::string_view delim="")
 Appends data at the end of a record of a key. More...
 
virtual Status AppendMulti (const std::map< std::string_view, std::string_view > &records, std::string_view delim="")
 Appends data to multiple records, with a map of string views. More...
 
virtual Status AppendMulti (const std::initializer_list< std::pair< std::string_view, std::string_view >> &records, std::string_view delim="")
 Appends data to multiple records, with an initializer list. More...
 
virtual Status AppendMulti (const std::map< std::string, std::string > &records, std::string_view delim="")
 Appends data to multiple records, with a map of strings. More...
 
virtual Status CompareExchange (std::string_view key, std::string_view expected, std::string_view desired, std::string *actual=nullptr, bool *found=nullptr)
 Compares the value of a record and exchanges if the condition meets. More...
 
virtual Status Increment (std::string_view key, int64_t increment=1, int64_t *current=nullptr, int64_t initial=0)
 Increments the numeric value of a record. More...
 
virtual int64_t IncrementSimple (std::string_view key, int64_t increment=1, int64_t initial=0)
 Increments the numeric value of a record, in a simple way. More...
 
virtual Status ProcessMulti (const std::vector< std::pair< std::string_view, RecordProcessor * >> &key_proc_pairs, bool writable)=0
 Processes multiple records with processors. More...
 
virtual Status ProcessMulti (const std::vector< std::pair< std::string_view, RecordLambdaType >> &key_lambda_pairs, bool writable)
 Processes multiple records with lambda functions. More...
 
virtual Status CompareExchangeMulti (const std::vector< std::pair< std::string_view, std::string_view >> &expected, const std::vector< std::pair< std::string_view, std::string_view >> &desired)
 Compares the values of records and exchanges if the condition meets. More...
 
virtual Status Rekey (std::string_view old_key, std::string_view new_key, bool overwrite=true, bool copying=false, std::string *value=nullptr)
 Changes the key of a record. More...
 
virtual Status ProcessFirst (RecordLambdaType rec_lambda, bool writable)
 Processes the first record with a lambda function. More...
 
virtual Status PopFirst (std::string *key=nullptr, std::string *value=nullptr)
 Gets the first record and removes it. More...
 
virtual Status PushLast (std::string_view value, double wtime=-1, std::string *key=nullptr)
 Adds a record with a key of the current timestamp. More...
 
virtual Status ProcessEach (RecordLambdaType rec_lambda, bool writable)
 Processes each and every record in the database with a lambda function. More...
 
virtual int64_t CountSimple ()
 Gets the number of records, in a simple way. More...
 
virtual int64_t GetFileSizeSimple ()
 Gets the current file size of the database, in a simple way. More...
 
virtual std::string GetFilePathSimple ()
 Gets the path of the database file, in a simple way. More...
 
virtual double GetTimestampSimple ()
 Gets the timestamp of the last modified time, in a simple way. More...
 
virtual bool ShouldBeRebuiltSimple ()
 Checks whether the database should be rebuilt, in a simple way. More...
 
virtual Status CopyFileData (const std::string &dest_path, bool sync_hard=false)
 Copies the content of the database file to another file. More...
 
virtual Status Export (DBM *dest_dbm)
 Exports all records to another database. More...
 
const std::type_info & GetType () const
 Gets the type information of the actual class. More...
 

Static Public Member Functions

static std::vector< std::string > ReduceRemove (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by removing REMOVING_VALUE and past values. More...
 
static std::vector< std::string > ReduceToFirst (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by keeping the first value only. More...
 
static std::vector< std::string > ReduceToSecond (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by keeping the second or first value only. More...
 
static std::vector< std::string > ReduceToLast (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by keeping the last value only. More...
 
static std::vector< std::string > ReduceConcat (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by concatenating all values. More...
 
static std::vector< std::string > ReduceConcatWithNull (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by concatenating all values with null code. More...
 
static std::vector< std::string > ReduceConcatWithTab (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by concatenating all values with tab. More...
 
static std::vector< std::string > ReduceConcatWithLine (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by concatenating all values with linefeed. More...
 
static std::vector< std::string > ReduceToTotal (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by totaling numeric expressions. More...
 
static std::vector< std::string > ReduceToTotalBigEndian (const std::string &key, const std::vector< std::string > &values)
 Reduces the values of records of the same key by totaling big-endian binary expressions. More...
 
static Status ReadMetadata (File *file, int32_t *cyclic_magic, int32_t *pkg_major_version, int32_t *pkg_minor_version, int32_t *offset_width, int32_t *step_unit, int32_t *max_level, int32_t *closure_flags, int64_t *num_records, int64_t *eff_data_size, int64_t *file_size, int64_t *mod_time, int32_t *db_type, std::string *opaque)
 Reads metadata from a database file. More...
 
static Status RestoreDatabase (const std::string &old_file_path, const std::string &new_file_path)
 Restores a broken database as a new healthy database. More...
 

Static Public Attributes

static constexpr int32_t DEFAULT_OFFSET_WIDTH = 4
 The default value of the offset width. More...
 
static constexpr int32_t DEFAULT_STEP_UNIT = 4
 The default value of the step unit. More...
 
static constexpr int32_t DEFAULT_MAX_LEVEL = 14
 The default value of the maximum level. More...
 
static constexpr int64_t DEFAULT_SORT_MEM_SIZE = 256LL << 20
 The default value of the memory size used for sorting. More...
 
static constexpr int32_t DEFAULT_MAX_CACHED_RECORDS = 65536
 The default value of the maximum cached records. More...
 
static constexpr int32_t OPAQUE_METADATA_SIZE = 64
 The size of the opaque metadata. More...
 
static const std::string REMOVING_VALUE
 The special removing value. More...
 
- Static Public Attributes inherited from tkrzw::DBM
static const std::string_view ANY_DATA
 The special string_view value to represent any data. More...
 

Detailed Description

File database manager implementation based on skip list.

All operations are thread-safe; Multiple threads can access the same database concurrently. Every opened database must be closed explicitly to avoid data corruption.

Member Typedef Documentation

◆ ReducerType

typedef std::vector<std::string>(* tkrzw::SkipDBM::ReducerType) (const std::string &, const std::vector< std::string > &)

Interface of a function to reduce records of the same key in the at-random build mode.

The first parameter is the key. The second parameter is a vecgtor of record values. The return value is a vector of new values which are stored in the database.

Member Enumeration Documentation

◆ RestoreMode

Enumeration for restore modes.

Currently, RESTORE_DEFAULT and RESTORE_SYNC has the same meaning, and RESTORE_NO_SHORTCUTS is ignored.

Enumerator
RESTORE_DEFAULT 

The default behavior: to restore as many records as possible.

RESTORE_SYNC 

To restore to the last synchronized state.

RESTORE_READ_ONLY 

To make the database read-only.

RESTORE_NOOP 

To do nothing.

RESTORE_NO_SHORTCUTS 

Additional bit to not apply shortcuts.

RESTORE_WITH_HARDSYNC 

Additional bit to do physical synchronization.

Constructor & Destructor Documentation

◆ SkipDBM() [1/3]

tkrzw::SkipDBM::SkipDBM ( )

Default constructor.

MemoryMapParallelFile is used to handle the data.

◆ SkipDBM() [2/3]

tkrzw::SkipDBM::SkipDBM ( std::unique_ptr< File file)
explicit

Constructor with a file object.

Parameters
fileThe file object to handle the data. The ownership is taken.

◆ ~SkipDBM()

tkrzw::SkipDBM::~SkipDBM ( )

Destructor.

◆ SkipDBM() [3/3]

tkrzw::SkipDBM::SkipDBM ( const SkipDBM rhs)
explicitdelete

Copy and assignment are disabled.

Member Function Documentation

◆ Open()

Status tkrzw::SkipDBM::Open ( const std::string &  path,
bool  writable,
int32_t  options = File::OPEN_DEFAULT 
)
overridevirtual

Opens a database file.

Parameters
pathA path of the file.
writableIf true, the file is writable. If false, it is read-only.
optionsBit-sum options of File::OpenOption enums for opening the file.
Returns
The result status.

Precondition: The database is not opened.

Implements tkrzw::DBM.

◆ OpenAdvanced()

Status tkrzw::SkipDBM::OpenAdvanced ( const std::string &  path,
bool  writable,
int32_t  options = File::OPEN_DEFAULT,
const TuningParameters tuning_params = TuningParameters() 
)

Opens a database file, in an advanced way.

Parameters
pathA path of the file.
writableIf true, the file is writable. If false, it is read-only.
optionsBit-sum options for opening the file.
tuning_paramsA structure for tuning parameters.
Returns
The result status.

Precondition: The database is not opened.

◆ Close()

Status tkrzw::SkipDBM::Close ( )
overridevirtual

Closes the database file.

Returns
The result status.

Precondition: The database is opened.

Implements tkrzw::DBM.

◆ Process()

Status tkrzw::SkipDBM::Process ( std::string_view  key,
RecordProcessor proc,
bool  writable 
)
overridevirtual

Processes a record with a processor.

Parameters
keyThe key of the record.
procThe pointer to the processor object.
writableTrue if the processor can edit the record.
Returns
The result status.

Precondition: The database is opened. The writable parameter should be consistent to the open mode.

If the specified record exists, the ProcessFull of the processor is called. Otherwise, the ProcessEmpty of the processor is called. Inserted records are not visible until the database is synchronized. The database allows duplication of keys. Duplication can be solved by the reducer applied when the database is synchronized. Removing records is implemented as inserting a special value REMOVING_VALUE, which is handled when the database is synchronized.

Implements tkrzw::DBM.

◆ Set()

Status tkrzw::SkipDBM::Set ( std::string_view  key,
std::string_view  value,
bool  overwrite = true,
std::string *  old_value = nullptr 
)
overridevirtual

Sets a record of a key and a value.

Parameters
keyThe key of the record.
valueThe value of the record.
overwriteWhether to overwrite the existing value if there's a record with the same key. If true, the existing value is ovewritten by the new value. If false, the operation is given up and an error status is returned.
old_valueThe pointer to a string object to contain the old value. Assignment is done even on the duplication error. If it is nullptr, it is ignored.
Returns
The result status. If overwriting is abandoned, DUPLICATION_ERROR is returned.

Precondition: The database is opened as writable.

Reimplemented from tkrzw::DBM.

◆ Remove()

Status tkrzw::SkipDBM::Remove ( std::string_view  key,
std::string *  old_value = nullptr 
)
overridevirtual

Removes a record of a key.

Parameters
keyThe key of the record.
Returns
The result status. If there's no matching record, NOT_FOUND_ERROR is returned.
Parameters
old_valueThe pointer to a string object to contain the old value. If it is nullptr, it is ignored.

Even if there's no matching record, this doesn't report failure.

Precondition: The database is opened as writable.

Reimplemented from tkrzw::DBM.

◆ GetByIndex()

Status tkrzw::SkipDBM::GetByIndex ( int64_t  index,
std::string *  key = nullptr,
std::string *  value = nullptr 
)

Gets the key and the value of the record of an index.

Parameters
indexThe index of the record to retrieve.
keyThe pointer to a string object to contain the record key. If it is nullptr, the key data is ignored.
valueThe pointer to a string object to contain the record value. If it is nullptr, the value data is ignored.
Returns
The result status.

Precondition: The database is opened.

◆ ProcessMulti()

Status tkrzw::SkipDBM::ProcessMulti ( const std::vector< std::pair< std::string_view, DBM::RecordProcessor * >> &  key_proc_pairs,
bool  writable 
)
override

Processes multiple records with processors.

Parameters
key_proc_pairsPairs of the keys and their processor objects.
writableTrue if the processors can edit the records.
Returns
The result status.

Precondition: The database is opened. The writable parameter should be consistent to the open mode.

If the specified record exists, the ProcessFull of the processor is called. Otherwise, the ProcessEmpty of the processor is called.

◆ ProcessFirst()

Status tkrzw::SkipDBM::ProcessFirst ( RecordProcessor proc,
bool  writable 
)
overridevirtual

Processes the first record with a processor.

Parameters
procThe pointer to the processor object.
writableTrue if the processor can edit the record.
Returns
The result status.

Precondition: The database is opened. The writable parameter should be consistent to the open mode.

If the first record exists, the ProcessFull of the processor is called. Otherwise, this method fails and no method of the processor is called. The first record has the lowest key of all.

Implements tkrzw::DBM.

◆ ProcessEach()

Status tkrzw::SkipDBM::ProcessEach ( RecordProcessor proc,
bool  writable 
)
overridevirtual

Processes each and every record in the database with a processor.

Parameters
procThe pointer to the processor object.
writableTrue if the processor can edit the record.
Returns
The result status.

Precondition: The database is opened. The writable parameter should be consistent to the open mode.

The ProcessFull of the processor is called repeatedly for each record. The ProcessEmpty of the processor is called once before the iteration and once after the iteration.

Implements tkrzw::DBM.

◆ Count()

Status tkrzw::SkipDBM::Count ( int64_t *  count)
overridevirtual

Gets the number of records.

Parameters
countThe pointer to an integer object to contain the result count.
Returns
The result status.

Precondition: The database is opened.

Implements tkrzw::DBM.

◆ GetFileSize()

Status tkrzw::SkipDBM::GetFileSize ( int64_t *  size)
overridevirtual

Gets the current file size of the database.

Parameters
sizeThe pointer to an integer object to contain the result size.
Returns
The result status.

Precondition: The database is opened.

Implements tkrzw::DBM.

◆ GetFilePath()

Status tkrzw::SkipDBM::GetFilePath ( std::string *  path)
overridevirtual

Gets the path of the database file.

Parameters
pathThe pointer to a string object to contain the result path.
Returns
The result status.

Precondition: The database is opened.

Implements tkrzw::DBM.

◆ GetTimestamp()

Status tkrzw::SkipDBM::GetTimestamp ( double *  timestamp)
overridevirtual

Gets the timestamp in seconds of the last modified time.

Parameters
timestampThe pointer to a double object to contain the timestamp.
Returns
The result status.

Precondition: The database is opened.

The timestamp is updated when the database opened in the writable mode is closed or synchronized, even if no updating opertion is done.

Implements tkrzw::DBM.

◆ Clear()

Status tkrzw::SkipDBM::Clear ( )
overridevirtual

Removes all records.

Returns
The result status.

Precondition: The database is opened as writable.

Implements tkrzw::DBM.

◆ Rebuild()

Status tkrzw::SkipDBM::Rebuild ( )
overridevirtual

Rebuilds the entire database.

Returns
The result status.

Precondition: The database is opened as writable.

Rebuilding a database is useful to optimize the size of the file. All tuning parameters are succeeded or calculated implicitly.

Implements tkrzw::DBM.

◆ RebuildAdvanced()

Status tkrzw::SkipDBM::RebuildAdvanced ( const TuningParameters tuning_params = TuningParameters(),
bool  skip_broken_records = false,
bool  sync_hard = false 
)

Rebuilds the entire database, in an advanced way.

Parameters
tuning_paramsA structure for tuning parameters. The default value of each parameter means that the current setting is succeeded or calculated implicitly.
skip_broken_recordsIf true, the operation continues even if there are broken records which can be skipped. This is not used currently.
sync_hardTrue to do physical synchronization with the hardware before finishing the rebuilt file.
Returns
The result status.

Precondition: The database is opened as writable.

Rebuilding a database is useful to optimize the size of the file.

◆ ShouldBeRebuilt()

Status tkrzw::SkipDBM::ShouldBeRebuilt ( bool *  tobe)
overridevirtual

Checks whether the database should be rebuilt.

Parameters
tobeThe pointer to a boolean object to contain the result decision.
Returns
The result status.

Precondition: The database is opened.

Implements tkrzw::DBM.

◆ Synchronize()

Status tkrzw::SkipDBM::Synchronize ( bool  hard,
FileProcessor proc = nullptr 
)
overridevirtual

Synchronizes the content of the database to the file system.

Parameters
hardTrue to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
procThe pointer to the file processor object, whose Process method is called while the content of the file is synchronized. If it is nullptr, it is ignored.
Returns
The result status.

Precondition: The database is opened as writable.

All inserted records are written in the database file and they become visible. This operation rebuilds the entire database by merging the existing records and inserted records. No reducer is applied to records so that all records with duplicated keys are kept intact.

Implements tkrzw::DBM.

◆ SynchronizeAdvanced()

Status tkrzw::SkipDBM::SynchronizeAdvanced ( bool  hard,
FileProcessor proc = nullptr,
ReducerType  reducer = nullptr 
)

Synchronizes the content of the database to the file system.

Parameters
hardTrue to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
procThe pointer to the file processor object, whose Process method is called while the content of the file is synchronized. If it is nullptr, it is ignored.
Returns
The result status.
Parameters
reducerA reducer applied before stroing records. If it is nullptr, no reducer is applied and all records are kept intact.
Returns
The result status.

Precondition: The database is opened as writable.

All inserted records are written in the database file and they become visible. This operation rebuilds the entire database by merging the existing records and inserted records. Records of the same key given to the reducer are ordered in a way that the old value comes earlier than the new value. Thus, if you want to keep the old value, ReduceToFirst should be set. If you want to replace the old value with the new value, ReduceToLast should be set.

◆ Inspect()

std::vector<std::pair<std::string, std::string> > tkrzw::SkipDBM::Inspect ( )
overridevirtual

Inspects the database.

Returns
A vector of pairs of a property name and its value.

Implements tkrzw::DBM.

◆ IsOpen()

bool tkrzw::SkipDBM::IsOpen ( ) const
overridevirtual

Checks whether the database is open.

Returns
True if the database is open, or false if not.

Implements tkrzw::DBM.

◆ IsWritable()

bool tkrzw::SkipDBM::IsWritable ( ) const
overridevirtual

Checks whether the database is writable.

Returns
True if the database is writable, or false if not.

Implements tkrzw::DBM.

◆ IsHealthy()

bool tkrzw::SkipDBM::IsHealthy ( ) const
overridevirtual

Checks whether the database condition is healthy.

Returns
True if the database condition is healthy, or false if not.

Precondition: The database is opened.

Implements tkrzw::DBM.

◆ IsAutoRestored()

bool tkrzw::SkipDBM::IsAutoRestored ( ) const

Checks whether the database has been restored automatically.

Returns
True if the database condition has been restored by the last Open method.

Precondition: The database is opened.

◆ IsOrdered()

bool tkrzw::SkipDBM::IsOrdered ( ) const
overridevirtual

Checks whether ordered operations are supported.

Returns
Always true. Ordered operations are supported.

Implements tkrzw::DBM.

◆ MakeIterator()

std::unique_ptr<DBM::Iterator> tkrzw::SkipDBM::MakeIterator ( )
overridevirtual

Makes an iterator for each record.

Returns
The iterator for each record.

Precondition: The database is opened.

Implements tkrzw::DBM.

◆ MakeDBM()

std::unique_ptr<DBM> tkrzw::SkipDBM::MakeDBM ( ) const
overridevirtual

Makes a new DBM object of the same concrete class.

Returns
The new DBM object.

Implements tkrzw::DBM.

◆ GetUpdateLogger()

UpdateLogger* tkrzw::SkipDBM::GetUpdateLogger ( ) const
overridevirtual

Gets the logger to write all update operations.

Returns
The update logger if it has been set or nullptr if it hasn't.

Implements tkrzw::DBM.

◆ SetUpdateLogger()

void tkrzw::SkipDBM::SetUpdateLogger ( UpdateLogger update_logger)
overridevirtual

Sets the logger to write all update operations.

Parameters
update_loggerThe pointer to the update logger object. Ownership is not taken. If it is nullptr, no logger is used.

Implements tkrzw::DBM.

◆ GetInternalFile()

File* tkrzw::SkipDBM::GetInternalFile ( ) const

Gets the pointer to the internal file object.

Returns
The pointer to the internal file object.

Accessing the internal file viorates encapsulation policy. This should be used only for testing and debugging.

◆ GetEffectiveDataSize()

int64_t tkrzw::SkipDBM::GetEffectiveDataSize ( )

Gets the effective data size.

Returns
The effective data size, or -1 on failure.

Precondition: The database is opened.

The effective data size means the total size of the keys and the values.

◆ GetDatabaseType()

int32_t tkrzw::SkipDBM::GetDatabaseType ( )

Gets the database type.

Returns
The database type, or -1 on failure.

Precondition: The database is opened.

◆ SetDatabaseType()

Status tkrzw::SkipDBM::SetDatabaseType ( uint32_t  db_type)

Sets the database type.

Parameters
db_typeThe database type.
Returns
The result status.

Precondition: The database is opened as writable.

This data is just for applications and not used by the database implementation.

◆ GetOpaqueMetadata()

std::string tkrzw::SkipDBM::GetOpaqueMetadata ( )

Gets the opaque metadata.

Returns
The opaque metadata, or an empty string on failure.

Precondition: The database is opened.

◆ SetOpaqueMetadata()

Status tkrzw::SkipDBM::SetOpaqueMetadata ( const std::string &  opaque)

Sets the opaque metadata.

Parameters
opaqueThe opaque metadata, of which leading 64 bytes are stored in the file.
Returns
The result status.

Precondition: The database is opened as writable.

This data is just for applications and not used by the database implementation.

◆ Revert()

Status tkrzw::SkipDBM::Revert ( )

Reverts all updates.

Precondition: The database is opened as writable and not finished.

All inserted records are discarded and the contents are returned to the state when the database was synchronized at the last time.

◆ IsUpdated()

bool tkrzw::SkipDBM::IsUpdated ( )

Checks whether the database has been updated after being opened.

Returns
True if the database has been updated after being opened.

Precondition: The database is opened.

◆ MergeSkipDatabase()

Status tkrzw::SkipDBM::MergeSkipDatabase ( const std::string &  src_path)

Merges the contents of another skip database file.

Parameters
src_pathA path to the source database file.
Returns
The result status.

Precondition: The database is opened as writable.

Merged records are shown after the database is synchronized. Among records of the same key, synchronized existing records come first, added records come second, and unsynchronized existing records come last.

◆ ValidateRecords()

Status tkrzw::SkipDBM::ValidateRecords ( )

Validates records.

Returns
The result status.

◆ ReduceRemove()

static std::vector<std::string> tkrzw::SkipDBM::ReduceRemove ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by removing REMOVING_VALUE and past values.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vector containing surviving values.

◆ ReduceToFirst()

static std::vector<std::string> tkrzw::SkipDBM::ReduceToFirst ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by keeping the first value only.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vector containing the first value.

◆ ReduceToSecond()

static std::vector<std::string> tkrzw::SkipDBM::ReduceToSecond ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by keeping the second or first value only.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vector containing the first value.

◆ ReduceToLast()

static std::vector<std::string> tkrzw::SkipDBM::ReduceToLast ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by keeping the last value only.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vector containing the last value.

◆ ReduceConcat()

static std::vector<std::string> tkrzw::SkipDBM::ReduceConcat ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by concatenating all values.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vectro containing the concatenated values.

◆ ReduceConcatWithNull()

static std::vector<std::string> tkrzw::SkipDBM::ReduceConcatWithNull ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by concatenating all values with null code.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vectro containing the concatenated values.

◆ ReduceConcatWithTab()

static std::vector<std::string> tkrzw::SkipDBM::ReduceConcatWithTab ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by concatenating all values with tab.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vectro containing the concatenated values.

◆ ReduceConcatWithLine()

static std::vector<std::string> tkrzw::SkipDBM::ReduceConcatWithLine ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by concatenating all values with linefeed.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vectro containing the concatenated values.

◆ ReduceToTotal()

static std::vector<std::string> tkrzw::SkipDBM::ReduceToTotal ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by totaling numeric expressions.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vector containing the total numeric expression.

◆ ReduceToTotalBigEndian()

static std::vector<std::string> tkrzw::SkipDBM::ReduceToTotalBigEndian ( const std::string &  key,
const std::vector< std::string > &  values 
)
static

Reduces the values of records of the same key by totaling big-endian binary expressions.

Parameters
keyThe common key of the records.
valuesThe values of the records of the same key.
Returns
A vector containing the total big-endian binary expression.

◆ ReadMetadata()

static Status tkrzw::SkipDBM::ReadMetadata ( File file,
int32_t *  cyclic_magic,
int32_t *  pkg_major_version,
int32_t *  pkg_minor_version,
int32_t *  offset_width,
int32_t *  step_unit,
int32_t *  max_level,
int32_t *  closure_flags,
int64_t *  num_records,
int64_t *  eff_data_size,
int64_t *  file_size,
int64_t *  mod_time,
int32_t *  db_type,
std::string *  opaque 
)
static

Reads metadata from a database file.

Parameters
fileA file object having opened the database file.
cyclic_magicThe pointer to a variable to store the cyclic magic data.
pkg_major_versionThe pointer to a variable to store the package major version.
pkg_minor_versionThe pointer to a variable to store the package minor version.
offset_widthThe pointer to a variable to store the offset width.
step_unitThe pointer to a variable to store the step unit.
max_levelThe pointer to a variable to store the max level.
closure_flagsThe pointer to a variable to store the closure flags.
num_recordsThe pointer to a variable to store the number of records.
eff_data_sizeThe pointer to a variable to store the effective data size.
file_sizeThe pointer to a variable to store the file size.
mod_timeThe pointer to a variable to store the last modified time.
db_typeThe pointer to a variable to store the database type.
opaqueThe pointer to a variable to store the opaque data.
Returns
The result status.

If the leading magic data is inconsistent, it returns failure. If the cyclic magic data is inconsistent, it returns success and -1 is assigned to cyclic_magic.

◆ RestoreDatabase()

static Status tkrzw::SkipDBM::RestoreDatabase ( const std::string &  old_file_path,
const std::string &  new_file_path 
)
static

Restores a broken database as a new healthy database.

Parameters
old_file_pathThe path of the broken database.
new_file_pathThe path of the new database to be created.
Returns
The result status.

Member Data Documentation

◆ DEFAULT_OFFSET_WIDTH

constexpr int32_t tkrzw::SkipDBM::DEFAULT_OFFSET_WIDTH = 4
staticconstexpr

The default value of the offset width.

◆ DEFAULT_STEP_UNIT

constexpr int32_t tkrzw::SkipDBM::DEFAULT_STEP_UNIT = 4
staticconstexpr

The default value of the step unit.

◆ DEFAULT_MAX_LEVEL

constexpr int32_t tkrzw::SkipDBM::DEFAULT_MAX_LEVEL = 14
staticconstexpr

The default value of the maximum level.

◆ DEFAULT_SORT_MEM_SIZE

constexpr int64_t tkrzw::SkipDBM::DEFAULT_SORT_MEM_SIZE = 256LL << 20
staticconstexpr

The default value of the memory size used for sorting.

◆ DEFAULT_MAX_CACHED_RECORDS

constexpr int32_t tkrzw::SkipDBM::DEFAULT_MAX_CACHED_RECORDS = 65536
staticconstexpr

The default value of the maximum cached records.

◆ OPAQUE_METADATA_SIZE

constexpr int32_t tkrzw::SkipDBM::OPAQUE_METADATA_SIZE = 64
staticconstexpr

The size of the opaque metadata.

◆ REMOVING_VALUE

const std::string tkrzw::SkipDBM::REMOVING_VALUE
static

The special removing value.