Tkrzw
Classes | Public Member Functions | List of all members
tkrzw::MemoryMapParallelFile Class Referencefinal

File implementation by memory mapping and locking for parallel operations. More...

#include <tkrzw_file_mmap.h>

Classes

class  Zone
 Structure to make a shared section where a region can be accessed. More...
 

Public Member Functions

 MemoryMapParallelFile ()
 Default constructor. More...
 
 ~MemoryMapParallelFile ()
 Destructor. More...
 
 MemoryMapParallelFile (const MemoryMapParallelFile &rhs)=delete
 Copy and assignment are disabled. More...
 
MemoryMapParallelFileoperator= (const MemoryMapParallelFile &rhs)=delete
 
Status Open (const std::string &path, bool writable, int32_t options=OPEN_DEFAULT) override
 Opens a file. More...
 
Status Close () override
 Closes the file. More...
 
Status MakeZone (bool writable, int64_t off, size_t size, std::unique_ptr< Zone > *zone)
 Makes an accessible zone. More...
 
Status Read (int64_t off, void *buf, size_t size) override
 Reads data. More...
 
std::string ReadSimple (int64_t off, size_t size) override
 Reads data, in a simple way. More...
 
Status Write (int64_t off, const void *buf, size_t size) override
 Writes data. More...
 
Status Append (const void *buf, size_t size, int64_t *off=nullptr) override
 Appends data at the end of the file. More...
 
Status Expand (size_t inc_size, int64_t *old_size=nullptr) override
 Expands the file size without writing data. More...
 
Status Truncate (int64_t size) override
 Truncates the file. More...
 
Status TruncateFakely (int64_t size) override
 Truncate the file fakely. More...
 
Status Synchronize (bool hard, int64_t off=0, int64_t size=0) override
 Synchronizes the content of the file to the file system. More...
 
Status GetSize (int64_t *size) override
 Gets the size of the file. More...
 
Status SetAllocationStrategy (int64_t init_size, double inc_factor) override
 Sets allocation strategy. More...
 
Status CopyProperties (File *file) override
 Copies internal properties to another file object. More...
 
Status GetPath (std::string *path) override
 Gets the path of the file. More...
 
Status Rename (const std::string &new_path) override
 Renames the file. More...
 
Status DisablePathOperations () override
 Disables operations related to the path. More...
 
bool IsOpen () const override
 Checks whether the file is open. More...
 
bool IsMemoryMapping () const override
 Checks whether operations are done by memory mapping. More...
 
bool IsAtomic () const override
 Checks whether updating operations are atomic and thread-safe. More...
 
std::unique_ptr< FileMakeFile () const override
 Makes a new file object of the same concrete class. More...
 
- Public Member Functions inherited from tkrzw::MemoryMapFile
virtual ~MemoryMapFile ()=default
 Destructor. More...
 
- Public Member Functions inherited from tkrzw::File
virtual ~File ()=default
 Destructor. More...
 
virtual bool WriteSimple (int64_t off, std::string_view data)
 Writes data, in a simple way. More...
 
virtual int64_t AppendSimple (const std::string &data)
 Appends data at the end of the file, in a simple way. More...
 
virtual int64_t ExpandSimple (size_t inc_size)
 Expands the file size without writing data, in a simple way. More...
 
virtual int64_t GetSizeSimple ()
 Gets the size of the file, in a simple way. More...
 
virtual std::string GetPathSimple ()
 Gets the path of the file, in a simple way. More...
 
const std::type_info & GetType () const
 Gets the type information of the actual class. More...
 

Additional Inherited Members

- Public Types inherited from tkrzw::File
enum  OpenOption : int32_t {
  OPEN_DEFAULT = 0 , OPEN_TRUNCATE = 1 << 0 , OPEN_NO_CREATE = 1 << 1 , OPEN_NO_WAIT = 1 << 2 ,
  OPEN_NO_LOCK = 1 << 3 , OPEN_SYNC_HARD = 1 << 4
}
 Enumeration of options for Open. More...
 
- Static Public Attributes inherited from tkrzw::File
static constexpr int64_t DEFAULT_ALLOC_INIT_SIZE = 1LL << 20
 The default value of the initial allocation size. More...
 
static constexpr double DEFAULT_ALLOC_INC_FACTOR = 2.0
 The default value of the allocation increment factor. More...
 

Detailed Description

File implementation by memory mapping and locking for parallel operations.

Reading and writing operations are thread-safe; Multiple threads can access the same file concurrently. Other operations including Open, Close, Truncate, and Synchronize are not thread-safe. Moreover, locking doesn't assure atomicity of reading and writing operations.

Constructor & Destructor Documentation

◆ MemoryMapParallelFile() [1/2]

tkrzw::MemoryMapParallelFile::MemoryMapParallelFile ( )

Default constructor.

◆ ~MemoryMapParallelFile()

tkrzw::MemoryMapParallelFile::~MemoryMapParallelFile ( )

Destructor.

◆ MemoryMapParallelFile() [2/2]

tkrzw::MemoryMapParallelFile::MemoryMapParallelFile ( const MemoryMapParallelFile rhs)
explicitdelete

Copy and assignment are disabled.

Member Function Documentation

◆ Open()

Status tkrzw::MemoryMapParallelFile::Open ( const std::string &  path,
bool  writable,
int32_t  options = OPEN_DEFAULT 
)
overridevirtual

Opens a file.

Parameters
pathA path of the file.
writableIf true, the file is writable. If false, it is read-only.
optionsBit-sum options of File::OpenOption enums.
Returns
The result status.

By default, exclusive locking against other processes is done for a writer and shared locking against other processes is done for a reader.

Implements tkrzw::File.

◆ Close()

Status tkrzw::MemoryMapParallelFile::Close ( )
overridevirtual

Closes the file.

Returns
The result status.

Implements tkrzw::File.

◆ MakeZone()

Status tkrzw::MemoryMapParallelFile::MakeZone ( bool  writable,
int64_t  off,
size_t  size,
std::unique_ptr< Zone > *  zone 
)

Makes an accessible zone.

Parameters
writableIf true, the region is for reading and writing. If false, the region is only for reading.
offThe offset of the region to access.
sizeThe size of the region to access.
zoneA unique pointer to own the writable zone.
Returns
The result status.

If the writable flag is true and the region is out of the current file size, the file is expanded. If the writable flag is true and the offset is negative, the offset is set at the end of the file. If the writable flag is false and the region is out of the current file size, the size of the region is fitted to the file size or the operation fails.

◆ Read()

Status tkrzw::MemoryMapParallelFile::Read ( int64_t  off,
void *  buf,
size_t  size 
)
overridevirtual

Reads data.

Parameters
offThe offset of a source region.
bufThe pointer to the destination buffer.
sizeThe size of the data to be read.
Returns
The result status.

Implements tkrzw::File.

◆ ReadSimple()

std::string tkrzw::MemoryMapParallelFile::ReadSimple ( int64_t  off,
size_t  size 
)
overridevirtual

Reads data, in a simple way.

Parameters
offThe offset of a source region.
sizeThe size of the data to be read.
Returns
A string of the read data. It is empty on failure.

Reimplemented from tkrzw::File.

◆ Write()

Status tkrzw::MemoryMapParallelFile::Write ( int64_t  off,
const void *  buf,
size_t  size 
)
overridevirtual

Writes data.

Parameters
offThe offset of the destination region.
bufThe pointer to the source buffer.
sizeThe size of the data to be written.
Returns
The result status.

Implements tkrzw::File.

◆ Append()

Status tkrzw::MemoryMapParallelFile::Append ( const void *  buf,
size_t  size,
int64_t *  off = nullptr 
)
overridevirtual

Appends data at the end of the file.

Parameters
bufThe pointer to the source buffer.
sizeThe size of the data to be written.
offThe pointer to an integer object to contain the offset at which the data has been put. If it is nullptr, it is ignored.
Returns
The result status.

Implements tkrzw::File.

◆ Expand()

Status tkrzw::MemoryMapParallelFile::Expand ( size_t  inc_size,
int64_t *  old_size = nullptr 
)
overridevirtual

Expands the file size without writing data.

Parameters
inc_sizeThe size to increment the file size by.
old_sizeThe pointer to an integer object to contain the old size of the file. If it is nullptr, it is ignored.
Returns
The result status.

Implements tkrzw::File.

◆ Truncate()

Status tkrzw::MemoryMapParallelFile::Truncate ( int64_t  size)
overridevirtual

Truncates the file.

Parameters
sizeThe new size of the file.
Returns
The result status.

Implements tkrzw::File.

◆ TruncateFakely()

Status tkrzw::MemoryMapParallelFile::TruncateFakely ( int64_t  size)
overridevirtual

Truncate the file fakely.

Parameters
sizeThe new size of the file.
Returns
The result status.

This doesn't modify the actual file but modifies the internal length parameter, which affects behavior of Close, Synchronize, Append, Expand, and GetSize. If the specified size is more than the actual file size, the operation fails.

Implements tkrzw::File.

◆ Synchronize()

Status tkrzw::MemoryMapParallelFile::Synchronize ( bool  hard,
int64_t  off = 0,
int64_t  size = 0 
)
overridevirtual

Synchronizes the content of the file to the file system.

Parameters
hardTrue to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
offThe offset of the region to be synchronized.
sizeThe size of the region to be synchronized. If it is zero, the length to the end of file is specified.
Returns
The result status.

The pysical file size can be larger than the logical size in order to improve performance by reducing frequency of allocation. Thus, you should call this function before accessing the file with external tools.

Implements tkrzw::File.

◆ GetSize()

Status tkrzw::MemoryMapParallelFile::GetSize ( int64_t *  size)
overridevirtual

Gets the size of the file.

Parameters
sizeThe pointer to an integer object to contain the result size.
Returns
The result status.

Implements tkrzw::File.

◆ SetAllocationStrategy()

Status tkrzw::MemoryMapParallelFile::SetAllocationStrategy ( int64_t  init_size,
double  inc_factor 
)
overridevirtual

Sets allocation strategy.

Parameters
init_sizeAn initial size of allocation.
inc_factorA factor to increase the size of allocation.
Returns
The result status.

By default, the initial size is 1MB and the increasing factor is 2. Note that a memory map cannot be empty and its size must be aligned to the page size. This adjustment is done implicitly. This method must be called before the file is opened.

Implements tkrzw::File.

◆ CopyProperties()

Status tkrzw::MemoryMapParallelFile::CopyProperties ( File file)
overridevirtual

Copies internal properties to another file object.

Parameters
fileThe other file object.
Returns
The result status.

Implements tkrzw::File.

◆ GetPath()

Status tkrzw::MemoryMapParallelFile::GetPath ( std::string *  path)
overridevirtual

Gets the path of the file.

Parameters
pathThe pointer to a string object to store the path.
Returns
The result status.

Implements tkrzw::File.

◆ Rename()

Status tkrzw::MemoryMapParallelFile::Rename ( const std::string &  new_path)
overridevirtual

Renames the file.

Parameters
new_pathA new path of the file.
Returns
The result status.

Implements tkrzw::File.

◆ DisablePathOperations()

Status tkrzw::MemoryMapParallelFile::DisablePathOperations ( )
overridevirtual

Disables operations related to the path.

Returns
The result status.

This should be called if the file is overwritten by external operations.

Implements tkrzw::File.

◆ IsOpen()

bool tkrzw::MemoryMapParallelFile::IsOpen ( ) const
overridevirtual

Checks whether the file is open.

Returns
True if the file is open, or false if not.

Implements tkrzw::File.

◆ IsMemoryMapping()

bool tkrzw::MemoryMapParallelFile::IsMemoryMapping ( ) const
overridevirtual

Checks whether operations are done by memory mapping.

Returns
Always true. This is fast, but the file size cannot exceed the virtual memory.

Implements tkrzw::File.

◆ IsAtomic()

bool tkrzw::MemoryMapParallelFile::IsAtomic ( ) const
overridevirtual

Checks whether updating operations are atomic and thread-safe.

Returns
Always false. Atomicity is not assured. Some operations are not thread-safe. are not.

Implements tkrzw::File.

◆ MakeFile()

std::unique_ptr<File> tkrzw::MemoryMapParallelFile::MakeFile ( ) const
overridevirtual

Makes a new file object of the same concrete class.

Returns
The new file object.

Implements tkrzw::File.