File implementation by memory mapping and locking for parallel operations. More...

#include <tkrzw_file_mmap.h>

Classes
class	Zone
	Structure to make a shared section where a region can be accessed. More...

Public Member Functions
	MemoryMapParallelFile ()
	Default constructor. More...

	~MemoryMapParallelFile ()
	Destructor. More...

	MemoryMapParallelFile (const MemoryMapParallelFile &rhs)=delete
	Copy and assignment are disabled. More...

MemoryMapParallelFile &	operator= (const MemoryMapParallelFile &rhs)=delete

Status	Open (const std::string &path, bool writable, int32_t options=OPEN_DEFAULT) override
	Opens a file. More...

Status	Close () override
	Closes the file. More...

Status	MakeZone (bool writable, int64_t off, size_t size, std::unique_ptr< Zone > *zone)
	Makes an accessible zone. More...

Status	Read (int64_t off, void *buf, size_t size) override
	Reads data. More...

std::string	ReadSimple (int64_t off, size_t size) override
	Reads data, in a simple way. More...

Status	Write (int64_t off, const void *buf, size_t size) override
	Writes data. More...

Status	Append (const void buf, size_t size, int64_t off=nullptr) override
	Appends data at the end of the file. More...

Status	Expand (size_t inc_size, int64_t *old_size=nullptr) override
	Expands the file size without writing data. More...

Status	Truncate (int64_t size) override
	Truncates the file. More...

Status	TruncateFakely (int64_t size) override
	Truncate the file fakely. More...

Status	Synchronize (bool hard, int64_t off=0, int64_t size=0) override
	Synchronizes the content of the file to the file system. More...

Status	GetSize (int64_t *size) override
	Gets the size of the file. More...

Status	SetAllocationStrategy (int64_t init_size, double inc_factor) override
	Sets allocation strategy. More...

Status	CopyProperties (File *file) override
	Copies internal properties to another file object. More...

Status	GetPath (std::string *path) override
	Gets the path of the file. More...

Status	Rename (const std::string &new_path) override
	Renames the file. More...

Status	DisablePathOperations () override
	Disables operations related to the path. More...

bool	IsOpen () const override
	Checks whether the file is open. More...

bool	IsMemoryMapping () const override
	Checks whether operations are done by memory mapping. More...

bool	IsAtomic () const override
	Checks whether updating operations are atomic and thread-safe. More...

std::unique_ptr< File >	MakeFile () const override
	Makes a new file object of the same concrete class. More...

Public Member Functions inherited from tkrzw::MemoryMapFile
virtual	~MemoryMapFile ()=default
	Destructor. More...

Public Member Functions inherited from tkrzw::File
virtual	~File ()=default
	Destructor. More...

virtual bool	WriteSimple (int64_t off, std::string_view data)
	Writes data, in a simple way. More...

virtual int64_t	AppendSimple (const std::string &data)
	Appends data at the end of the file, in a simple way. More...

virtual int64_t	ExpandSimple (size_t inc_size)
	Expands the file size without writing data, in a simple way. More...

virtual int64_t	GetSizeSimple ()
	Gets the size of the file, in a simple way. More...

virtual std::string	GetPathSimple ()
	Gets the path of the file, in a simple way. More...

const std::type_info &	GetType () const
	Gets the type information of the actual class. More...

Additional Inherited Members
Public Types inherited from tkrzw::File
enum	OpenOption : int32_t { OPEN_DEFAULT = 0 , OPEN_TRUNCATE = 1 << 0 , OPEN_NO_CREATE = 1 << 1 , OPEN_NO_WAIT = 1 << 2 , OPEN_NO_LOCK = 1 << 3 , OPEN_SYNC_HARD = 1 << 4 }
	Enumeration of options for Open. More...

Static Public Attributes inherited from tkrzw::File
static constexpr int64_t	DEFAULT_ALLOC_INIT_SIZE = 1LL << 20
	The default value of the initial allocation size. More...

static constexpr double	DEFAULT_ALLOC_INC_FACTOR = 2.0
	The default value of the allocation increment factor. More...

Detailed Description

File implementation by memory mapping and locking for parallel operations.

Reading and writing operations are thread-safe; Multiple threads can access the same file concurrently. Other operations including Open, Close, Truncate, and Synchronize are not thread-safe. Moreover, locking doesn't assure atomicity of reading and writing operations.

Constructor & Destructor Documentation

◆ MemoryMapParallelFile() [1/2]

tkrzw::MemoryMapParallelFile::MemoryMapParallelFile ( )

Default constructor.

◆ ~MemoryMapParallelFile()

tkrzw::MemoryMapParallelFile::~MemoryMapParallelFile ( )

Destructor.

◆ MemoryMapParallelFile() [2/2]

tkrzw::MemoryMapParallelFile::MemoryMapParallelFile ( const MemoryMapParallelFile & rhs )

explicitdelete

Copy and assignment are disabled.

Member Function Documentation

◆ Open()

Status tkrzw::MemoryMapParallelFile::Open	(	const std::string &	path,
		bool	writable,
		int32_t	options = `OPEN_DEFAULT`
	)

overridevirtual

Opens a file.

Parameters

path	A path of the file.
writable	If true, the file is writable. If false, it is read-only.
options	Bit-sum options of File::OpenOption enums.

Returns: The result status.

By default, exclusive locking against other processes is done for a writer and shared locking against other processes is done for a reader.

Implements tkrzw::File.

◆ Close()

Status tkrzw::MemoryMapParallelFile::Close ( )

overridevirtual

Closes the file.

Returns: The result status.

Implements tkrzw::File.

◆ MakeZone()

Status tkrzw::MemoryMapParallelFile::MakeZone	(	bool	writable,
		int64_t	off,
		size_t	size,
		std::unique_ptr< Zone > *	zone
	)

Makes an accessible zone.

Parameters

writable	If true, the region is for reading and writing. If false, the region is only for reading.
off	The offset of the region to access.
size	The size of the region to access.
zone	A unique pointer to own the writable zone.

Returns: The result status.

If the writable flag is true and the region is out of the current file size, the file is expanded. If the writable flag is true and the offset is negative, the offset is set at the end of the file. If the writable flag is false and the region is out of the current file size, the size of the region is fitted to the file size or the operation fails.

◆ Read()

Status tkrzw::MemoryMapParallelFile::Read	(	int64_t	off,
		void *	buf,
		size_t	size
	)

overridevirtual

Reads data.

Parameters

off	The offset of a source region.
buf	The pointer to the destination buffer.
size	The size of the data to be read.

Returns: The result status.

Implements tkrzw::File.

◆ ReadSimple()

std::string tkrzw::MemoryMapParallelFile::ReadSimple	(	int64_t	off,
		size_t	size
	)

overridevirtual

Reads data, in a simple way.

Parameters

off	The offset of a source region.
size	The size of the data to be read.

Returns: A string of the read data. It is empty on failure.

Reimplemented from tkrzw::File.

◆ Write()

Status tkrzw::MemoryMapParallelFile::Write	(	int64_t	off,
		const void *	buf,
		size_t	size
	)

overridevirtual

Writes data.

Parameters

off	The offset of the destination region.
buf	The pointer to the source buffer.
size	The size of the data to be written.

Returns: The result status.

Implements tkrzw::File.

◆ Append()

Status tkrzw::MemoryMapParallelFile::Append	(	const void *	buf,
		size_t	size,
		int64_t *	off = `nullptr`
	)

overridevirtual

Appends data at the end of the file.

Parameters

buf	The pointer to the source buffer.
size	The size of the data to be written.
off	The pointer to an integer object to contain the offset at which the data has been put. If it is nullptr, it is ignored.

Returns: The result status.

Implements tkrzw::File.

◆ Expand()

Status tkrzw::MemoryMapParallelFile::Expand	(	size_t	inc_size,
		int64_t *	old_size = `nullptr`
	)

overridevirtual

Expands the file size without writing data.

Parameters

inc_size	The size to increment the file size by.
old_size	The pointer to an integer object to contain the old size of the file. If it is nullptr, it is ignored.

Returns: The result status.

Implements tkrzw::File.

◆ Truncate()

Status tkrzw::MemoryMapParallelFile::Truncate ( int64_t size )

overridevirtual

Truncates the file.

Parameters

size	The new size of the file.

Returns: The result status.

Implements tkrzw::File.

◆ TruncateFakely()

Status tkrzw::MemoryMapParallelFile::TruncateFakely ( int64_t size )

overridevirtual

Truncate the file fakely.

Parameters

size	The new size of the file.

Returns: The result status.

This doesn't modify the actual file but modifies the internal length parameter, which affects behavior of Close, Synchronize, Append, Expand, and GetSize. If the specified size is more than the actual file size, the operation fails.

Implements tkrzw::File.

◆ Synchronize()

Status tkrzw::MemoryMapParallelFile::Synchronize	(	bool	hard,
		int64_t	off = `0`,
		int64_t	size = `0`
	)

overridevirtual

Synchronizes the content of the file to the file system.

Parameters

hard	True to do physical synchronization with the hardware or false to do only logical synchronization with the file system.
off	The offset of the region to be synchronized.
size	The size of the region to be synchronized. If it is zero, the length to the end of file is specified.

Returns: The result status.

The pysical file size can be larger than the logical size in order to improve performance by reducing frequency of allocation. Thus, you should call this function before accessing the file with external tools.

Implements tkrzw::File.

◆ GetSize()

Status tkrzw::MemoryMapParallelFile::GetSize ( int64_t * size )

overridevirtual

Gets the size of the file.

Parameters

size	The pointer to an integer object to contain the result size.

Returns: The result status.

Implements tkrzw::File.

◆ SetAllocationStrategy()

Status tkrzw::MemoryMapParallelFile::SetAllocationStrategy	(	int64_t	init_size,
		double	inc_factor
	)

overridevirtual

Sets allocation strategy.

Parameters

init_size	An initial size of allocation.
inc_factor	A factor to increase the size of allocation.

Returns: The result status.

By default, the initial size is 1MB and the increasing factor is 2. Note that a memory map cannot be empty and its size must be aligned to the page size. This adjustment is done implicitly. This method must be called before the file is opened.

Implements tkrzw::File.

◆ CopyProperties()

Status tkrzw::MemoryMapParallelFile::CopyProperties ( File * file )

overridevirtual

Copies internal properties to another file object.

Parameters

file	The other file object.

Returns: The result status.

Implements tkrzw::File.

◆ GetPath()

Status tkrzw::MemoryMapParallelFile::GetPath ( std::string * path )

overridevirtual

Gets the path of the file.

Parameters

path	The pointer to a string object to store the path.

Returns: The result status.

Implements tkrzw::File.

◆ Rename()

Status tkrzw::MemoryMapParallelFile::Rename ( const std::string & new_path )

overridevirtual

Renames the file.

Parameters

new_path A new path of the file.

Returns: The result status.

Implements tkrzw::File.

◆ DisablePathOperations()

Status tkrzw::MemoryMapParallelFile::DisablePathOperations ( )

overridevirtual

Disables operations related to the path.

Returns: The result status.

This should be called if the file is overwritten by external operations.

Implements tkrzw::File.

◆ IsOpen()

bool tkrzw::MemoryMapParallelFile::IsOpen ( ) const

overridevirtual

Checks whether the file is open.

Returns: True if the file is open, or false if not.

Implements tkrzw::File.

◆ IsMemoryMapping()

bool tkrzw::MemoryMapParallelFile::IsMemoryMapping ( ) const

overridevirtual

Checks whether operations are done by memory mapping.

Returns: Always true. This is fast, but the file size cannot exceed the virtual memory.

Implements tkrzw::File.

◆ IsAtomic()

bool tkrzw::MemoryMapParallelFile::IsAtomic ( ) const

overridevirtual

Checks whether updating operations are atomic and thread-safe.

Returns: Always false. Atomicity is not assured. Some operations are not thread-safe. are not.

Implements tkrzw::File.

◆ MakeFile()

std::unique_ptr<File> tkrzw::MemoryMapParallelFile::MakeFile ( ) const

overridevirtual

Makes a new file object of the same concrete class.

Returns: The new file object.

Implements tkrzw::File.

Classes

Public Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ MemoryMapParallelFile() [1/2]

◆ ~MemoryMapParallelFile()

◆ MemoryMapParallelFile() [2/2]

Member Function Documentation

◆ Open()

◆ Close()

◆ MakeZone()

◆ Read()

◆ ReadSimple()

◆ Write()

◆ Append()

◆ Expand()

◆ Truncate()

◆ TruncateFakely()

◆ Synchronize()

◆ GetSize()

◆ SetAllocationStrategy()

◆ CopyProperties()

◆ GetPath()

◆ Rename()

◆ DisablePathOperations()

◆ IsOpen()

◆ IsMemoryMapping()

◆ IsAtomic()

◆ MakeFile()