Lua Binding of Kyoto Cabinet.

Kyoto Cabinet is a straightforward implementation of DBM.

Introduction

Kyoto Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. Each key must be unique within a database. There is neither concept of data tables nor data types. Records are organized in hash table or B+ tree.

The following access methods are provided to the database: storing a record with a key and a value, deleting a record by a key, retrieving a record by a key. Moreover, traversal access to every key are provided. These access methods are similar to ones of the original DBM (and its followers: NDBM and GDBM) library defined in the UNIX standard. Kyoto Cabinet is an alternative for the DBM because of its higher performance.

Each operation of the hash database has the time complexity of "O(1)". Therefore, in theory, the performance is constant regardless of the scale of the database. In practice, the performance is determined by the speed of the main memory or the storage device. If the size of the database is less than the capacity of the main memory, the performance will seem on-memory speed, which is faster than std::map of STL. Of course, the database size can be greater than the capacity of the main memory and the upper limit is 8 exabytes. Even in that case, each operation needs only one or two seeking of the storage device.

Each operation of the B+ tree database has the time complexity of "O(log N)". Therefore, in theory, the performance is logarithmic to the scale of the database. Although the performance of random access of the B+ tree database is slower than that of the hash database, the B+ tree database supports sequential access in order of the keys, which realizes forward matching search for strings and range search for integers. The performance of sequential access is much faster than that of random access.

This library wraps the polymorphic database of the C++ API. So, you can select the internal data structure by specifying the database name in runtime. This library is thread-safe.

Installation

Install the latest version of Kyoto Cabinet beforehand and get the package of the Lua binding of Kyoto Cabinet. Lua 5.1 or later is also required.

Enter the directory of the extracted package then perform installation.

./configure
make
make check
su
make install

When a series of work finishes, the shared object file `kyotocabinet.so' is installed under a directory in the library path of Lua.

Let the library search path include `/usr/local/lib'.

LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"
export LD_LIBRARY_PATH

The module `tokyocabinet' should be loaded in each source file of application programs.

kc = require("tokyocabinet")

All symbols of Kyoto Cabinet are defined in the table returned by the above "require" function.

Example

The following code is a typical example to use a database.

kc = require("kyotocabinet")

-- create the database object
db = kc.DB:new()

-- open the database
if not db:open("casket.kch", kc.DB.OWRITER + kc.DB.OCREATE) then
   print("open error: " .. tostring(db:error()))
end

-- store records
if not db:set("foo", "hop") or
   not db:set("bar", "step") or
   not db:set("baz", "jump") then
   print("set error: " .. tostring(db:error()))
end

-- retrieve records
value = db:get("foo")
if value then
   print(value)
else
   print("get error: " .. tostring(db:error()))
end

-- traverse records
cur = db:cursor()
cur:jump()
while true do
   local key, value = cur:get(true)
   if not key then break end
   print(key .. ":" .. value)
end
cur:disable()

-- close the database
if not db:close() then
   print("close error: " .. tostring(db:error()))
end

The following code is a more complex example, which uses the Visitor pattern.

kc = require("kyotocabinet")

-- create the database object
db = kc.DB:new()

-- open the database
if not db:open("casket.kch", kc.DB.OREADER) then
   print("open error: " .. tostring(db:error()))
end

-- define the visitor
VisitorImpl = {}
function VisitorImpl:new()
   local obj = {}
   function obj:visit_full(key, value)
      print(key .. ":" .. value)
      return kc.Visitor.NOP
   end
   function obj:visit_empty(key)
      print(key .. " is missing")
      return kc.Visitor.NOP
   end
   local base = kc.Visitor:new()
   setmetatable(obj, { __index = base})
   return obj
end
visitor = VisitorImpl:new()

-- retrieve a record with visitor
if not db:accept("foo", visitor, false) or
   not db:accept("dummy", visitor, false) then
   print("accept error: " .. tostring(db:error()))
end

-- traverse records with visitor
if not db:iterate(visitor, false) then
   print("iterate error: " .. tostring(db:error()))
end

-- close the database
if not db:close() then
   print("close error: " .. tostring(db:error()))
end

The following code is also a complex example, which is suited to the Lua style.

kc = require("kyotocabinet")

-- define the functor
function dbproc(db)

   -- store records
   db["foo"] = "hop"
   db["bar"] = "step"
   db[3] = "jump"

   -- update records in transaction
   function tranproc()
      db["foo"] = 2.71828
      return true
   end
   db:transaction(tranproc)

   -- multiply a record value
   function mulproc(key, value)
      return tonumber(value) * 2
   end
   db:accept("foo", mulproc)

   -- traverse records by iterator
   for key, value in db:pairs() do
      print(key .. ":" .. value)
   end

   -- upcase values by iterator
   function upproc(key, value)
      return string.upper(value)
   end
   db:iterate(upproc)

   -- traverse records by cursor
   function curproc(cur)
      cur:jump()
      function printproc(key, value)
         print(key .. ":" .. value)
         return Visitor.NOP
      end
      while cur:accept(printproc) do
         cur:step()
      end
   end
   db:cursor_process(curproc)

end

-- process the database by the functor
kc.DB:process(dbproc, "casket.kch")

The following code is an example of word counting with the MapReduce framework.

kc = require("kyotocabinet")

-- create the database object
db = kc.DB:new()

-- open the database
if not db:open() then
   print("open error: " .. tostring(db:error()))
end

-- store records
db["1"] = "this is a pen"
db["2"] = "what a beautiful pen this is"
db["3"] = "she is beautiful"

-- define the papper
function map(key, value, emit)
   local values = kc.split(value, " ")
   for i = 1, #values do
      if not emit(values[i], "") then
         return false
      end
   end
   return true
end

-- define the reducer
function reduce(key, iter)
   local count = 0
   while true do
      local value = iter()
      if not value then
         break
      end
      count = count + 1
   end
   print(key .. ": " .. count)
   return true
end

-- execute the MapReduce process
if not db:mapreduce(map, reduce) then
   print("mapreduce error: " .. tostring(db:error()))
end

-- close the database
if not db:close() then
   print("close error: " .. tostring(db:error()))
end

License

Copyright (C) 2009-2010 FAL Labs

Kyoto Cabinet is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or any later version.

Kyoto Cabinet is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Modules

kyotocabinet The Lua binding of Kyoto Cabinet.
Kyoto Cabinet Manual