Lua Binding of Tokyo Cabinet

Tokyo Cabinet: a modern implementation of DBM

INTRODUCTION

Tokyo Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Records are organized in hash table, B+ tree, or fixed-length array.

As for database of hash table, each key must be unique within a database, so it is impossible to store two or more records with a key overlaps. The following access methods are provided to the database: storing a record with a key and a value, deleting a record by a key, retrieving a record by a key. Moreover, traversal access to every key are provided, although the order is arbitrary. These access methods are similar to ones of DBM (or its followers: NDBM and GDBM) library defined in the UNIX standard. Tokyo Cabinet is an alternative for DBM because of its higher performance.

As for database of B+ tree, records whose keys are duplicated can be stored. Access methods of storing, deleting, and retrieving are provided as with the database of hash table. Records are stored in order by a comparison function assigned by a user. It is possible to access each record with the cursor in ascending or descending order. According to this mechanism, forward matching search for strings and range search for integers are realized.

As for database of fixed-length array, records are stored with unique natural numbers. It is impossible to store two or more records with a key overlaps. Moreover, the length of each record is limited by the specified length. Provided operations are the same as ones of hash database.

Table database is also provided as a variant of hash database. Each record is identified by the primary key and has a set of named columns. Although there is no concept of data schema, it is possible to search for records with complex conditions efficiently by using indices of arbitrary columns.

Such utility functions are also provided as encoding/decoding, pack/unpack, and file system operations.

Setting

Install the latest version of Tokyo Cabinet beforehand and get the package of the Lua binding of Tokyo Cabinet.

Enter the directory of the extracted package then perform installation.

./configure
make
su
make install

The package `tokyocabinet' should be loaded in each source file of application programs.

require("tokyocabinet")

EXAMPLE

The following code is an example to use a hash database.

require("tokyocabinet")

-- create the object
hdb = tokyocabinet.hdbnew()

-- open the database
if not hdb:open("casket.tch", hdb.OWRITER + hdb.OCREAT) then
   ecode = hdb:ecode()
   print("open error: " .. hdb:errmsg(ecode))
end

-- store records
if not hdb:put("foo", "hop")
   or not hdb:put("bar", "step")
   or not hdb:put("baz", "jump") then
   ecode = hdb:ecode()
   print("put error: " .. hdb:errmsg(ecode))
end

-- retrieve records
value = hdb:get("foo")
if value then
   print(value)
else
   ecode = hdb:ecode()
   print("get error: " .. hdb:errmsg(ecode))
end

-- traverse records
hdb:iterinit()
while true do
   key = hdb:iternext()
   if not key then break end
   value = hdb:get(key)
   if value then
      print(key .. ":" .. value)
   end
end

-- table-like usage
hdb["quux"] = "touchdown"
print(hdb["quux"])
for key, value in hdb:pairs() do
   print(key .. ":" .. value)
end

-- close the database
if not hdb:close() then
   ecode = hdb:ecode()
   print("close error: " .. hdb:errmsg(ecode))
end

The following code is an example to use a B+ tree database.

require("tokyocabinet")

-- create the object
bdb = tokyocabinet.bdbnew()

-- open the database
if not bdb:open("casket.tcb", bdb.OWRITER + bdb.OCREAT) then
   ecode = bdb:ecode()
   print("open error: " .. bdb:errmsg(ecode))
end

-- store records
if not bdb:put("foo", "hop")
   or not bdb:put("bar", "step")
   or not bdb:put("baz", "jump") then
   ecode = bdb:ecode()
   print("put error: " .. bdb:errmsg(ecode))
end

-- retrieve records
value = bdb:get("foo")
if value then
   print(value)
else
   ecode = bdb:ecode()
   print("get error: " .. bdb:errmsg(ecode))
end

-- traverse records
cur = tokyocabinet.bdbcurnew(bdb)
cur:first()
while true do
   key = cur:key()
   if not key then break end
   value = cur:val()
   if value then
      print(key .. ":" .. value)
   end
   cur:next()
end

-- table-like usage
bdb["quux"] = "touchdown"
print(bdb["quux"])
for key, value in bdb:pairs() do
   print(key .. ":" .. value)
end

-- close the database
if not bdb:close() then
   ecode = bdb:ecode()
   print("close error: " .. bdb:errmsg(ecode))
end

The following code is an example to use a fixed-length database.

require("tokyocabinet")

-- create the object
fdb = tokyocabinet.fdbnew()

-- open the database
if not fdb:open("casket.tcf", fdb.OWRITER + fdb.OCREAT) then
   ecode = fdb:ecode()
   print("open error: " .. fdb:errmsg(ecode))
end

-- store records
if not fdb:put(1, "one")
   or not fdb:put(12, "twelve")
   or not fdb:put(144, "one forty four") then
   ecode = fdb:ecode()
   print("put error: " .. fdb:errmsg(ecode))
end

-- retrieve records
value = fdb:get(1)
if value then
   print(value)
else
   ecode = fdb:ecode()
   print("get error: " .. fdb:errmsg(ecode))
end

-- traverse records
fdb:iterinit()
while true do
   key = fdb:iternext()
   if not key then break end
   value = fdb:get(key)
   if value then
      print(key .. ":" .. value)
   end
end

-- table-like usage
fdb[1728] = "touchdown"
print(fdb[1728])
for key, value in fdb:pairs() do
   print(key .. ":" .. value)
end

-- close the database
if not fdb:close() then
   ecode = fdb:ecode()
   print("close error: " .. fdb:errmsg(ecode))
end

The following code is an example to use a table database.

-- create the object
tdb = tokyocabinet.tdbnew()

-- open the database
if not tdb:open("casket.tct", tdb.OWRITER + tdb.OCREAT) then
   ecode = tdb:ecode()
   print("open error: " .. tdb:errmsg(ecode))
end

-- store a record
pkey = tdb:genuid()
cols = { name = "mikio", age = "30", lang = "ja,en,c" }
if not tdb:put(pkey, cols) then
   ecode = tdb:ecode()
   print("get error: " .. tdb:errmsg(ecode))
end

-- store another record
cols = { name = "falcon", age = "31", lang = "ja", skill = "cook,blog" }
if not tdb:put("x12345", cols) then
   ecode = tdb:ecode()
   print("get error: " .. tdb.errmsg(ecode))
end

-- search for records
qry = tokyocabinet.tdbqrynew(tdb)
qry:addcond("age", qry.QCNUMGE, "20")
qry:addcond("lang", qry.QCSTROR, "ja,en")
qry:setorder("name", qry.QOSTRASC)
qry:setlimit(10)
res = qry:search()
for i = 1, #res do
   rcols = tdb:get(res[i])
   print("name:" .. rcols.name)
end

-- table-like usage
tdb.joker = { name = "ozma", lang = "en", skill = "song,dance" }
print(tdb.joker.name)
for pkey, cols in tdb:pairs() do
   print(pkey .. ":" .. cols.name)
end

-- close the database
if not tdb:close() then
   ecode = tdb:ecode()
   print("close error: " .. tdb:errmsg(ecode))
end

The following code is an example to use an abstract database.

require("tokyocabinet")

-- create the object
adb = tokyocabinet.adbnew()

-- open the database
if not adb:open("casket.tch") then
   ecode = adb:ecode()
   print("open error: " .. adb:errmsg(ecode))
end

-- store records
if not adb:put("foo", "hop")
   or not adb:put("bar", "step")
   or not adb:put("baz", "jump") then
   ecode = adb:ecode()
   print("put error: " .. adb:errmsg(ecode))
end

-- retrieve records
value = adb:get("foo")
if value then
   print(value)
else
   ecode = adb:ecode()
   print("get error: " .. adb:errmsg(ecode))
end

-- traverse records
adb:iterinit()
while true do
   key = adb:iternext()
   if not key then break end
   value = adb:get(key)
   if value then
      print(key .. ":" .. value)
   end
end

-- table-like usage
adb["quux"] = "touchdown"
print(adb["quux"])
for key, value in adb:pairs() do
   print(key .. ":" .. value)
end

-- close the database
if not adb:close() then
   ecode = adb:ecode()
   print("close error: " .. adb:errmsg(ecode))
end

LICENSE

Copyright (C) 2006-2010 FAL Labs
All rights reserved.

Tokyo Cabinet is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License or any later version. Tokyo Cabinet is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with Tokyo Cabinet; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.

Modules

tokyocabinet The Lua binding of Tokyo Cabinet.
Tokyo Cabinet Manual