From Fedora Project Wiki
No edit summary
No edit summary
 
(17 intermediate revisions by 2 users not shown)
Line 1: Line 1:
* '''[https://github.com/erthink/libmdbx libmdbx]'''
** '''Written in:''' C
** '''Wrappers for:'''C
** '''Packages available for:''' Not yet available in Fedora
** '''Latest release:''' 0.8.2 / 2020-07-6
** '''License:''' [https://github.com/erthink/libmdbx/blob/master/LICENSE The OpenLDAP Public License]
** '''Search key length:''' minimum 0, maximum ≈¼ pagesize (1300 bytes for default 4K pagesize, 21780 bytes for 64K pagesize). Keys could be more than 2 times longer than LMDB.
** '''Read-only access to DB:''' Supports concurrent readers.
** '''Supported architectures:'''
** '''Notes:''' libmdbx should be Compact and friendly for fully embedding. Only 25KLOC of C11, 64K x86 binary code, no internal threads neither processes, but implements a simplified variant of the Berkeley DB and dbm API.
* '''[https://symas.com/products/lightning-memory-mapped-database/ LMDB]'''
* '''[https://symas.com/products/lightning-memory-mapped-database/ LMDB]'''
** '''Written in:''' C
** '''Written in:''' C
** '''Wrappers for:''' C++, Java, Python, Perl, Ruby, Go, Erlang, PHP, Lua
** '''Wrappers for:''' C++, Java, Python, Perl, Ruby, Go, Erlang, PHP, Lua, .NET, Common Lisp, node.js, Haskell, Tcl
** '''Packages available for:''' Fedora 23+, EPEL6+
** '''Packages available for:''' Fedora, EPEL6+, RHEL8+
** '''Latest release:''' 0.9.18 / 2016-02-05
** '''Latest release:''' 0.9.18 / 2016-02-05
** '''License:''' [http://www.openldap.org/software/release/license.html OpenLDAP Public License]
** '''License:''' [http://www.openldap.org/software/release/license.html OpenLDAP Public License]
** '''Search key length:''' Maximum length of 1/3 of a page (~1365B)
** '''Search key length:''' Maximum length of 1/3 of a page (~1365B)
** '''Read-only access to DB:'''
** '''Read-only access to DB:''' The library is fully thread-aware and supports concurrent read/write access from multiple processes and threads. There is normally no pure read-only mode, since readers need write access to locks and lock file. [http://lmdb.tech/doc/ (docs)]
** '''Supported architectures:''' i386/x86-64/PowerPC/ARM
** '''Supported architectures:''' i386/x86-64/PowerPC/ARM
** '''Query keys larger than X:''' Using cursors: [http://lmdb.tech/doc/group__mdb.html#gga1206b2af8b95e7f6b0ef6b28708c9127af9feb0557c2954dbf7732eee5e1b59e7 MDB_SET_RANGE] - Position at first key greater than or equal to specified key.
** '''Query keys larger than X:''' Using cursors: [http://lmdb.tech/doc/group__mdb.html#gga1206b2af8b95e7f6b0ef6b28708c9127af9feb0557c2954dbf7732eee5e1b59e7 MDB_SET_RANGE] - Position at first key greater than or equal to specified key.
** '''Notes:''' Developed as a replacement for Berkeley DB in the OpenLDAP project - the API (while simplified) is very similar to BDB's. Optimized for read performance. B+Tree based.


* '''[http://fallabs.com/kyotocabinet/ Kyoto Cabinet]''' (replacing Tokyo Cabinet)
* '''[http://fallabs.com/kyotocabinet/ Kyoto Cabinet]''' (replacing Tokyo Cabinet)
** '''Written in:''' C++
** '''Written in:''' C++
** '''Wrappers for:''' C, Java, Python, Ruby, Perl, and Lua
** '''Wrappers for:''' C, Java, Python, Ruby, Perl, and Lua
** '''Packages available for:''' Fedora 23+, EPEL5+
** '''Packages available for:''' Fedora, EPEL5+
** '''Latest release:''' 1.2.76 / 2012-05-24
** '''Latest release:''' 1.2.76 / 2012-05-24
** '''License:''' [https://www.gnu.org/licenses/gpl.html GNU GPL]
** '''License:''' [https://www.gnu.org/licenses/gpl.html GNU GPL]
Line 20: Line 32:
** '''Supported architectures:''' Linux 2.6 and later (i386/x86-64/PowerPC/Alpha/SPARC)
** '''Supported architectures:''' Linux 2.6 and later (i386/x86-64/PowerPC/Alpha/SPARC)
** '''Query keys larger than X:''' DB supports [http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1PolyDB.html#a23cafa21e981567d1d3f82b437ebcd9d matching by regex], which should be possible to use for this kind of query. Other possibilities are [http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1PolyDB.html#ae1a3c5cf00e8b44d058e13bef372cc4f match by prefix] and [http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1PolyDB.html#af822c534ddc01b51a3b21ccd4bcdedea match by similarity].
** '''Query keys larger than X:''' DB supports [http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1PolyDB.html#a23cafa21e981567d1d3f82b437ebcd9d matching by regex], which should be possible to use for this kind of query. Other possibilities are [http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1PolyDB.html#ae1a3c5cf00e8b44d058e13bef372cc4f match by prefix] and [http://fallabs.com/kyotocabinet/api/classkyotocabinet_1_1PolyDB.html#af822c534ddc01b51a3b21ccd4bcdedea match by similarity].
** '''Notes:'''  Records are organized in hash table or B+ tree.
* '''[https://tdb.samba.org/ TDB]'''
** '''Written in:''' C
** '''Wrappers for:''' Python
** '''Packages available for:''' Fedora, RHEL6+
** '''Latest release:''' 1.3.10 2016-07-28
** '''License:''' [https://www.gnu.org/licenses/lgpl.html GNU LGPL]
** '''Search key length:''' No limit apart from the data type used for holding the key and data lengths (int).
** '''Read-only access to DB:''' Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock. Internal locks in place to provide the ability for mutliple writer access (on different parts of the DB).
** '''Supported architectures:''' i386/x86-64/ARM
** '''Query keys larger than X:'''
** '''Notes:'''  Records are organized in hash table.
* '''[http://www.gnu.org.ua/software/gdbm/ GDBM]'''
** '''Written in:''' C
** '''Wrappers for:'''
** '''Packages available for:''' Fedora, RHEL6+
** '''Latest release:''' 1.12 / 2016-05-16
** '''License:''' [https://www.gnu.org/licenses/gpl-3.0.en.html GNU GPLv3]
** '''Search key length:''' No limit apart from the data type used for holding the key and data lengths (int).
** '''Read-only access to DB:''' Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock. Can be disabled by the user to provide his own way of exclusion control.
** '''Supported architectures:''' i386/x86-64/ARM
** '''Query keys larger than X:'''
** '''Notes:'''  Records are organized in hash table.


* '''[http://leveldb.org/ LevelDB]'''
* '''[http://leveldb.org/ LevelDB]'''
** '''Written in:''' C++
** '''Written in:''' C++
** '''Wrappers for:''' Java, Go, Erlang, node.js
** '''Wrappers for:''' C, Java, Go, Erlang, node.js
** '''Packages available for:''' Fedora 23+, EPEL6+
** '''Packages available for:''' Fedora, EPEL6+
** '''Latest release:''' 1.18 / 2014-09-16
** '''Latest release:''' 1.18 / 2014-09-16
** '''License:''' [https://opensource.org/licenses/BSD-3-Clause 3-Clause BSD]
** '''License:''' [https://opensource.org/licenses/BSD-3-Clause 3-Clause BSD]
** '''Search key length:'''
** '''Search key length:''' No limit apart from the data type used for holding the key length (size_t)
** '''Read-only access to DB:'''
** '''Read-only access to DB:''' A database may only be opened by one process at a time. The leveldb implementation acquires a lock from the operating system to prevent misuse. Within a single process, the same leveldb::DB object may be safely shared by multiple concurrent threads. [https://rawgit.com/google/leveldb/master/doc/index.html (docs)]
** '''Supported architectures:''' i386/x86-64/PowerPC/ARM
** '''Supported architectures:''' i386/x86-64/PowerPC/ARM
** '''Query keys larger than X:'''
** '''Query keys larger than X:''' Native library's Iterator can Seek (Position at the first key in the source that at or past target) + reverse, node.js wrapper is able to use basic filters (greater than, lower than)
** '''Notes:''' Does not support transactions. Records are organized in an LSM tree.


* '''[http://rocksdb.org/ RocksDB]''' (Built on LevelDB)
* '''[http://rocksdb.org/ RocksDB]''' (Built on LevelDB)
** '''Written in:'''
** '''Written in:''' C++
** '''Wrappers for:'''
** '''Wrappers for:''' C, Java (In development)
** '''Packages available for:''' Not available in Fedora?
** '''Packages available for:''' Not yet available in Fedora
** '''Latest release:''' 4.9.0 / 2016-06-09
** '''Latest release:''' 4.9.0 / 2016-06-09
** '''License:''' [https://opensource.org/licenses/BSD-3-Clause 3-Clause BSD]
** '''License:''' [https://opensource.org/licenses/BSD-3-Clause 3-Clause BSD]
** '''Search key length:'''
** '''Search key length:''' No limit apart from the data type used for holding the key length (size_t)
** '''Read-only access to DB:'''
** '''Read-only access to DB:''' A database may only be opened by one process at a time. The rocksdb implementation acquires a lock from the operating system to prevent misuse. Within a single process, the same rocksdb::DB object may be safely shared by multiple concurrent threads. [https://github.com/facebook/rocksdb/wiki/Basic-Operations (docs)]
** '''Supported architectures:''' x86-64/PowerPC/ARM
** '''Supported architectures:''' x86-64/PowerPC/ARM
** '''Query keys larger than X:'''
** '''Query keys larger than X:''' Iterator can Seek (Position at the first key in the source that at or past target) + reverse
** '''Notes:''' Records are organized in an LSM tree.


* '''[http://unqlite.org/ UnQlite]'''
* '''[http://unqlite.org/ UnQlite]'''
Line 49: Line 88:
** '''Latest release:''' 1.1.6 / 2015-08-26
** '''Latest release:''' 1.1.6 / 2015-08-26
** '''License:''' [https://opensource.org/licenses/BSD-2-Clause 2-Clause BSD]
** '''License:''' [https://opensource.org/licenses/BSD-2-Clause 2-Clause BSD]
** '''Search key length:'''
** '''Search key length:''' No limit apart from the data type used for holding the key length (int). Data length limited by the data type unqlite_int64 (64b int).
** '''Read-only access to DB:'''
** '''Read-only access to DB:''' Supports concurrent readers. In order to be thread-safe, UnQLite must be compiled with the UNQLITE_ENABLE_THREADS directive defined.
** '''Supported architectures:'''
** '''Supported architectures:'''
** '''Query keys larger than X:'''
** '''Query keys larger than X:''' UNQLITE_CURSOR_MATCH_LE and UNQLITE_CURSOR_MATCH_GE when using [https://unqlite.org/c_api/unqlite_kv_cursor.html cursors]. This options have sense only if the underlying key/value storage subsystem support range search (i.e: B+Tree, R+Tree, etc.).
** '''Notes:''' UnQLite works with run-time interchangeable storage engines (i.e. Hash, B+Tree, R+Tree, LSM, etc.).


== Older (probably unmantained) projects ==
== Older (probably unmantained) projects ==
Line 60: Line 100:
** '''Written in:''' C
** '''Written in:''' C
** '''Wrappers for:''' Perl, Ruby, Java, and Lua
** '''Wrappers for:''' Perl, Ruby, Java, and Lua
** '''Packages available for:''' Fedora 23+, EPEL5+, RHEL6+
** '''Packages available for:''' Fedora, EPEL5+, RHEL6+
** '''Latest release:''' 1.4.48 / 2012-08-18
** '''Latest release:''' 1.4.48 / 2012-08-18
** '''License:''' [https://www.gnu.org/licenses/lgpl.html GNU LGPL]
** '''License:''' [https://www.gnu.org/licenses/lgpl.html GNU LGPL]
** '''Search key length:'''
** '''Search key length:''' No limit apart from the data type used for holding the key and data lengths (int).
** '''Read-only access to DB:'''
** '''Read-only access to DB:''' Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock.
** '''Supported architectures:''' Linux 2.4 and later (x86-32/x86-64/PowerPC/Alpha/SPARC)
** '''Supported architectures:''' Linux 2.4 and later (x86-32/x86-64/PowerPC/Alpha/SPARC)
** '''Query keys larger than X:'''
** '''Query keys larger than X:''' It is possible to access each record with the cursor in ascending or descending order. According to this mechanism, forward matching search for strings and range search for integers are realized.
** '''Notes:''' Records are organized in hash table, B+ tree, or fixed-length array.


* '''[http://fallabs.com/qdbm/spex.html QDBM]'''
* '''[http://fallabs.com/qdbm/ QDBM]'''
** '''Written in:''' C
** '''Written in:''' C
** '''Wrappers for:''' C++, Java, Perl, and Ruby
** '''Wrappers for:''' C++, Java, Perl, and Ruby
** '''Packages available for:''' Fedora 23+
** '''Packages available for:''' Fedora
** '''Latest release:''' 1.8.78 / 2007-03-07
** '''Latest release:''' 1.8.78 / 2007-03-07
** '''License:''' [https://www.gnu.org/licenses/lgpl.html GNU LGPL]
** '''License:''' [https://www.gnu.org/licenses/lgpl.html GNU LGPL]
** '''Search key length:'''
** '''Search key length:''' No limit apart from the data type used for holding the key and data lengths (int).
** '''Read-only access to DB:'''
** '''Read-only access to DB:''' Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock.
** '''Supported architectures:''' Linux (2.2, 2.4, 2.6) (IA32, IA64, AMD64, PA-RISC, Alpha, PowerPC, M68000, ARM)
** '''Supported architectures:''' Linux (2.2, 2.4, 2.6) (IA32, IA64, AMD64, PA-RISC, Alpha, PowerPC, M68000, ARM)
** '''Query keys larger than X:'''
** '''Query keys larger than X:''' It is possible to access each record with the cursor in ascending or descending order. According to this mechanism, forward matching search for strings and range search for integers are realized.
 
** '''Notes:''' Records are organized in hash table or B+ tree.
* '''NDBM'''
** '''Written in:'''
** '''Wrappers for:'''
** '''Packages available for:'''
** '''Latest release:'''
** '''License:'''
** '''Search key length:'''
** '''Read-only access to DB:'''
** '''Supported architectures:'''
** '''Query keys larger than X:'''
 
* '''GDBM'''
** '''Written in:'''
** '''Wrappers for:'''
** '''Packages available for:'''
** '''Latest release:'''
** '''License:'''
** '''Search key length:'''
** '''Read-only access to DB:'''
** '''Supported architectures:'''
** '''Query keys larger than X:'''

Latest revision as of 07:30, 3 August 2020

  • libmdbx
    • Written in: C
    • Wrappers for:C
    • Packages available for: Not yet available in Fedora
    • Latest release: 0.8.2 / 2020-07-6
    • License: The OpenLDAP Public License
    • Search key length: minimum 0, maximum ≈¼ pagesize (1300 bytes for default 4K pagesize, 21780 bytes for 64K pagesize). Keys could be more than 2 times longer than LMDB.
    • Read-only access to DB: Supports concurrent readers.
    • Supported architectures:
    • Notes: libmdbx should be Compact and friendly for fully embedding. Only 25KLOC of C11, 64K x86 binary code, no internal threads neither processes, but implements a simplified variant of the Berkeley DB and dbm API.
  • LMDB
    • Written in: C
    • Wrappers for: C++, Java, Python, Perl, Ruby, Go, Erlang, PHP, Lua, .NET, Common Lisp, node.js, Haskell, Tcl
    • Packages available for: Fedora, EPEL6+, RHEL8+
    • Latest release: 0.9.18 / 2016-02-05
    • License: OpenLDAP Public License
    • Search key length: Maximum length of 1/3 of a page (~1365B)
    • Read-only access to DB: The library is fully thread-aware and supports concurrent read/write access from multiple processes and threads. There is normally no pure read-only mode, since readers need write access to locks and lock file. (docs)
    • Supported architectures: i386/x86-64/PowerPC/ARM
    • Query keys larger than X: Using cursors: MDB_SET_RANGE - Position at first key greater than or equal to specified key.
    • Notes: Developed as a replacement for Berkeley DB in the OpenLDAP project - the API (while simplified) is very similar to BDB's. Optimized for read performance. B+Tree based.
  • Kyoto Cabinet (replacing Tokyo Cabinet)
    • Written in: C++
    • Wrappers for: C, Java, Python, Ruby, Perl, and Lua
    • Packages available for: Fedora, EPEL5+
    • Latest release: 1.2.76 / 2012-05-24
    • License: GNU GPL
    • Search key length: No limit from database itself (uses c++ std::string or combination of c-style string and its length).
    • Read-only access to DB: See http://fallabs.com/kyotocabinet/spex.html, part Sharing One database by Multiple Processes. TL;DR: One DB cannot be accessed by multiple processes, DB file is protected by reader-writer lock (which means it will probably require write access even for read-only operation).
    • Supported architectures: Linux 2.6 and later (i386/x86-64/PowerPC/Alpha/SPARC)
    • Query keys larger than X: DB supports matching by regex, which should be possible to use for this kind of query. Other possibilities are match by prefix and match by similarity.
    • Notes: Records are organized in hash table or B+ tree.
  • TDB
    • Written in: C
    • Wrappers for: Python
    • Packages available for: Fedora, RHEL6+
    • Latest release: 1.3.10 2016-07-28
    • License: GNU LGPL
    • Search key length: No limit apart from the data type used for holding the key and data lengths (int).
    • Read-only access to DB: Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock. Internal locks in place to provide the ability for mutliple writer access (on different parts of the DB).
    • Supported architectures: i386/x86-64/ARM
    • Query keys larger than X:
    • Notes: Records are organized in hash table.
  • GDBM
    • Written in: C
    • Wrappers for:
    • Packages available for: Fedora, RHEL6+
    • Latest release: 1.12 / 2016-05-16
    • License: GNU GPLv3
    • Search key length: No limit apart from the data type used for holding the key and data lengths (int).
    • Read-only access to DB: Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock. Can be disabled by the user to provide his own way of exclusion control.
    • Supported architectures: i386/x86-64/ARM
    • Query keys larger than X:
    • Notes: Records are organized in hash table.
  • LevelDB
    • Written in: C++
    • Wrappers for: C, Java, Go, Erlang, node.js
    • Packages available for: Fedora, EPEL6+
    • Latest release: 1.18 / 2014-09-16
    • License: 3-Clause BSD
    • Search key length: No limit apart from the data type used for holding the key length (size_t)
    • Read-only access to DB: A database may only be opened by one process at a time. The leveldb implementation acquires a lock from the operating system to prevent misuse. Within a single process, the same leveldb::DB object may be safely shared by multiple concurrent threads. (docs)
    • Supported architectures: i386/x86-64/PowerPC/ARM
    • Query keys larger than X: Native library's Iterator can Seek (Position at the first key in the source that at or past target) + reverse, node.js wrapper is able to use basic filters (greater than, lower than)
    • Notes: Does not support transactions. Records are organized in an LSM tree.
  • RocksDB (Built on LevelDB)
    • Written in: C++
    • Wrappers for: C, Java (In development)
    • Packages available for: Not yet available in Fedora
    • Latest release: 4.9.0 / 2016-06-09
    • License: 3-Clause BSD
    • Search key length: No limit apart from the data type used for holding the key length (size_t)
    • Read-only access to DB: A database may only be opened by one process at a time. The rocksdb implementation acquires a lock from the operating system to prevent misuse. Within a single process, the same rocksdb::DB object may be safely shared by multiple concurrent threads. (docs)
    • Supported architectures: x86-64/PowerPC/ARM
    • Query keys larger than X: Iterator can Seek (Position at the first key in the source that at or past target) + reverse
    • Notes: Records are organized in an LSM tree.
  • UnQlite
    • Written in: C
    • Wrappers for:
    • Packages available for: Not yet available in Fedora
    • Latest release: 1.1.6 / 2015-08-26
    • License: 2-Clause BSD
    • Search key length: No limit apart from the data type used for holding the key length (int). Data length limited by the data type unqlite_int64 (64b int).
    • Read-only access to DB: Supports concurrent readers. In order to be thread-safe, UnQLite must be compiled with the UNQLITE_ENABLE_THREADS directive defined.
    • Supported architectures:
    • Query keys larger than X: UNQLITE_CURSOR_MATCH_LE and UNQLITE_CURSOR_MATCH_GE when using cursors. This options have sense only if the underlying key/value storage subsystem support range search (i.e: B+Tree, R+Tree, etc.).
    • Notes: UnQLite works with run-time interchangeable storage engines (i.e. Hash, B+Tree, R+Tree, LSM, etc.).

Older (probably unmantained) projects

  • TokyoCabinet (replacing QDBM)
    • Written in: C
    • Wrappers for: Perl, Ruby, Java, and Lua
    • Packages available for: Fedora, EPEL5+, RHEL6+
    • Latest release: 1.4.48 / 2012-08-18
    • License: GNU LGPL
    • Search key length: No limit apart from the data type used for holding the key and data lengths (int).
    • Read-only access to DB: Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock.
    • Supported architectures: Linux 2.4 and later (x86-32/x86-64/PowerPC/Alpha/SPARC)
    • Query keys larger than X: It is possible to access each record with the cursor in ascending or descending order. According to this mechanism, forward matching search for strings and range search for integers are realized.
    • Notes: Records are organized in hash table, B+ tree, or fixed-length array.
  • QDBM
    • Written in: C
    • Wrappers for: C++, Java, Perl, and Ruby
    • Packages available for: Fedora
    • Latest release: 1.8.78 / 2007-03-07
    • License: GNU LGPL
    • Search key length: No limit apart from the data type used for holding the key and data lengths (int).
    • Read-only access to DB: Multiple processes can read a single database. Exclusion control is managed by a reader/writer lock.
    • Supported architectures: Linux (2.2, 2.4, 2.6) (IA32, IA64, AMD64, PA-RISC, Alpha, PowerPC, M68000, ARM)
    • Query keys larger than X: It is possible to access each record with the cursor in ascending or descending order. According to this mechanism, forward matching search for strings and range search for integers are realized.
    • Notes: Records are organized in hash table or B+ tree.
Retrieved from "https://fedoraproject.org/w/index.php?title=User:Pkubat/BerkeleyDB_alternatives&oldid=583999"