Is LevelDB batching writes or is there something more interesting going on?
And that is exactly what LevelDB is doing: writing a log (sequential), and when the memorychunk is full, it is writing it to disk sorted (this is also sequential).
Yes, updates can be done in one atomic batch. Please correct me if I'm wrong, but I don't think Tokyo Cabinet allows it without Tokyo Tyrant.
You turn off write-through caching on disks when you run a database unless you are willing to accept corruption (which is worse than data loss) on power outage. And that's why you can't get acceptable write performance out of database without a battery-backed RAID controller (or something other kind of RAM-based write cache with a battery backup).
Here's a simple way to test # fsyncs/s (a.k.a. commit rate) on your system:
sysbench --test=fileio --file-fsync-freq=1 --file-num=1 \
--file-total-size=16384 --file-test-mode=rndwr run --max-time=10 \
| grep "Requests/sec"LevelDB is a persistence library.
That makes LevelDB the kind of thing you plug into membase to get the unique properties it has to offer (or at least for fun).
LevelDB is for if you need ordered data, and a more appropriate comparison would be against a B+\tree database.
LevelDB is slower with random reads, but that doesn't mean you shouldn't use it for unordered data - it's still quite fast.
http://www.oracle.com/technetwork/database/berkeleydb/overvi...
It's essentially drop-in compatible with SQLite, but with added concurrency and speed for most operations. (The concurrency addresses a major issue usually keeping SQLite as a prototyping/single-user-only option in web development.)
With LevelDB as a BSD-licensed alternative to BDB, I wonder:
(1) How would the LevelDB-vs-SQLite benchmarks change against SQLite+BDB backend?
(2) Could a SQLite fork with a LevelDB backend get a performance boost?
One thing I didn't get about SQL API for BDB, how does something like
select * from users where name!='tom'
Voldemort to LevelDB is what MySQL is to InnoDB: Voldemort is a distributed system that allows multiple engines to be plugged in. Mostly commonly, companies use either BerkeleyDB or MySQL as a storage engine. LinkedIn, Mendeley, EBay and others also use the read only storage engine, where the data is pre-built in Hadoop and loaded into Voldemort.
I am really excited about LevelDB: while there are higher priority projects on my plate right now, we'd very much like to see a LevelDB storage engine. If anyone is interested in contributing one, they're welcome.
The steps are:
1) Creating JNI bindings to LevelDB (or creating a .so version of LevelDB and creating JNA bindings)
2) Implementing the StorageEngine interface with the bindings, including passing in any configuration.
Here is an example of a third party InnoDB/Haildb storage engine for Voldemort:
That initial checkin: http://code.google.com/p/leveldb/source/detail?r=2
Previous discussion - http://news.ycombinator.com/item?id=2526032
Benchmarks vs Kyoto TreeDB and SQLite3 - http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html (discussion - http://news.ycombinator.com/item?id=2813061)
Benchmarks vs InnoDB - http://blog.basho.com/2011/07/01/Leveling-the-Field/
LevelDB sounds like something I would like to contribute to but if the reception is going to be chilly I won't bother, maybe pick up mongo or redis instead.
It's worth noting that the makefile includes options to build for iOS. I've successfully done it and my next iOS app will include LevelDB. Also worth noting, thanks to the iOS devices SSDs, it's much faster than with the traditional HDD machines.
Really excited about seeing IndexedDB run atop of this