Sophia – An embeddable key-value database (opens in new tab)

I know it seems superficial but beautiful docs are one of my most trusted heuristics when I'm considering using a library. If the author cares about the aesthetics of the docs, it often means they care about the aesthetics of the code, which really does matter a lot. We can make ugly things or we can make beautiful things. I really respect people who take the time to make beautiful tools.

Anyone know anything about how this compares in practice to Lightning MDB (which uses a memory mapped B-tree, I think, and is apparently insanely faster than most of the other enbedded key-value stores people normally examine)?

Having just gone through the exercise of picking an embedded key-value store for a project, some things that would be nice: how does it compare to other things besides leveldb (which, to be frank, isn't a stellar performer)? In particular, how does it compare to Tokyo/Kyoto Cabinet, Lightning MDB, or Sqlite4's LSM? Does it support data compression (either with a single pre-selected algorithm like LevelDB and Snappy, or in pluggable fashion like LSM)? How does it deal with concurrent access by multiple processes?

For Java / Android, I've been using H2's MVStore, which is log structured and uses counted B+-trees. It's nice not having to go through JNI for good performance in Java.

http://www.h2database.com/html/mvstore.html

Typography is hard to read.

Cute website. Some random thoughts:

Concurrency:

- No mention of it. There appear to be spin locks in the source. No multi-threaded tests.

Stability and data safety:

- Github has 2 days of history, and 4kLoC of test code. Why should I trust my data to you?

"high load environment":

- So what exactly does it do in a "high load environment"? How do you define "high load" in the first place? CPU load? I/O load from other processes? What shortcomings of the competition under a "high load environment" are you trying overcome?

Backup:

- How do I do hot backup?

Benchmark:

- LevelDB is not a fair comparison as it offers additional non-trivial functionality (snapshots) that cannot be built up on top of Sophia. LevelDB APIs are also safe for concurrent use, which adds overhead. Kyoto Cabinet would have been more suitable as a peer to benchmark with.

- 3 million records with 16-byte keys and 100-byte values is not really an interesting benchmark dataset.

- Iteration over a static database is not interesting, either. Is there any alternative other than locking an entire mutating database for the duration of iteration?

I, for one, wouldn't trust my data to a library by somebody who uses the same text decoration for hyperlinks as for plain text.

Really guys, can you give more constructive or at least more (not based in bullshit assumptions) comments? if not, just shut up..

This is a non-trivial effort, and all people do is to complain about the font face or if the punctuation was right?

First, in the benchmarks it just crush leveldb, this is already by itself a great achievement. can you confront the benchmarks? you do it one yourself with a different configuration? no?

Second, if you are not a database expert and can create proper critics (constructive or not), just keep it to yourself.. i wonder how so many people get up with all of this conclusions so fast, without a proper look at the source code and to have a reasonable amount of time to know what are they talking about.

its very hard to create things like this, but very easy to critisize without any background.. dont forget about it

if you have something to say about a small thing, that do not have a direct relation to the product or thing itself, if theres already one comment about it, that enough! do not spam, answering it, or creating new comments about it, this is just so rude and unrespectful..

really, things are getting creepy on HN.. and its not only in this thread

It says that the benchmark source is on github, but I can't find it.

It doesn't appear to be in their primary repo.

I would like to try and do my own test against another embedded data store like Berkeley DB but I want to know more about the conditions on the test. How many threads were used, that kind of thing.

Very cool. As a suggestion, increase the link size under the main title, wasn't clear to me what the next step was at first after reading the introduction text.

In case any of the devs read this:

1. Can multiple processes use the same database concurrently? (Separate address space processes, not fork()'d)

2. Have you tested this with uClib/cross compiler? (I would like to use it on a MIPs embedded router)

The reason I ask this is because I recently had the displeasure of having to hack a non-volatile RAM library to work with shared memory / thread safe and something small like this would be a perfect replacement with a lot less pain.

Was this man's computer use being charged by the key stroke? I mean I understand using a few abbreviations here and there, but i.c ? At least name your files descriptively.

I would avoid using this for realzies if only for the fact that if something broke trying to fix it in that code base would be prohibitive

Looks nice, but note this doesn't appear to support consistent reads (unlike LevelDB snapshots)

I love the simplicity of the site and the C code. I will definitely use it, thank you.

I've got a need for something like this, but would like to have the keys and values delta encoded to achieve simple, yet effective, compression.

Had a quick, 30 min bash at a win32 port using msinttypes and pthread-win32 but no luck yet :( Would love to see one though...

Something for Mozilla to consider using inside Firefox inplace of LevelDB ( If that was ever landed )

I'd be far more interested in benchmarks versus BDB (and maybe even sqlite).

Snapshots? I could find it in the documentation..

any word on support for unicode keys?

Awesome work, congratulations!