I benchmarked six Go SQLite drivers (opens in new tab)

(github.com)

192 pointscvilsmeier2y ago60 comments

60 comments

51 comments · 13 top-level

acatton2y ago· 19 in thread

In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled.

> modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go.

Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.com/cznic/ccgo

psanford2y ago

I cross compile mattn/go-sqlite3 a lot. It's about the easiest cgo project to cross compile since it has no external dependencies.

Just use zig as your c compiler and it all just works.

makeworld2y ago

Do you have any links for using Zig for this?

2 more replies

vbezhenar2y ago

Sqlite developers go to very great lengths making sure their software works reliably. You can't just introduce some transpiler in-between and expect that it'll work. It's a different software and it should not even be called sqlite IMO.

The only proper way to use sqlite is to use FFI.

arp2422y ago

> You can't just introduce some transpiler in-between and expect that it'll work

Obviously it "works" and they didn't "just" transpile it but spent quite a bit of effort on this. Whether it's free of bugs that are not present in SQLite is a different matter. And while it's not run against the SQLite proprietary tests, it does pass all of the TCL tests IIRC, which are quite comprehensive on their own, so there shouldn't be huge glaring bugs.

jchw2y ago

If the transpiler is not breaking any defined behavior in C, there is no reason to believe it cannot be a perfectly reasonable solution. There is nothing inherently wrong with targeting another programming language when compiling code, it does not mean there is anything wrong or broken with the generated code or its semantics.

It is a less battle-tested C compiler, sure, but it's a C compiler. IMO, you can't argue that something shouldn't be called sqlite because it contains a transpiled copy of sqlite. To me this isn't really substantially different from transpiling sqlite to JS using Emscripten.

shp0ngle2y ago

They run the whole testsuite on the transpiled code. I think.

But yeah it does give me a little shiver to think you are transpiling C code to go code, and the go-code is not really "pure go", but has a platform-dependent unsafe operations. Just look at the repo

https://gitlab.com/cznic/sqlite/-/tree/master/lib?ref_type=h...

mook2y ago

My understanding is that the full test suite is actually proprietary and it's part of how sqlite is funded — the maintainers can provide better claims. There is a public test suite but it's a subset of what they do.

https://www.sqlite.org/testing.html

shepherdjerred2y ago

We use go-sqlite3 at work and have had a positive experience, with the benefit of having fewer system dependencies and a simpler build process.

mariusor2y ago

My only complain so far about the package is that it required transpiled versions of the whole dependency chain for sqlite in order to get it working, from libc to tcl. The whole tree for the latest version is about 2G. This is not a trivial amount of traffic/space to use for every compilation.

Aissen2y ago

The biggest issue of the modernc approach is that it can't use the proprietary sqlite test suites and fuzzer to ensure that it works like the original: https://www.sqlite.org/testing.html

Aissen2y ago

(I just realized that probably goes for libsql as well)

shp0ngle2y ago

> Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled.

Which one do you mean? The WASM one? It includes WASM, it needs to be compiled too.

Ah github.com/cvilsmeier/sqinn-go. Which... is made by the same person that made this benchmark...?

edit: and that requires some random binary to be pre-installed...? Which is in C anyway?

https://github.com/cvilsmeier/sqinn-go

https://github.com/cvilsmeier/sqinn

so I don't see any "actually written in go".

ncruces2y ago

For the WASM one (disclaimer: I built it) it's at least reasonably simple to reproduce the build [1], and the resulting blob is fully cross platform.

Also, although I do (lightly) patch SQLite which might invalidate results in your view, the SQLite team is producing/testing/releasing WASM builds of SQLite built with much the same toolchain, which, hopefully, smokes out compiler bugs.

[1]: https://github.com/ncruces/go-sqlite3/blob/main/.github/work...

acatton2y ago

The modernc one is the one I was refering to.

https://gitlab.com/cznic/sqlite

shp0ngle2y ago

But that's not pure go, that's C transpiled to go, but even the transpilation is using OS specific and CPU-specific unsafe code.

tedunangst2y ago

Have you tested that the cross-compiled pure go sqlite runs, not just compiles, on every platform? (Spoiler: it does not.)

zombiezen2y ago

zombiezen.com/go/sqlite is built on top of modernc.org/sqlite, so also has the same properties.

alephnan2y ago

How come Zombiezen came out faster ? I’d expect the opposite

cvilsmeierOP2y ago

> How come Zombiezen came out faster ? I’d expect the opposite

(OP here) Honestly, me too. Maybe it has to do with leaving out that database/sql driver layer. I guess it would need another round of pprof to find out the real reason.

1 more reply

Traubenfuchs2y ago· 6 in thread

What a curious state, there is just one jdbc (=java) driver for SQLite. Why are there 6 (or more!) for Go?

eatonphil2y ago

If you search "topic:jdbc topic:sqlite" on GitHub [0] (and look closely, since most results are not relevant) you'll see there are more than one (e.g. [1]). xerial may be the defacto SQLite driver for Java. mattn/go-sqlite3 is the defacto SQLite driver for Go. But there are other options.

[0] https://github.com/search?q=topic%3Ajdbc%20topic%3Asqlite&ty...

[1] https://github.com/gwenn/sqlite-jna

morelisp2y ago

Aside from the threading mismatch,

- There is a version transpiled from C to Go. A transpilation from C to Java (or even the JVM) would be considerably more difficult, a naive translation would likely undergo a much larger performance hit. Maybe with Valhalla this will change.

- Modern Java deployment is not generally complicated by JNI/JNA (it's already that complicated to start with; Maven already wrangles some of it, although it's still a pain in many cases). Go deployments are simpler if no C linkage is involved.

- Some of these are not database/sql, the equivalent of JDBC, drivers. They're purpose-built drivers that expose rich SQLite-specific features. With the huge popularity of Spring most developers don't even interact at the JDBC level today, only e.g. JPA. IMO Java developers are missing out on richer SQL features in their DBs, but well, they seem to mostly manage.

erik_seaberg2y ago

For light integration testing, Apache Derby has pretty good fidelity with prod SQL databases. Never benched it but perf wasn’t a problem.

cvilsmeierOP2y ago

Maybe because Go devs are more allergic to 'non-Go' solutions that Java devs are to 'non-Java' solutions? (Explain: Java's xerial driver is a DLL/SO wrapped in a Java library)

lmz2y ago

Also because there's a mismatch between goroutines and C threads as described here https://www.cockroachlabs.com/blog/the-cost-and-complexity-o... while Java threads can map 1:1 to C threads.

butlerm2y ago

Perhaps someone should define a new C compatible threading API to allow C libraries (including glibc or a wrapper around glibc) to work with something other than native pthreads. Such as goroutines or Java threads and so on.

1 more reply

dave782y ago· 5 in thread

I hadn't heard of sqinn before. According to these benchmarks it beats Cgo-based solutions most of the time, which makes it a very interesting candidate. Anyone have real-world experience using it?

KomoD2y ago

(Sqinn is made by the same guy that made these benchmarks in case you didn't realize)

dave782y ago

I did not notice that, thanks for pointing it out.

cvilsmeierOP2y ago

Yes, in fact the benchmark started out as a comparison between sqinn and mattn. (There was no modernc at that time.)

cvilsmeierOP2y ago

Sqinn author here. Yes, sqinn performs quite well compared to the 'standard' mattn driver. The only use case it clearly breaks down is when SELECTing very large (gigabytes) resultsets.

microtherion2y ago

That stood out to me as well. Any insights why sqinn and zombie underperform in this case, and is the problem inherent to their design?

1 more reply

ncruces2y ago· 3 in thread

I'm the author or the WASM (+wazero) based github.com/ncruces/go-sqlite3. Happy to field questions.

W.r.t. benchmark results.

wazero's current compiler is somewhat naive, which may explain a large performance delta in CPU bound tests. A new compiler is in the works [1].

OTOH it seems interesting that in the (IO bound?) large test I'm doing better than modernc. I wonder why.

I'll dig deeper into the results.

[1]: https://github.com/tetratelabs/wazero/pull/1869

ncruces2y ago

To follow up, I made a new release with performance fixes, and here are the new results: https://github.com/ncruces/go-sqlite-bench/blob/98553a08199b...

ncruces2y ago

The performance difference was larger than I had expected. But this is good.

To fix a recent crash [1] that was happening due to a particular case of reentrancy, which only showed up when I implemented virtual tables and queried other tables to implement one (e.g.: Go calls sqlite3_step to execute a query, which calls Go because it's a query on virtual table, which calls sqlite3_step to scan another table) I introduced a performance regression.

The fix [2] was not to reuse some objects I was allocating once per connection. A mitigation for the regression was (very naive) caching [3].

TLDR: my caching is just not good enough. Simply caching more will go a long way (confirmed already by doubling cache size), but now that I have a good benchmark, I'll do better.

I expect to cut numbers for CPU bound tests in half due to this mishap.

So, thanks cvilsmeier!

[1]: https://github.com/ncruces/go-sqlite3/commit/a9e32fd3f0b9f39... [2]: https://github.com/ncruces/go-sqlite3/commit/d862f47d95d522f... [3]: https://github.com/ncruces/go-sqlite3/commit/9c562f5d8bf7436...

ncruces2y ago

OK, so if I'm looking at this right, a smarter, wider cache goes a great length to fixing the issue.

In [1] I implemented a simple PLRU bit cache, and I'm seeing an 8x performance improvement in some of the tests I was doing worse in:

Before:

  bench-ncruces    -               simple       insert        query       dbsize
  bench-ncruces    -               simple        21224        16495     58687488
  bench-ncruces    -   complex/200/100/20       insert        query       dbsize
  bench-ncruces    -   complex/200/100/20        14993        15228     25354240
  bench-ncruces    -            many/N=10        query       dbsize
  bench-ncruces    -            many/N=10          483        36864
  bench-ncruces    -           many/N=100        query       dbsize
  bench-ncruces    -           many/N=100         3129        36864
  bench-ncruces    -          many/N=1000        query       dbsize
  bench-ncruces    -          many/N=1000        28034        94208
  bench-ncruces    -        large/N=50000        query       dbsize
  bench-ncruces    -        large/N=50000          428    501981184
  bench-ncruces    -       large/N=100000        query       dbsize
  bench-ncruces    -       large/N=100000          779   1003761664
  bench-ncruces    -       large/N=200000        query       dbsize
  bench-ncruces    -       large/N=200000         1475   2007330816
  bench-ncruces    -       concurrent/N=2        query       dbsize
  bench-ncruces    -       concurrent/N=2        13091     56573952
  bench-ncruces    -       concurrent/N=4        query       dbsize
  bench-ncruces    -       concurrent/N=4        14731     56573952
  bench-ncruces    -       concurrent/N=8        query       dbsize
  bench-ncruces    -       concurrent/N=8        24730     56573952

After:

  bench-ncruces    -               simple       insert        query       dbsize
  bench-ncruces    -               simple         5128         3026     58687488
  bench-ncruces    -   complex/200/100/20       insert        query       dbsize
  bench-ncruces    -   complex/200/100/20         3127         3730     25354240
  bench-ncruces    -            many/N=10        query       dbsize
  bench-ncruces    -            many/N=10           93        36864
  bench-ncruces    -           many/N=100        query       dbsize
  bench-ncruces    -           many/N=100          403        36864
  bench-ncruces    -          many/N=1000        query       dbsize
  bench-ncruces    -          many/N=1000         3470        94208
  bench-ncruces    -        large/N=50000        query       dbsize
  bench-ncruces    -        large/N=50000          444    501981184
  bench-ncruces    -       large/N=100000        query       dbsize
  bench-ncruces    -       large/N=100000          717   1003761664
  bench-ncruces    -       large/N=200000        query       dbsize
  bench-ncruces    -       large/N=200000         1401   2007330816
  bench-ncruces    -       concurrent/N=2        query       dbsize
  bench-ncruces    -       concurrent/N=2         3275     56573952
  bench-ncruces    -       concurrent/N=4        query       dbsize
  bench-ncruces    -       concurrent/N=4         3404     56573952
  bench-ncruces    -       concurrent/N=8        query       dbsize
  bench-ncruces    -       concurrent/N=8         4918     56573952

There's still work to do. I could use ints rather than strings for function identifiers. I'll evaluate that later.

[1]: https://github.com/ncruces/go-sqlite3/commit/964a42c76deb9c7...

neonsunset2y ago· 2 in thread

There was a comment from https://news.ycombinator.com/user?id=HackerThemAll with neutral rating that was likely taken down by a mod action despite not violating the rules.

I am reposting it below shortly with minor stylistic changes for politeness.

“Go's SQLite drivers exhibit surprisingly poor performance. In C# I was able to conduct 2 millions of point queries per second on my laptop. And it's not the fastest language in the world.

Go is overrated. It's crudely trying to imitate what Pascal had in '80s using awkward syntax and tooling, but giving you extra CVEs for free.”

Personally, I did not expect it to be this bad…and C# SQLite drivers aren’t even something new - most of them have been written eons ago and consist of fairly standard somewhat allocatey code.

I wonder if it’s because of significant interop overhead in Go, or just fundamental language limitations and quality issues.

zmj2y ago

FFI was high priority for C# in the early days, when most meaningful .NET software was running on Windows and doing interop with win32. The runtime was designed to ensure that scenario would be low-overhead, and that legacy lives on in modern .NET.

neonsunset2y ago

FFI very much remains a priority today which can be seen with changes and migration to [LibraryImport], bespoke NativeAOT modes to produce native dynamically or statically linked libraries which expose C ABI functions with [UnmanagedCallersOnly] and now there is also a project to implement direct Swift library evolution ABI interop for improved support of iOS with MAUI as one of its main goals.

cp92y ago· 1 in thread

the thing about the mattn driver is that it supports all the features that sqlite itself supports. you can compile in vtables, extra stat stuff, which FTS option you want, anything. and if you use zig as your cross compiler you don't even need separate toolchains for the different arch/OS combos, it all just works

ncruces2y ago

It doesn't support everything. No driver likely does.

An example of one thing missing is the pointer passing interface which limits the ability to create complex extensions. If support was added, the way function creation and virtual tables where wrapped would make it hard to use.

But even more trivial things like a decent interface to incremental blob IO have been left unaddressed.

https://www.sqlite.org/bindptr.html

https://github.com/mattn/go-sqlite3/issues/239

auspiv2y ago· 1 in thread

Sure showing time and N works. But it'd be a lot easier to interpret the data quickly if it was shown in terms of operations per second. Smaller bars do not usually mean improved performance.

jen202y ago

… what? Lots of graphs for performance (especially latency) use smaller bars to mean better numbers.

tedunangst2y ago· 1 in thread

> Mattn, although the de-facto standard, is not the best overall solution.

What's wrong with it?

cvilsmeierOP2y ago

Nothing wrong with it, but there are other drivers that cross-compile better and are faster.

eatonphil2y ago

I've done some variations of this as well recently. For inserts: https://github.com/eatonphil/databases-intuition/blob/main/R.... And for selects: https://github.com/eatonphil/databases-intuition/blob/main/R....

Our workloads are a bit different and obviously our machines are a bit different.

Mine only compares mattn/go-sqlite3 to my own fork of https://github.com/bvinc/go-sqlite-lite. go-sqlite-lite seemed like an easier-to-use version of crawshaw's package but it was abandoned so I forked it to bring it up to date.

I agree, for best performance you shouldn't use mattn/go-sqlite3. It does some extra work in hotpaths. It is also higher level and easier to use though. So pick what's important to you.

matharmin2y ago

I'd recommend doing these benchmarks with WAL mode and synchronous = NORMAL as well. It makes quite a big difference in performance especially with many small transactions (e.g. the first example that inserts without explicit transactions).

Groxx2y ago

"without cgo" benchmarks should probably have comparisons for stuff like indexing too, since that's no longer in the shared C library that every other library uses - I would generally expect them to be slower here. And interop with Go code for custom funcs, since sqlite makes that so easy, where I'd expect them to be faster in at least some cases (no repeated cgo overhead).

"Complex" kinda covers this, since there are foreign keys involved, and it's also where e.g. Zombie shows an uncharacteristic slowdown compared to the other benchmarks. Seems like it's probably not a coincidence?

iansinnott2y ago

Thanks for creating the benchmarks, I hadn't heard of most of those libs.

I seem to remember that modernc worked great until i added a FTS5 [1] table and things became very slow. This was a while ago now, so it may have changed or I may mave misattributed the slowness to the non-CGO implementation.

I'd be curious to see how each performs with a fts index and some triggers.

[1]: https://www.sqlite.org/fts5.html

justinclift2y ago

As a data point, the gwenn/gosqlite one for GitHub returns field data correctly even when an individual fields' data type doesn't match the column definition.

https://github.com/gwenn/gosqlite

That's important when processing data from untrusted sources (user generated content, etc).

No idea how it compares to the others performance wise though. :)

j / k navigate · click thread line to collapse

60 comments

51 comments · 13 top-level

acatton2y ago· 19 in thread

In my personal opinion and usage, the performance doesn't matter. Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled.

> modernc, modernc.org/sqlite, a pure Go solution. This is a newer library, based on the SQLite C code re-written in Go.

Unless I'm mistaken, this is not a re-write in Go. This is a transpilation of the the SQLite C library into go, using https://gitlab.com/cznic/ccgo

psanford2y ago

I cross compile mattn/go-sqlite3 a lot. It's about the easiest cgo project to cross compile since it has no external dependencies.

Just use zig as your c compiler and it all just works.

makeworld2y ago

Do you have any links for using Zig for this?

2 more replies

vbezhenar2y ago

The only proper way to use sqlite is to use FFI.

arp2422y ago

> You can't just introduce some transpiler in-between and expect that it'll work

jchw2y ago

shp0ngle2y ago

They run the whole testsuite on the transpiled code. I think.

But yeah it does give me a little shiver to think you are transpiling C code to go code, and the go-code is not really "pure go", but has a platform-dependent unsafe operations. Just look at the repo

https://gitlab.com/cznic/sqlite/-/tree/master/lib?ref_type=h...

mook2y ago

https://www.sqlite.org/testing.html

shepherdjerred2y ago

We use go-sqlite3 at work and have had a positive experience, with the benefit of having fewer system dependencies and a simpler build process.

mariusor2y ago

Aissen2y ago

The biggest issue of the modernc approach is that it can't use the proprietary sqlite test suites and fuzzer to ensure that it works like the original: https://www.sqlite.org/testing.html

Aissen2y ago

(I just realized that probably goes for libsql as well)

shp0ngle2y ago

> Only one driver is written in pure go, and can be easily statically compiled and/or cross-compiled.

Which one do you mean? The WASM one? It includes WASM, it needs to be compiled too.

Ah github.com/cvilsmeier/sqinn-go. Which... is made by the same person that made this benchmark...?

edit: and that requires some random binary to be pre-installed...? Which is in C anyway?

https://github.com/cvilsmeier/sqinn-go

https://github.com/cvilsmeier/sqinn

so I don't see any "actually written in go".

ncruces2y ago

For the WASM one (disclaimer: I built it) it's at least reasonably simple to reproduce the build [1], and the resulting blob is fully cross platform.

[1]: https://github.com/ncruces/go-sqlite3/blob/main/.github/work...

acatton2y ago

The modernc one is the one I was refering to.

https://gitlab.com/cznic/sqlite

shp0ngle2y ago

But that's not pure go, that's C transpiled to go, but even the transpilation is using OS specific and CPU-specific unsafe code.

tedunangst2y ago

Have you tested that the cross-compiled pure go sqlite runs, not just compiles, on every platform? (Spoiler: it does not.)

zombiezen2y ago

zombiezen.com/go/sqlite is built on top of modernc.org/sqlite, so also has the same properties.

alephnan2y ago

How come Zombiezen came out faster ? I’d expect the opposite

cvilsmeierOP2y ago

> How come Zombiezen came out faster ? I’d expect the opposite

(OP here) Honestly, me too. Maybe it has to do with leaving out that database/sql driver layer. I guess it would need another round of pprof to find out the real reason.

1 more reply

Traubenfuchs2y ago· 6 in thread

What a curious state, there is just one jdbc (=java) driver for SQLite. Why are there 6 (or more!) for Go?

eatonphil2y ago

[0] https://github.com/search?q=topic%3Ajdbc%20topic%3Asqlite&ty...

[1] https://github.com/gwenn/sqlite-jna

morelisp2y ago

Aside from the threading mismatch,

erik_seaberg2y ago

For light integration testing, Apache Derby has pretty good fidelity with prod SQL databases. Never benched it but perf wasn’t a problem.

cvilsmeierOP2y ago

Maybe because Go devs are more allergic to 'non-Go' solutions that Java devs are to 'non-Java' solutions? (Explain: Java's xerial driver is a DLL/SO wrapped in a Java library)

lmz2y ago

Also because there's a mismatch between goroutines and C threads as described here https://www.cockroachlabs.com/blog/the-cost-and-complexity-o... while Java threads can map 1:1 to C threads.

butlerm2y ago

1 more reply

dave782y ago· 5 in thread

I hadn't heard of sqinn before. According to these benchmarks it beats Cgo-based solutions most of the time, which makes it a very interesting candidate. Anyone have real-world experience using it?

KomoD2y ago

(Sqinn is made by the same guy that made these benchmarks in case you didn't realize)

dave782y ago

I did not notice that, thanks for pointing it out.

cvilsmeierOP2y ago

Yes, in fact the benchmark started out as a comparison between sqinn and mattn. (There was no modernc at that time.)

cvilsmeierOP2y ago

Sqinn author here. Yes, sqinn performs quite well compared to the 'standard' mattn driver. The only use case it clearly breaks down is when SELECTing very large (gigabytes) resultsets.

microtherion2y ago

That stood out to me as well. Any insights why sqinn and zombie underperform in this case, and is the problem inherent to their design?

1 more reply

ncruces2y ago· 3 in thread

I'm the author or the WASM (+wazero) based github.com/ncruces/go-sqlite3. Happy to field questions.

W.r.t. benchmark results.

wazero's current compiler is somewhat naive, which may explain a large performance delta in CPU bound tests. A new compiler is in the works [1].

OTOH it seems interesting that in the (IO bound?) large test I'm doing better than modernc. I wonder why.

I'll dig deeper into the results.

[1]: https://github.com/tetratelabs/wazero/pull/1869

ncruces2y ago

To follow up, I made a new release with performance fixes, and here are the new results: https://github.com/ncruces/go-sqlite-bench/blob/98553a08199b...

ncruces2y ago

The performance difference was larger than I had expected. But this is good.

The fix [2] was not to reuse some objects I was allocating once per connection. A mitigation for the regression was (very naive) caching [3].

TLDR: my caching is just not good enough. Simply caching more will go a long way (confirmed already by doubling cache size), but now that I have a good benchmark, I'll do better.

I expect to cut numbers for CPU bound tests in half due to this mishap.

So, thanks cvilsmeier!

ncruces2y ago

OK, so if I'm looking at this right, a smarter, wider cache goes a great length to fixing the issue.

In [1] I implemented a simple PLRU bit cache, and I'm seeing an 8x performance improvement in some of the tests I was doing worse in:

Before:

  bench-ncruces    -               simple       insert        query       dbsize
  bench-ncruces    -               simple        21224        16495     58687488
  bench-ncruces    -   complex/200/100/20       insert        query       dbsize
  bench-ncruces    -   complex/200/100/20        14993        15228     25354240
  bench-ncruces    -            many/N=10        query       dbsize
  bench-ncruces    -            many/N=10          483        36864
  bench-ncruces    -           many/N=100        query       dbsize
  bench-ncruces    -           many/N=100         3129        36864
  bench-ncruces    -          many/N=1000        query       dbsize
  bench-ncruces    -          many/N=1000        28034        94208
  bench-ncruces    -        large/N=50000        query       dbsize
  bench-ncruces    -        large/N=50000          428    501981184
  bench-ncruces    -       large/N=100000        query       dbsize
  bench-ncruces    -       large/N=100000          779   1003761664
  bench-ncruces    -       large/N=200000        query       dbsize
  bench-ncruces    -       large/N=200000         1475   2007330816
  bench-ncruces    -       concurrent/N=2        query       dbsize
  bench-ncruces    -       concurrent/N=2        13091     56573952
  bench-ncruces    -       concurrent/N=4        query       dbsize
  bench-ncruces    -       concurrent/N=4        14731     56573952
  bench-ncruces    -       concurrent/N=8        query       dbsize
  bench-ncruces    -       concurrent/N=8        24730     56573952

After:

  bench-ncruces    -               simple       insert        query       dbsize
  bench-ncruces    -               simple         5128         3026     58687488
  bench-ncruces    -   complex/200/100/20       insert        query       dbsize
  bench-ncruces    -   complex/200/100/20         3127         3730     25354240
  bench-ncruces    -            many/N=10        query       dbsize
  bench-ncruces    -            many/N=10           93        36864
  bench-ncruces    -           many/N=100        query       dbsize
  bench-ncruces    -           many/N=100          403        36864
  bench-ncruces    -          many/N=1000        query       dbsize
  bench-ncruces    -          many/N=1000         3470        94208
  bench-ncruces    -        large/N=50000        query       dbsize
  bench-ncruces    -        large/N=50000          444    501981184
  bench-ncruces    -       large/N=100000        query       dbsize
  bench-ncruces    -       large/N=100000          717   1003761664
  bench-ncruces    -       large/N=200000        query       dbsize
  bench-ncruces    -       large/N=200000         1401   2007330816
  bench-ncruces    -       concurrent/N=2        query       dbsize
  bench-ncruces    -       concurrent/N=2         3275     56573952
  bench-ncruces    -       concurrent/N=4        query       dbsize
  bench-ncruces    -       concurrent/N=4         3404     56573952
  bench-ncruces    -       concurrent/N=8        query       dbsize
  bench-ncruces    -       concurrent/N=8         4918     56573952

There's still work to do. I could use ints rather than strings for function identifiers. I'll evaluate that later.

[1]: https://github.com/ncruces/go-sqlite3/commit/964a42c76deb9c7...

neonsunset2y ago· 2 in thread

There was a comment from https://news.ycombinator.com/user?id=HackerThemAll with neutral rating that was likely taken down by a mod action despite not violating the rules.

I am reposting it below shortly with minor stylistic changes for politeness.

“Go's SQLite drivers exhibit surprisingly poor performance. In C# I was able to conduct 2 millions of point queries per second on my laptop. And it's not the fastest language in the world.

Go is overrated. It's crudely trying to imitate what Pascal had in '80s using awkward syntax and tooling, but giving you extra CVEs for free.”

Personally, I did not expect it to be this bad…and C# SQLite drivers aren’t even something new - most of them have been written eons ago and consist of fairly standard somewhat allocatey code.

I wonder if it’s because of significant interop overhead in Go, or just fundamental language limitations and quality issues.

zmj2y ago

neonsunset2y ago

cp92y ago· 1 in thread

ncruces2y ago

It doesn't support everything. No driver likely does.

But even more trivial things like a decent interface to incremental blob IO have been left unaddressed.

https://www.sqlite.org/bindptr.html

https://github.com/mattn/go-sqlite3/issues/239

auspiv2y ago· 1 in thread

Sure showing time and N works. But it'd be a lot easier to interpret the data quickly if it was shown in terms of operations per second. Smaller bars do not usually mean improved performance.

jen202y ago

… what? Lots of graphs for performance (especially latency) use smaller bars to mean better numbers.

tedunangst2y ago· 1 in thread

> Mattn, although the de-facto standard, is not the best overall solution.

What's wrong with it?

cvilsmeierOP2y ago

Nothing wrong with it, but there are other drivers that cross-compile better and are faster.

eatonphil2y ago

Our workloads are a bit different and obviously our machines are a bit different.

I agree, for best performance you shouldn't use mattn/go-sqlite3. It does some extra work in hotpaths. It is also higher level and easier to use though. So pick what's important to you.

matharmin2y ago

Groxx2y ago

iansinnott2y ago

Thanks for creating the benchmarks, I hadn't heard of most of those libs.

I'd be curious to see how each performs with a fts index and some triggers.

[1]: https://www.sqlite.org/fts5.html

justinclift2y ago

As a data point, the gwenn/gosqlite one for GitHub returns field data correctly even when an individual fields' data type doesn't match the column definition.

https://github.com/gwenn/gosqlite

That's important when processing data from untrusted sources (user generated content, etc).

No idea how it compares to the others performance wise though. :)

j / k navigate · click thread line to collapse