undefined | Better HN

0 pointsg0del_was_wr0ng9y ago0 comments

It's doing 12k rows in 3k user-issued write operations/transaction per second.

Counting any kind of "internal write effects" that result from a user write (i.e. write amplification) is obviously done to mislead in the benchmark and does not make it comparable to key-value stores.

12k writes/s is the number of rows that are written from a user perspective. So 12k/s is also the number you have to use when comparing it to key value stores. But of course, comparing Fauna with eventually consistent systems is not a really fair comparison. You don't make it fairer by misleading in your benchmark though.

Also, just because some other vendor posted a misleading benchmark on hn (I don't know if they did) that doesn't make it right or means you should do it. Just call them out on it too.

0 comments

evanweaver9y ago

Indexes aren't internal write effects, they are user-defined. But we will have additional benchmarks later on that focus on row commits only.

We tried to replicate a realistic workload rather than just target the best case or worst case performance profile.

g0del_was_wr0ngOP9y ago

By that logic a write to a postgres table would also count as N+1 writes, where N is the number of secondary indexes defined on that table?

And also, by the same logic, replicating a write to three machines counts as three writes when the replication factor is user-defined, right?

evanweaver9y ago

For purposes of this benchmark, the first yes, the second no. But neither of them are free.

g0del_was_wr0ngOP9y ago

Well fair enough but that method of counting is not what everyone else does or assumes, so somebody just reading the title "120k writes per second" gets the wrong impression of whats going on.

(An uninitiated reader would assume you're comitting 120k rows per second from the title, whereas it's "only" 12k rows and "only" 3k actual operations over the wire. Still, 3-12k is pretty impressive)

j / k navigate · click thread line to collapse

0 pointsg0del_was_wr0ng9y ago0 comments

It's doing 12k rows in 3k user-issued write operations/transaction per second.

Also, just because some other vendor posted a misleading benchmark on hn (I don't know if they did) that doesn't make it right or means you should do it. Just call them out on it too.

0 comments

evanweaver9y ago

Indexes aren't internal write effects, they are user-defined. But we will have additional benchmarks later on that focus on row commits only.

We tried to replicate a realistic workload rather than just target the best case or worst case performance profile.

g0del_was_wr0ngOP9y ago

By that logic a write to a postgres table would also count as N+1 writes, where N is the number of secondary indexes defined on that table?

And also, by the same logic, replicating a write to three machines counts as three writes when the replication factor is user-defined, right?

evanweaver9y ago

For purposes of this benchmark, the first yes, the second no. But neither of them are free.

g0del_was_wr0ngOP9y ago

Well fair enough but that method of counting is not what everyone else does or assumes, so somebody just reading the title "120k writes per second" gets the wrong impression of whats going on.

j / k navigate · click thread line to collapse