In the past, we relied heavily on using EVAL[SHA] with 200+ loc Lua scripts in order to implement high throughput, atomic transactions for realtime systems. We also used the JSON & Redis Query Language (previous named "full-text search") to build a more maintainable & strongly consistent system than using raw key-values and manually building secondary indexes.
We’ve since migrated to a native FoundationDB and SQLite hybrid setup, but this approach would have been really helpful for early-stage prototyping with a higher performance ceiling (thanks to FDB sharding) than a single-node Redis with AOF.
Related: Redis Cluster is a world of pain when handling clustering keys and cross-node queries and orchestration. DragonflyDB is chasing after the market of companies considering sharding Redis because of performance issues by providing better single-node performance. There's probably an alternative approach that could work by using an architecture like this.
Exactly. It's a distributed list, map, etc that is often used as a cache, and sometimes as a queue, but it's bigger than all that.
Oh? I'd be interested in hearing more about that. Is this common?
JVM is fast, but raw Java is not seen so often in corpo-rat world. And if for single endpoint getting data from database and encoding to JSON you have to schedule 2 cores and 4G of RAM for every few hundred QPS - something is wrong.
The ecosystem of Java is so huge. Most people who use Java barely have any idea what the rest of the software industry is doing.
Java is the most performant runtime outside of C/C++/Rust. It is a first choice for any project.
This was quite eye opening when thinking about this, I am aware of how underappreciated the performance of JVM is, but I never thought about how widely deployed it is
Maybe this is it.
[1] https://redis.io/resources/building-large-databases-redis-en...
[1] https://kvrocks.apache.org/ [2] https://github.com/apache/kvrocks
https://github.com/dragonflydb/dragonfly DragonflyDB (not open source, BuSL-1.1) with more performance
https://github.com/apache/kvrocks Apache Kvrocks (Apache-2.0) uses disk-based NoSQL database to lower memory usage
