Selling an event out takes a long time to do frequently because tickets are VERY frequently not purchased--they're just reserved and then they fall back into open seating. This is done by true fans, but also frequently by bots run by professional brokers or amateur resellers. And Cloudflare and every other state of the art bot detection platform doesn't detect them. Hell, some of the bots are built on Cloudflare workers themselves in my experience...
So whatever velocity you achieve in the lab--in the real world you'll do a fraction of it when it comes to actual purchases. That depends upon the event really. Events that fly under the radar may get you a higher actual conversion rate.
Also, an act like Oasis is going to have a lot of reserved seating. Running through algorithms to find contiguous seats is going to be tougher than this example and it's difficult to parallelize if you're truly giving the next person in the queue the actual best seats remaining.
There are many other business rules that accrue after years of features to win Oasis like business unfortunately that will result in more DB calls and add contention.
TigerBeetle actually includes native support for "two phase pending transfers" out of the box, to make it easy to coordinate with third party payment systems while users have inventory in their cart:
https://docs.tigerbeetle.com/coding/two-phase-transfers/
> Also, an act like Oasis is going to have a lot of reserved seating. Running through algorithms to find contiguous seats is going to be tougher than this example and it's difficult to parallelize if you're truly giving the next person in the queue the actual best seats remaining.
It's actually not that hard (and probably easier) to express this in TigerBeetle using transfers with deterministic IDs. For example, you could check (and reserve) up to 8K contiguous seats in a single query to TigerBeetle, with a P100 less than 100ms.
> There are many other business rules that accrue after years of features to win Oasis like business unfortunately that will result in more DB calls and add contention.
Yes, contention is the killer.
We added an Amdahl's Law calculator to TigerBeetle's homepage to let you see the impact: https://tigerbeetle.com/#general-purpose-databases-have-an-o...
As you move "the data to the code" in interactive transactions with multiple queries, to process more and more business rules, you're holding row locks across the network. TigerBeetle's design inverts this, to move "the code to the data" in declarative queries, to let the DBMS enforce the transactional business rules directly in the database, with a rich set of debit/credit primitives and audit trail.
The shopping process and queuing process puts considerably more load on our systems than the final purchase transaction, which ultimately is constrained by the size of the venue, which we can control by managing the queue throughput.
Even with a queue system in place, you inevitably end up with the thundering heard problem when ticket sales open, as a large majority of users will refresh their browsers regardless of instructions to the contrary
In other words, to count not only the money changing hands, but also the corresponding goods/services being exchanged.
These are all transactions: goods/services and the corresponding money.
----
In our testing:
For batch transactions, Tigerbeetle delivered truly impressive speeds: ~250,000 writes/sec.
For processing transactions one-by-one individually, we found a large slowdown: ~105 writes/sec.
This is much slower than PostgreSQL, which row updates at ~5495 sec. (However, in practice PostgreSQL row updates will be way lower in real world OLTP workloads due to hot fee accounts and aggregate accounts for sub-accounts.)
One way to keep those faster speeds in Tigerbeetle for real-time workloads is microbatching incoming real-time transactions to Tigerbeetle at an interval of every second or lower, to take advantage of Tigerbeetle's blazing fast batch processing speeds. Nonetheless, this remains an important caveat to understand about its speed.
> One way to keep those faster speeds in Tigerbeetle for real-time workloads is microbatching incoming real-time transactions to Tigerbeetle at an interval of every second or lower, to take advantage of Tigerbeetle's blazing fast batch processing speeds.
We don’t recommend artificially holding transfers just for batching purposes. René actually had to implement a batching worker API to work around a limitation in Python’s FastAPI, which handled requests per process, and he’s been very clear in suggesting that such would be better reimplemented in Go.
Unlike most connection-oriented database clients, the TigerBeetle client doesn’t use a connection pool, because there’s no concept of a “connection” in TigerBeetle’s VSR protocol.
This means that, although you can create multiple client instances, in practice less is better. You should have a single long-lived client instance per process, shared across tasks, coroutines, or threads (think of a web server handling many concurrent requests).
In such a scenario, the client can efficiently pack multiple events into the same request, while your application logic focuses solely on business-event-oriented chains of transfers. Typically, each business event involves only a handful of transfers, which isn't a problem of underutilization, as they'll be submitted together with other concurrent events as soon as possible.
However, if you’re dealing with a non-concurrent workload, for example, a batch process that bills thousands of customers for their monthly invoices, then you can simply submit all transfers at once.
> For processing transactions one-by-one individually
If you're artificially restricting the load going into TigerBeetle, by sending transactions in one-by-one individually, then I think predictable latency (and not TPS) would be a better metric.
For example, TB's multi-region/multi-AZ fault-tolerance will work around gray failure (fail slow of hardware, as opposed to fail stop) in your network links or SSDs. You're also getting significantly stronger durability guarantees with TB [0][1].
It sounds like you were benchmarking on EBS? We recommend NVMe. We have customers running extremely tight 1 second SLAs, seeing microsecond latencies, even for one at a time workloads. Before TB, they were bottlenecking on PG. After TB, they saturated their central bank limit.
I would also be curious to what scale you tested? We test TB to literally 100 billion transactions. It's going to be incredibly hard to replicate that with PG's storage engine. PG is a great string DBMS but it's simply not optimized for integers the way TB is. Granted, your scale likely won't require it, but if you're comparing TPS then you should at least compare sustained scale.
There's also the safety factor of trying to reimplement TB's debit/credit primitives over PG to consider. Rolling it yourself. For example, did you change PG's defaults away from Read-Committed to Serializable and enable checksums in your benchmarks? (PG's checksums, even if you enable them, are still not going to protect you from misdirected I/O like the recent XFS bug.) Even the business logic is deceptively hard, there are thousands of lines of complicated state machine code, and we've invested literally millions into testing and audits.
Finally, it's important that your architecture as a whole, the gateways around TB, designs for concurrency first class, and isn't "one at a time", or TigerBeetle is probably not going to be your bottleneck.
There was also poor Ruby support for Tigerbeetle at the time, but that has improved recently and there is now a (3rd party) Ruby client: https://github.com/antstorm/tigerbeetle-ruby/
On the one hand, yes, you could use a general purpose string database to count/move integers, up to a certain scale. But a specialized integer database like TigerBeetle can take you further. It's the same reason, that yes, you could use Postgres as object storage or as queue, or you could use S3 and Kafka and get separation of concerns in your architecture.
I did a talk diving into all this recently, looking at the power law, OLTP contention, and how this interacts with Amdahl's Law and Postgres and TigerBeetle: https://www.youtube.com/watch?v=yKgfk8lTQuE
Edit: Yes, I think I misunderstood something here. The user wouldn't even see their request as having returned a valid "pending" ticket sale since the batcher would be active as the request is active. The request won't return until its own transfer had been sent off to TigerBeetle as pending.
