Before we had 3-tier architectures, people would have designed a shopping cart use-case as a single SQL transaction that would last maybe 10 minutes. The DB would make sure everything stays consistent until the final commit. The GUI would keep an open connection to the DB the whole time.
In the web age, you want stateless services and HA. It means a transaction can't last more than a single web page. It becomes more challenging to design a shopping cart, because the DB can't handle a long-running transaction anymore.
Writing a correct system that reserves the items you put in a shopping cart and doesn't leak items and doesn't sell the same item twice is not easy. A transaction Rollback will not do the cleanup for you, because there's no long running transaction anymore.
So SQL transactions can't help as much as you think.
Mongodb doesn't have transactions, but updates are atomic, which allows CAS and optimistic locking use cases. I agree it's less than ideal when you need to provide ACID behavior, but don't believe it's easy with SQL transactions. It's not.
The author regrets the book's suggestion of putting each object in stock in its own document, and I agree it's probably a recipe for disaster. Atomic updates make this design absurd.
You could easily db.products.update({_id: productId}, {$inc: {inStock: -5}, $addToSet: {pendingCarts: {cartId: cartId, quantity: 5, timestamp: new Date()}}}). This has the exact same atomic behavior as a SQL transaction to remove 5 from the stock and add a new "shopping cart entry" in another table.
(you still need to expire cancelled shopping carts, and you may need a transactional way of completing the order: it's also manageable if designed as an idempotent operation)
Anyway don't over-simplify this use case and believe "a single big SQL ACID transaction would handle the problem". That's just not true.
In a warehouse inventory setting, when you _do_ have inconsistencies (e.g. lost items, misplaced orders, etc), a system which strictly enforces "inventory limits" will as often prevent employees from doing their job and shipping an item which could be sitting right in front on them but is not counted for in the system. Auditing combined with optimistic locking resolves this and allows both accountability, tracking, and flexibility.
Those are two real world examples which underly the idea that ACID guarantees and locking / transactions are two separate intents. CouchDB & Couchbase both provide ACID guarantees per document making it straightforward to implement multi-service applications using event base systems. It's equivalent to MongoDB's CAS operations. Really all that you need is to ensure that your changes are atomic and generally ACID compliance at a key/document level enables you to do this readily.
Personally, I find that SQL-style transactions just cause lots of issues with performance and locking contention while enabling developers to skimp on thinking deeply about how to appropriately design their data flow. Sometimes that's the right call for a team, but sometimes it's not.
"Designing data flow" would mean having to spend more time and money for the same task?
Or it is because you denormalized your model because your db engine's performance sucks in which case the transactions will probably just make it worse.
Good schema design and lock-free/wait-free (transaction-free) algorithms are not "reimplementing transactions in the client."
OPs example is garbage but his proposed transaction solution is garbage too.
A service bus was necessary, but the actual atomic transactions in MongoDB didn't fail us. We didn't lose data. While the nay-sayers discounted Mongo, we were raking in cash on top of it.
> In the web age, you want stateless services and HA. It means a transaction can't last more than a single web page. It becomes more challenging to design a shopping cart, because the DB can't handle a long-running transaction anymore.
Or you could cheat and not update the inventory until the purchase is made. ;)
The problem with the "adjust inventory on cart" in low inventory situations is you'll have 80% of your carts holding items that won't convert until a cart expiration. You only need the actual purchase to be atomic. Then, once the queued credit card transaction completes you adjust the order to refund the inventory [declined] or ship the order [completed].
That pattern absolves you of needing complex logic and allows you to distribute the activity relatively trivially as a set of two independent idempotent operations. And if the analog portion of the process fails, the picker hits a button and the order gets queued for a refund. Once the order is cancelled, another service contacts the customer.
Cart expiration, etc. makes the system unnaturally brittle by adding non-critical steps to the process.
That is unlikely to work well at much scale. At least last I knew, Mongo docs are limited to 16MB and the entire doc is read then written in cases like this, very slow on large docs. Given the amount of data that may be attached to a `product`, it's not hard to hit these limits.
So I wanted to say that denormalization and lack of foreign keys in MongoDB is much worse that lack of transactions.
This problem was solved in 1965 by CICS for the use case of "you're on the phone to a travel agent and they're finding you a ticket on their terminal". No "10 minute single transactions" anywhere...
In the web age, you want stateless services and HA
Those who forget history are doomed to repeated it.
> The example is that if you have 10 rakes in the stores,
> you can only sell 10 rakes. The approach that is taken
> is quite nice, by simulating the notion of having a
> document per each of the rakes in the store and allowing
> users to place them in their cart.
One would subtract product from amout field in products table to set new inventory level. A new order document gets created with all information needed to describe an order. Fields like total, subtotal, date would sit at the root level and line items with product descriptions and prices would be embedded as an array of objects. Then a user object with user_id, name address would be embedded.
Dealing with documents is a different but being able to contain the entire dataset with some relations is nice.
What purpose does it have frankly? The only one I see might be GridFS for what it is worth, though I don't believe one second the performances are that great, but when it comes to document oriented DB, Postgres can store both JSON and XML and query them and also do partial atomic changes. Scaling? easier maybe... Now competition is good and I'm sure NoSQL db success kind of forced traditional players to innovate. But I see no reason to use MongoDB in 2017.
I would reply: replica sets and sharding and multi-threaded architecture and the absence of impedance mismatch, all in a single product that was designed for these features.
The truth is that MongoDB is well suited to a fairly narrow set of use-cases, but ends up being used for all sorts of stuff in practice. Hence the weird contortions which the author observes in the book they're reading.
Although Oracle, Sybase, SQL server and friends all transactions at that time, somehow the the mindset was that it was a complicated enterprise marketing gimmick, MySQL/mSQL are faster and simpler, and we can work around it in the client side. Seems like not much has changed.
It was made worse by the MySQL team actively advocating against features they didn't have "you don't need transactions, do it in your application", "you don't need foreign keys, do it in your application" blah blah.
20+ years later they're still struggling to shoehorn it in.
MongoDB should not be used for an online ordering system, period. But if the programmer had no better alternative than Mongo, then please use Mongo's atomic operations [1] and nested documents to make sure nasty Bad Things don't happen.
[1] https://docs.mongodb.com/manual/tutorial/model-data-for-atom...
As you say, bad idea in the first place.
Operations that span multiple rows can be safely performed with a bus in front of it.
You can't judge the idea. I don't think you've quite grasped it. I agree the book example isn't great, but that's not the technology's fault.
DB team insisted on writing "DAOs" that ended up pulling 1GB+ of data back from Mongo to merge in EACH of 100+ data points from a scanned machine. Similar issues in UI presentation. With multiple threads doing each of these things simultaneously there were many out of memory dumps. I analyzed these multiple times and told the DK VP Engineering what the problems were, and they didn't follow up for 6 months. He was gone soon after.
That DB team shouldn't be allowed near any database. Why on earth would they go for such a moronic abstraction?
"Mongodb, the ultimate Maybe monad. With a built in fromMaybe mempty call for your convenience."
Per Hmemcpy and Michael Snoyman on Twitter.
One way is to add a field to an item that show its status: whether it is in a warehouse, in someone's cart, ordered or sold. Then adding an item to a cart means updating those fields. There probably is a way to do several similar updates atomically.
Another way is to use append-only collection, that keeps a list of events, like "Item X added to cart Y", "Item X sent to delivery".
But I guess when there are more entities and relations this would become too complex to manage. While SQL databases have no problems with hundreds of tables and thousands of columns.
Atomicity is document level, not collection level. So you can't update multiple documents atomically. Or do you plan on having `{status: 'in-cart', cartOwner: 'customer-id' | null}` and single document for every stocked item (like 1k copies of the same book would be 1k db documents and you also have all those sold from before)?
> Another way is to use append-only collection
How does it help with overselling? To decide if it's okay to append, you have to know if current number of items is greater than 0 (don't forget to lock other clients out of appending this whole process, so they wait for you to finish).
The rest of us will just use a hammer.
Using a database that doesn't offer ACID, in a manner that requires ACID has non-trivial associated costs. This may also leave you open to a number of strange situations, where inventory quantities are unknown or incorrect.
Are you tracking individual lots of inventory and the costs you paid for them? Against which lot did you sell this one? You don't have any - so how do you calculate the margin for this item you sold but don't know how much it cost or where it came from? If it is returned, do you restock that inventory?
Mongo, SQL - they have there differences, but doing inventory management is tricky no matter what technology you use.