Which you obviously still have for an in-memory DB, but you should expect it to require a lot more picked low-hanging fruit before that becomes your bottleneck.
I am fairly certain the issues are similar for IndexedDB.
Why would you expect that you wouldn’t need to do this just because it’s an in-memory database? You might still be modifying it concurrently through multiple tabs open to the same site modifying the same DB or because you are interleaving “concurrent” actions for some reason within a page. Has nothing to do with I/O to my knowledge.
Try this experiment out with any transactional in-memory tables on any database and they should all show some amount of degradation (although maybe not necessarily quite as severe as IndexDB - 10k inserted rows/s seems below what SQLite should be putting out).
It warrants an audible "duh". More work is always going to run slower.
(grouping there does not seem to help)
The solution would probably be, to not have lots of object stores, but this would have made my db setup much more complicated.
But FileSystem API seems to finally getting broad support.
> But FileSystem API seems to finally getting broad support.
I'm not seeing much beyond Chrome & it's associates for it?
That said that could just be because "filesystem" as a word is mentioned in a dozen different web APIs from allowing file input reading locally, to the deprecated API chrome had for local file access, to the new thing that came out of "project Fugu" for PWAs to have native file access and I'm missing which one is which in terms of support & docs.
I'm using localForage which lets me persist data to either sessionStorage, localStorage or IndexedDB (depending on user configuration), however writes to IndexedDB were excruciating slow.
I didn't want to give up on it however, because localStorage has a storage limit that power users were constantly coming up against, and IndexedDB is also necessary for progressive web apps.
Eventually I solved the problem by batching IndexedDB writes with a utility function I wrote called mergebounce[2]. It combines Lodash's "debounce" and "merge" functions to combine multiple function calls into a single one, while keeping and merging the data that was passed to the individual function calls.
That way, you can call a function to persist data to IndexedDB many times, but it only executes once (after a small delay). I wrote a blog post about this approach[3]. It's generic enough that anyone else who uses localForage for IndexedDB could use it.
1. https://conversejs.org 2. https://github.com/conversejs/mergebounce 3. https://opkode.com/blog/2021-05-05-mergebounce-indexeddb/
localStorage doesn't work for progressive web apps?
You grab stuff easily out of it, but not much'll stay in there.
If you were using IndexedDB as a database, you wouldn't bulk-insert data in this way, as it's the slowest way to do it.
The conclusion should be "use IndexedDB as you would use a database".
Please provide a real world example of this in a client side/PWA context for the class...
On the contrary, I've found onbeforeunload far from reliable (albeit my use-case was a bit different). You should probably not rely on it for anything important.
Because IndexedDB uses structured clone, it can store javascript data as is, which includes storing correctly a number of interesting/useful types, including ArrayBuffers, Files/Blobs, CryptoKey instances (useful to avoid needing export flag on them), the cost of this is the performance hit compared to JSON.parse on a string.
Also the cache is less persistent than IndexedDB when storage pressure increases.
1. https://localforage.github.io/localForage/ 2. https://github.com/localForage/localForage
https://pubkey.github.io/client-side-databases/database-comp...
If you are building an offline enabled web app for the cost of a 1mb wasm download it’s worth it for full sql support and the speed increase.
I think there is a good chance Absurd-SQL will get folded into SQLite.js.
What kind of sorcery is this!? (WASM)
So the problem is that the author is using it wrong, since you only need to wait for transaction commit when you need guaranteed durability, and you only need guaranteed durability when you are communicating success to the user or some other system, which only needs to happen every frame, which is 16ms at 60fps which is plenty of time to wait for a 2ms transaction commit (network communication can also be throttled to the framerate).
So in practice all you need to do is batch writes in a single transaction per frame and there is no problem.
And of course if you don't need durability you can just keep the data in the JavaScript heap (and use inter-tab communication if needed).
In my brief experiment, it was 12% faster to read from the web Cache API [1], re-parse and re-transform that nested object than to read the fully transformed object using IndexedDB via idb-keyval [2]. That surprised me! I went on to learn that IndexedDB does a structured clone as part of such reads, which I suspect is the main cause of slowness in my use case.
Related commits to reproduce that finding are in [3], specifically [4].
[1] https://developer.mozilla.org/en-US/docs/Web/API/Cache
[2] https://github.com/jakearchibald/idb-keyval
[3] https://github.com/eweitz/ideogram/pull/285
[4] https://github.com/eweitz/ideogram/pull/285/commits/90e374a0...
Other comments mention using batch operations to speed-up IDB but maybe Chrome could also fix their implementation in the first place. It seems, though, that the Chrome team had no intention on improving the write performance of IDB as of 2019[2].
Edit: it seems that Chrome team did improve the performance of IDB since 2019 but it is still not at Firefox level of performance.
[1] https://dev.to/skhmt/why-are-indexeddb-operations-significan... [2] https://bugs.chromium.org/p/chromium/issues/detail?id=102545...
A declarative storage syntax for a declarative display engine. Seems a good fit.
