(first published 2007, before the more recent misuse of the term began)
From the two definitions found in https://www.sqlite.org/serverless.html
classic serverless --> "embedded database" exists and is often used
neo-serverless --> I suspect that this is a marketing term used to attract cool people to new cloud offerings (like "jamstack" for services such as netlify). There is no good replacement here because the term was tied to this new kind of infrastructure really early. But anytime I hear someone repeat "of course serverless does not mean there is no server!" I die a little more.
Naming things is hard but at least we ought to try to come up with terms that are not blatantly misleading.
No one has the power to change this. It's like saying "I think the word 'spoon' is stupid, let's all call it 'scooper' instead." This is our language now, it is what it is. Vendors offering serverless solutions have to call it "serverless" whether they like it or not because that's the word customers understand. You can keep complaining about it, but it won't change. You might as well accept it and move on.
"Cloud" is a stupid term too, your servers aren't really floating in the sky. But here we are.
That's my excuse for being so slow to grasp "cloud". When AWS made EC2 public, I really didn't understand why anyone would want to use it.
Cloud era "serverless" just pisses me off. The applicable cliche is "Every sufficiently complex C project recreates LISP, poorly."
It's seems like we're doomed to repeatedly reinvent the past. Championed by youngsters who never read the book.
Maybe this explains "worse is better".
I know, I know. I've become the old crank. All you noisy kids get off my lawn.
AWS marketing is very good.
Serverless was a creation of the AWS Lambda team / AWS marketing department "I helped start the serverless movement..." from the LinkedIn of Tim Wagner (https://www.linkedin.com/in/timawagner/)
Worker-based or FaaS (function as a service) don't roll off the tongue that well.
I'm not advocating for the term, just trying to explain why it is so popular.
Language does change and evolve. The misuse and subsequent additional meaning for “literally” to now also be a simile for “figuratively” is a good example of how this isn’t necessarily always a good thing.
My idea is to modify SQLite to use ajax with the HTTP Range header to fetch B+ pages from the server as they are needed. SQLite already has a VFS (virtual file system) so this shouldn't be too hard.
I am not sure how fast it would be and it would waste a lot of bandwidth. That's why I haven't made it yet.
This would only be useful for using it with Github pages.
They are used to share structured and unstructured data in a p2p way where a torrent could be a database(structured) or a fileset(unstructured) .
But all of this is part of a bigger project that is basically a new sort of p2p "app browser" with a local-first emphasis but where every app can share its functionality though RPC (locally or remotely) with distribution going over DHT's and torrents.
Even with an artificially slow connection it seems to be reactive enough. Actually fetching pages on-demand can only be better for the bandwidth, the real issue is going to be latency
A small DB to search in a list of storefronts which typical retail websites have, a database of flags or promotion codes which you don't want to send to the browser are some examples I came up with.
Personally I would write a code generator to embed the data in the code but to each their own.
We use KV and workers to serve static data (json) to browser based code.
This could be used to filter, sort or aggregate that data before sending the client.
Perhaps for inventory / product lookup on a e-commerce site.
My https://datasette.io/ project is built around a similar idea to this: the original inspiration for it was Zeit Now (now Vercel) and my realization that SQLite read-only workloads are an amazing fit for serverless providers, since you don't need to pay for an additional database server - you can literally bundle your data with the rest of your application code (the SQLite database is just a binary file).
If you want to serve a SQLite API in an edge-closest-to-the-user manner for much larger database files (100MB+) I've had some success running Datasette on https://fly.io/ - which runs Docker containers in a geo-load-balanced manner. I have a plugin for Datasette that can publish database files directly to Fly: https://docs.datasette.io/en/stable/publish.html#publishing-...
We are toying around with the idea of launching a Cloud SQLite product targeting this exact use case.
How large are your data sets?
Would you be interested in a quick call to discuss your needs in more detail?
It’s been working extremely well and it runs fully sandboxed in a separate worker thread with zero network traffic after the source data has been transferred across. With computational notebooks running alongside it, you can do some pretty powerful analytics with very low latency and it’s great to be able to do inserts and complex queries entirely client-side.
SQLite is one of the greatest software projects I know.
Seriously though, if it's Truing complete, you know it'll be abused to do something completely unintended, like run a ray tracer or something.
We've been building an experimental shared file system that specifically targets the FaaS setting. It supports SQLite and gets big performance gains over NFS/EFS, especially on read-mostly workloads, due to improved local caching and lock elision. See: https://arxiv.org/abs/2009.09845
How do you deal with ephemeral file storage? I guess you actually need the provider (say AWS) to build this into the platform?
Also, out of a cost perspective, running a VM with SSD for cheap with SQlite will give much more requests than a CF worker for much less.
Adding writes to this is also very limited due to the max 1sec write per KV key limit.
It runs out of CPU-time at 6MB of index though.
There's someone that made a WASM for full-text search too, it's definitely faster and can handle quite a lot more text.
Yes but now it runs on someone else server! Wait a minute, this makes no sense...
'serverless' is perhaps the stupidest marketing buzzword developers have come up with.
> Recently, folks have begun to use the word "serverless" to mean something subtly different from its intended meaning in this document. Here are two possible definitions of "serverless": Classic Serverless: [...] Neo-Serverless: [...]
SQLite was already "Classic Serverless". Now it's "Neo Serverless" as well.
[0] https://en.m.wikipedia.org/wiki/Two-phase_commit_protocol
But they are limited to read only files.
Cloud flare workers allow running simple programs at the CDN node closest to the user.
SQLite is embeddable C code that is like a normal SQL server just without the network parts - just the sync functions embedded into your process.
This wraps SQLite with the networking and CDN abilities of cloud flare, allowing you to http-get a subset of data dynamically using SQL (that would not be possible with the simple “download 100% of file X”).
[1] https://github.com/sql-js/sql.js [2] https://ml-ranking.geospiza.me
There's no ability to change the anything through the SQL. It spins up a new sqlite db every time, and builds the table in memory.
For running a write-enabled DB on a CF worker a major problem is that the only storage option that I could find has really low write limits [0], 1000 writes a day for the free option. "Durable Objects" is a beta API, perhaps its transactional-storage-api [1] has better limits?
--
0: https://developers.cloudflare.com/workers/platform/limits#kv...
1: https://developers.cloudflare.com/workers/runtime-apis/durab...
Client issued arbitrary queries is one of the use cases for GraphQL and publishing immutable data sets on the web is the main use case for Simon Wilson’s Datasette [2].
In-memory SQLite compiled to WASM works in the browser and Node.js too. In the future, we can expect proper ACID operations on any WASM runtime that supports fsync in WASI [3], a POSIX-like API.
Workers KV imposes 25MB limit per key. Worker memory limit is 128MB. Concatenating several values from the store or using sqlite's ATTACH DATABASE should make possible querying of about 100MB large databases, would be my guess.
