Right -- that's the one implementation detail in the OP that was interesting. It sounds like they ultimately used MinIO to replace S3. I've seen people use Ceph, but it's apparently a nightmare to operate a Ceph cluster. If you're on k8s I think the "cloud native" way might be Rook, haven't looked into that. Anyway, running an object store is painful.

Their notes here are a bit vague:

> When the migration reached mid-June, we had 300 servers running very smoothly with a total 200 million cached pages. We used Apache Cassandra nodes on each of the servers that were compatible with AWS S3.

> We broke the online migration into four steps, each a week or two apart. After testing whether Prerender pages could be cached in both S3 and minio, we slowly diverted traffic away from AWS S3 and towards minio. When the writes to S3 had been stopped completely, Prerender saved $200 a day on S3 API costs and signaled we were ready to start deleting data already cached in our Cassandra cluster.

> However, the big reveal came at the end of this phase around June 24th. In the last four weeks, we moved most of the cache workload from AWS S3 to our own Cassandra cluster. The daily cost of AWS was reduced to $1.1K per day, projecting to 35K per month, and the new servers’ monthly recurring cost was estimated to be around 14K.

It says (briefly, in passing) that they used Cassandra to implement the S3 API for their nodes, but maybe just to replicate the S3 API that they were previously using? That's an interesting choice I'd not heard of before. Perhaps all of their individual files are quite small?

Then they moved to MinIO, which would be the S3 equivalent that you are looking for.