Are Databases going to start dissappearing?

7 pointsseph-reed1y ago11 comments

I already don't trust anything a bot can POST to. So now I'm working towards making all my personal information live on servers that I control.

And when it comes to decentralized personal information, it's rare that I (anecdotally) have enough of anything to require a true DB. For instance:

- All of my health records (excluding imaging) could easily fit neatly in a 2MB JSON.

- A list of every photo I've ever taken, a file path to each, and a description would maybe take 10MB.

- A description of every event I've ever been to, and every person I've ever met might take 8MB.

- A lat, lng, and timestamp entry every minute for 10 years would be 5,256,000 entries × 26 bytes = ~130MB

Given how little overhead it takes to store the entire information of any individual, plus how infrequently it is likely to be modified, it seems there is no good reason to use databases in a decentralized context.

What do you think of this prediction?

11 comments

jhunter10161y ago

I’ve thought about this a lot before in my personal and professional life. If you combine JSON with a SQLite database, you can get the best of both worlds.

Disclaimer, I wrote this for work but it is based on ideas I developed outside of work. https://pinata.cloud/blog/will-json-work-as-a-database/

fridental1y ago

Ignoring the prediction that everything will be decentralized in the near future (it probably won't), how about this scenario: you want to see if the voting activity (recorded in your events) correlates with the number of times you went to the doctors in the year prior to election (as recorded in your health records). If we want to be able to run such a distributed query, we need to be sure that each node stores the data in predefined format ("tables") and that one of their services run a service that will receive the distributed query request, decide whether or not it wants to participate, and then execute the business logic of the conditions (healthevent.year > getdate() - 365), probably defined in some programming language ("SQL").

seph-reedOP1y ago

> decentralized in the near future (it probably won't)

I may be optimistic on this one. But it feels like dead internet is encroaching everywhere. There aren't many centralized services I can think of that people actually like. Perhaps Steam, but most of them are hated by their users. Currently, I'm imagining that more and more of our digital interactions are going to go through AI helpers. Agents that will filter out ads, and likely also hold onto our information for us. At a certain point, centralized DBs just become an unnecessary middleman with privacy concerns. If I want photos of my friends, why not just have my assistant ask their assistant?

> how about this scenario: ....

It seems like all of this data could easily fit into a 10MB of text. This is the kind of thing an assistant would be likely to churn through without issue. It could also search for other interesting correlations while it's at it.

mtmail1y ago

From your description all would easily fit into RAM and full-scan searches across all records will be fast enough. Browsers hardly sweat reading 200MB data files. No need for indexes, optimized storage, compression or special query languages. Unless you'd worry about scale (managing the same for 1000 people). It's a luxury of modern hardware. I think databases or similar data stores will remain, but for many projects looking at performance or small memory footprint will no longer matter. I've used Postgresql for tiny projects (all in docker containers) because it was convenient.

seph-reedOP1y ago

I often go with either SQLite or a JSON loaded to RAM and synced for really small stuff. But even SQLite is starting to seem like overkill. Honestly, I think most of my data will be able to fit into a single context window of an AI soon enough.

reportgunner1y ago

Absolutely not. The world runs on excel spreadsheets stored in databases.

someothherguyy1y ago

> What do you think of this prediction?

That you don't know what you are talking about. You might want to reconsider what you think a database is.

seph-reedOP1y ago

You seem to be human from your comment history. Are you being pedantic about your definition of databases, or do you legitimately think DB's are necessary for storing decentralized personal information? I'm not considering a filesystem a DB, for instance.

reportgunner1y ago

With all due respect, what do you think decentralized personal information is ?

I don't think that really is a thing except for a copy of some data that exists in some centralized (SQL) database(s).

someothherguyy1y ago

If you are trying to pigeonhole the definition of a database in a way that is going to prompt me to agree with you, you will be disappointed to learn that yes, I am a human.

1 more reply

j / k navigate · click thread line to collapse

Are Databases going to start dissappearing?

7 pointsseph-reed1y ago11 comments

I already don't trust anything a bot can POST to. So now I'm working towards making all my personal information live on servers that I control.

And when it comes to decentralized personal information, it's rare that I (anecdotally) have enough of anything to require a true DB. For instance:

- All of my health records (excluding imaging) could easily fit neatly in a 2MB JSON.

- A list of every photo I've ever taken, a file path to each, and a description would maybe take 10MB.

- A description of every event I've ever been to, and every person I've ever met might take 8MB.

- A lat, lng, and timestamp entry every minute for 10 years would be 5,256,000 entries × 26 bytes = ~130MB

What do you think of this prediction?

11 comments

jhunter10161y ago

I’ve thought about this a lot before in my personal and professional life. If you combine JSON with a SQLite database, you can get the best of both worlds.

Disclaimer, I wrote this for work but it is based on ideas I developed outside of work. https://pinata.cloud/blog/will-json-work-as-a-database/

fridental1y ago

seph-reedOP1y ago

> decentralized in the near future (it probably won't)

> how about this scenario: ....

mtmail1y ago

seph-reedOP1y ago

reportgunner1y ago

Absolutely not. The world runs on excel spreadsheets stored in databases.

someothherguyy1y ago

> What do you think of this prediction?

That you don't know what you are talking about. You might want to reconsider what you think a database is.

seph-reedOP1y ago

reportgunner1y ago

With all due respect, what do you think decentralized personal information is ?

I don't think that really is a thing except for a copy of some data that exists in some centralized (SQL) database(s).

someothherguyy1y ago

If you are trying to pigeonhole the definition of a database in a way that is going to prompt me to agree with you, you will be disappointed to learn that yes, I am a human.

1 more reply

j / k navigate · click thread line to collapse