Anyway, I came to the same conclusion as the author, but several years ago: in the end, nothing is actually decentralized, and maintaining this illusion of decentralization is actually costly, for no real purpose (other than the initial enjoyment of playing with a new tech, that is).
So I stopped maintaining it a few years ago. That decision was also because of the growing involvement of some of these projects with blockchain tech that I never wanted to be a part of. This is also why I cancelled my article series in 2600 before publishing those on IPFS and ZeroNet.
[1] See for example this archive of my HN profile page from 2016 with the link to it: https://web.archive.org/web/20161122210110/https://news.ycom...
It makes me more sense to me that someone would be much more willing to serve large databases and neural network weights that they actually use everyday, rather than 'that one guys website they went to that one time'.
I'm very surprised it's not as popular, if not more popular to just have @iroh-memoize decorators everywhere in people's database ETL code.
That's a better use case (sense the user has a vested interest in keeping the data up) than helping people host we sites.
Serving things that are mostly or nigh-exclusively used by machines connected to the power grid (and, ideally, great and consistent Internet connections) is a much better use case.
I believe their data is stored in a p2p - it might interest you!
Do you have any writing (blog posts, HN comments, etc.) where you explore this thought more? I'm in the thick of building p2p software, very interested in what you came to know during that time.
That is not to say that all p2p software is bad, especially since we call p2p a lot of things that are not entirely p2p. For example, BitTorrent is a p2p software, but its actual usage by humans relies on multiple more-or-less centralized point, trackers and torrent search engine.
And if you try to tackle centralization directly (like banning routers or something), you will often create an anti-centralization regulator, which is, you guessed it, another form of centralization.
So your decentralized P2P network is either small and works good, medium and works not so good, or large and not actually decentralized.
The best P2P networks know their limits and don't try to scale infinitely. For human-oriented network, Dunbar's Number (N=~150) is a decent rule of thumb; any P2P network larger than that almost certainly has some form of centralization (like trusted bootstrapping/coordination server addresses that are hard-coded in every client install, etc.)
For example, true p2p can only happen if you meet with someone and use a cable, bluetooth or local wifi. Anything over the internet needs to pass through routers and *poof* decentralization's gone and you now need to trust servers to a varying level of degrees
Decentralized, globally accessible resources still take some kind of coordination to discover those resources, and a shit-ton of redundant nodes and data and routing. There is always some coordination or consensus.
At least, that's my take on it. Does not tor have official records for exit and bridge nodes?
Someone is always going to want a short, unique, and memorable name. And when two people share the same name (McDonald, Nissan, etc) there needs to be a way to disambiguate them.
If people die and are unable to release a desirable name, that just makes the whole system less desirable.
I know one of the canonical hard problems in Computer Science is "naming things" and this is a prime example!
Blockchain enthusiasts have a history of talking out of their ass and being susceptible to the lies of others.
But for any other usage, it cannot work, blockchain are useless. Someone or something somewhere has to make sure either that what's written on the blockchain corresponds to the real world, or to make the real worlds corresponds to what's written on the blockchain. Either way you need to have a kind of central authority, or at least trusted third parties. And that means you don't actually need a blockchain in the first place. We have better, more secured, more efficient, less costly alternatives to using a blockchain.
Back to name resolutions.
Virtually no one is going to actually host locally the blockchain where all names are stored. That would be way too big and could only get bigger and bigger, as a blockchain stores transactions (i.e., diffs) rather than current state. So in practice people and their devices would ask resolvers, just like they currently do with DNS. These resolvers would need to keep a database of the state of all names up-to-date because querying a blockchain is way too inefficient, running such a resolvers would be a lot more costly than running a DNS servers so there would be less of them. Here we just lost decentralization which was the point of the system. But that's just a technical problem. There is more: what if someone gets a name and we as a society (i.e., justice, whatever) decides that they should not be in control of it? Either we are able to enforce this decision and it means the system is not actually decentralized (so, we don't need a blockchain), or we can't, and that's a problem. What if a private key is lost, the associated names are gone forever? What if your private key is leaked by mistake and someone hostile take control of your name?
Using a blockchain for names resolution doesn't actually work, not for a human society.
I very much enjoyed your articles on Tor and I2P :) I2P was entirely new to me, so I found that particularly interesting. I did idly wonder when the next article was coming, so I’m glad I didn’t just miss it in some issue. Totally understand where you’re coming from.
Oh yeah, I was referring strictly to IPFS+ENS websites. I have been working with it for several years so my mind goes for this use-case automatically.
It reminds me a bit of an early Go project called Upspin [1]. And also a bit of Solid [2]. Did you take any inspiration from them?
What excites me about your project is that you're addressing the elephant in the room when it comes to data sovereignty (~nobody wants to self-host a personal database but their personal devices aren't publicly accessible) in an elegant way.
By storing the data on my personal device and (presumably?) paying for a managed relay (and maybe an encrypted backup), I can keep my data in my physical possession, but I won't have to host anything on my own. Is that the idea?
> By storing the data on my personal device and (presumably?) paying for a managed relay (and maybe an encrypted backup), I can keep my data in my physical possession, but I won't have to host anything on my own. Is that the idea?
We're hoping to give that exact setup to app developers (maybe that's you :). We still have work to do on encryption at rest to keep the hosted server "dumb", and more plumbing into existing app development ecosystems like flutter, expo, tauri, etc. but yes, that's the hope. Give developers tools to ship apps that renegotiate the "user social contract".
With Iroh + CRDTs you have three choices: 1. Use iroh's connection & gossip layers in conjunction with a mature CRDT library like Automerge or Y.js. 2. Build a more sophisticated CRDT on top of iroh documents. 3. Worry a little less about weather your data structures form a semilattice & build on a last-writer wins key-value store (basically: just use documents)
We've seen uses for all three. Hope that helps!
I agree with this fully. But as said elsewhere, it's kind of far away from that, and also slightly misdirected.
Imagine asking someone to get started with web development by sending them to https://www.ietf.org/rfc/rfc793.txt (the TCP specification). Filecoin is just the protocol, and won't ever solve that particular problem, as it's not focused on solving that particular problem, it's up to client implementations to solve.
But the ecosystem is for sure missing an easy to use / end-user application like Dropbox for storing files in a decentralized and secure way.
Distributed file storage, if done correctly, can be a transformative technology. And it can be even more revolutionary implemented at the OS level.
That fucker's a PIG on cpu and ram.
Clearly much more going on but take a machine that can serve 10k req/s with [insert 100 things here] without flinching and watch it maybe, just maybe, do 10 with IPFS.
I'm not kidding.
This has always been my major UX gripe with IPFS. The fact that `ipfs add` in the command line does little but generate a hash and you need to actually pin things in order to "seed" them, so to speak. So "adding a file to IPFS", in the sense of "adding a file to the network", requires the user to know that (1) the "add" in `ipfs add` does not add a file to the network, and (2) you must pin everything you want to replicate manually. I remember as recently as 2021 having to manually pin each file in a directory since pinning the directory does not recursively pin files. Doing this by hand for small folders is okay, but large folders? Not so much.
More importantly, the BitTorrent duplication problems that IPFS has solved are also solved in BitTorrent v2, and BitTorrent v2 IMO solves these problems in a much better way (you can create "hybrid torrents" which allows a great deal of backwards compatibility with existing torrent software).
This isn't a UX issue, but another thing that makes it hard for me to recommend IPFS to friends is the increasing association with "Web3" and cryptocurrency. I don't have any strong opinions on "Web3", but to many people, it's an instant deal-breaker.
Would BitTorrent also be suitable for hosting embeddable content? I haven't seen that yet. A magnet URL is mainly a file hash and doesn't seem to encode a particular peer server, kinda like IPFS. But every time I've torrented Ubuntu, it's taken half a minute just to find the peers.
Anyone who has tried to torrent an old movie or lesser-known television show knows this is simply not true.
Same as IPFS: gateways can exist. It's not specific to Bittorrent, or IPFS.
> A magnet URL is mainly a file hash and doesn't seem to encode a particular peer server, kinda like IPFS.
Magnet links can include a HTTP server that also hosts the content
Can‘t you just use a glob?
In other words: Once a few big websites are established, no small website will ever be able to gain traction again because the big websites are simply easier to reach and thus more attractive to use. And just like an unpopular old torrent, eventually you run out of seeders and your site is lost forever.
One can argue about the value of low traffic websites, but I got to wonder: Who in their right mind thinks "Yeah, I want to make a website and then have others decide if it's allowed to live". Then again, maybe that kind of "survival of the fittest" is appealing to some folks.
As far as I am concerned, it sounds like a stupid idea. (Which the author goes into more detail, so that's a good write up)
So if you're not going to be publishing something that will always have multiple copies floating around, why use IPFS?
> The blog you’re reading now is built with Jekyll and is hosted on my own 10$ server.
> don’t get me wrong, I’m still an IPFS fanboy.
...how could you still be a fanboy? When IPFS cannot fulfill even the most basic function of globally serving static content, why does it deserve anyone's interest? It's not even new or cutting edge at this point. After 8 years of development how can the most basic functionality still not work for even an expert?
They're having growing pains due to scalability problems and some libraries, like Helia (the JS library of IPFS), being new. I guess I'm also quite stubborn in wanting to do it my way, without the aid of any services, and for the content I pin to be available in all places, including Helia in the browser.
My original aim was to write an IPFS blogging engine for my personal use, so I needed some dynamic loading from IPFS there.
Now I switched to Jekyll, and it would be easier to host the blog on Github indeed, but I'm kind of playing a quixotic game of trying to minimize the presence of Google/Microsoft/Amazon and other big-tech in my life.
[0] https://github.com/peergos/peergos [1] https://peergos.org
it is simple enough and free even on hosted solutions, and it keeps my Netlify and Vercel free during spikes in traffic
but the availability issue is perplexing, just like OP encountered
some people just randomly wont be able to resolve some assets on your site, sometimes! the gateways go up and down, their cache of your file comes and goes. browsers dont natively resolve ipfs:// uris. its very weird.
And so naturally relays pop up, and the relays end up being more convenient than actually using the underlying protocol.
As far as I understand this isn't a solved technical problem - but mostly a cultural quirk and probably just due to how the early torrent clients were configured
There is for instance a major Chinese torrent client (that name escapes me) that doesn't seed by default - so the whole thing could have easily not worked. If IPFS clients don't seed by default then that kinda sounds like either a design mistake or a "culture problem"
I've always wondered if there was a way to check if a client is reseeding (eg. request and download a bit from a different IP) and then blacklist them if they provide the data (or throttle them or something)
But if you want to fence them off, you can use private trackers with ul/dl ratio accounting.
Having tried fringe technologies over the years, spun up a server and run them for a few months, struggled and seen all the rough edges and loose threads, I often come to the point of feeling - this technology is good, but it's not ready yet. Time to let more determined people carry the torch.
The upside is:
- you tried, and so contributed to the ecosystem
- you told people what needs improving
Just quitting and not writing about your experience seems a waste for everyone, so good to know why web hosting on IPFS is still rough.
Wonder if there is actual use or need for such thing.
My personal website served from a peergos gateway (anyone can run one) is https://ianopolous.peergos.me/
If you want to read more check out our book: https://book.peergos.org
Services like https://dappling.network, https://spheron.network, https://fleek.co, etc?
I've seen some DeFi protocols use IPFS to add some resiliency to their frontends. If their centralized frontend with vercel or whatever is down, they can direct users to their IPFS/ENS entrypoint.
(Also I heard it's computationally costly, but I am not sure if it's true, I can't imagine why it would be the case actually.)
As a result it's actually more centralised than web, there are like 3 pinning services that everyone uses. At which point I don't get the extra hoops.
E.g there is already helia.
Just waiting for running a node on the browser tab to become insignificant, resource wise.
The current status is that I plan to bring back IPFS usage more in the future for my project, but will wait for the ecosystem to mature a bit more first with regards to libraries.