Now I can do this myself, because I'm all "+337" and what not. (I used trickle as described in the comment[2]. Seemed to work fine. Nice and stable.) But I can't recommend Joe and Jane Consumer to install IPFS and some other thing with a straight face, because they'll say, "Well bittorrent can do it!?" and I don't have a good answer.
Maybe there's an opportunity there to rent IPFS VMs to normal people? I dunno.
https://github.com/ipfs/go-ipfs/issues/3065
https://github.com/ipfs/go-ipfs/issues/3065#issuecomment-415...
So, while we get there, applications/tools can also manage their bandwidth, so the default settings are more gentle with using users bandwidth.
[1] https://github.com/ipfs/community/blob/master/code-of-conduc...
We do not and cannot control the data that each individual node is hosting in the IPFS Network. Each node can choose what they want to host - no central party can filter or blacklist content globally for the entire network. The ipfs.io Gateway is just one of many portals used to view content stored by third parties on the Internet.
That aside, we definitely don't want to 'apply copyright worldwide'. For one, it's not consistent! Trying to create a central rule for all nodes about what is and isn't "allowed" for the system as a whole doesn't work and is part of the problem with more centralized systems like Facebook, Google, Twitter, etc. Instead, give each node the power to decide what it does/doesn't want to host, and easy tooling to abide by local requirements/restrictions if they so choose.
The IPFS network itself is decentralized, there's no central authority that can police copyrighted content on individual nodes.
IPFS is not trying to be an anonymous file-sharing service afaik.
Todays IPFS is a long way from that - any Joe Random can DoS any particular hash by getting their node at the right place in the DHT and blackholing requests.
Key problems in this space:
- decentralized is something techies obsess about but that has as of yet no business value whatsoever. Customers don't ask for it. Centralized alternatives are generally available and far more mature/easy to manage. The business ecosystem around IPFS is basically not there.
- this space is dominated by hobbyists running stuff like this on their personal hardware doing this mostly for idealistic or other non incentivized (i.e. money) reasons. Nothing wrong with this but using it for something real brings a few requirements with it that are basically hard to address currently.
- Filecoin has been 'coined' as the solution for this for years but seems nowhere near delivering on it's published roadmap. Last time I checked it had undelivered milestones in the past. As of yet this looks increasingly like something that is a bit dead in the water / a big distraction for coming up with better/alternate solutions.
- Uptime guarantees for content are currently basically DYI. Nobody but you cares about your data. If you want your content to stay there, you basically need ... proper file hosting. As incentives and mechanisms for others to agree to host your content (aka pinning in ipfs) are not there, this is hard to fix.
- integration with existing centralized solutions is kind of meh/barely supported. We actually looked at using s3 as store for ipfs just so we could onboard customers and give them some decent SLA (bandwidth would be another issue for that). There are some fringe projects on github implementing this but when we looked at it the smallish blocksizes in ipfs are kind of a non starter for using this at scale (think insane numbers of requests to s3). This stuff is not exactly mainstream. Obviously this wouldn't be needed if we could incentivize others to 'pin' content. But we can't currently.
I don't really understand that point. If it works, then it has the business value of saving you all running server and maintenance costs. For most larger businesses these costs may be insignificant and easily recovered, but for small businesses with a lot of customers they can make a huge difference. For example, I'm looking into P2P options for implementing a decentralized message forum in a game-like emulator and it would make no sense to even implement this feature with constant running costs for server space.
Now getting the decentralized data management to work reliably out-of-the-box from behind various firewalls and different platforms, that's the big problem. So far, none of the libraries I've seen are very easy to use, some require a difficult installation and configuration or you need to your own STUN server or gateway, which kind of defeats the purpose.
You absolutely can incentivize others to pin content. Check out Infura (infura.io), Pinata (pinata.cloud), Temporal (temporal.cloud), and Eternum (eternum.io) - these are all services you can pay to host your IPFS data with reasonable uptime. They have an incentive to keep your content around because you pay them to. Filecoin is a distributed solution to that (and making active progress - they have a testnet out with over 5PB of proven storage: https://filecoin.io/blog/roadmap-update-april-2020/), but you don't have to wait for that to land.
Resilio (originally part of BitTorrent) specializes in decentralized storage synchronization for business.
Though we can't see if they are profitable their business use cases seem compelling especially if you're reluctant to otherwise use the cloud.
https://www.crunchbase.com/organization/resilio#section-over...
IPNS performance was abysmal last time I tried to use ipfs (took > 30 seconds a fair amount of the time). Curious to see what it's like now.
Since node IDs are random, node lookup may require multiple interplanetary hops, no?
For instance, from N1 on Earth to the nearest XORwise node in its routing table which might be N2 on Mars, whose routing table finds the target node N3 on Earth.
My impression is that DHTs fall down pretty hard under the latter scenario and are also pretty vulnerable to the Sybil scenario if the attacker has enough resources to mount a really serious attack. They're okay for low-value simple stuff that doesn't have much of an intrinsic bounty attached to it (like BitTorrent magnets), but trying to put a "decentralized web" on top of a DHT seems like a scenario where the instant it becomes popular it will get completely shredded for profit (spam, stealing Bitcoin, etc.).
My rule of thumb is that anything designed for serious or large scale use (in other words that might get popular) needs to be built to withstand either a "nation state level attacker" threat model or a "Zerg rush of hundreds of thousands of profit motivated black hats" threat model. The Internet today is a war zone because today you can make money and gain power (e.g. by influencing elections) by messing with it.
For Sybils: You've left the attack you're worried about pretty vague. IPFS itself doesn't need to tackle many of the sybil-related issues by being content addressable (so only worrying about availability - not integrity) and not being a discovery platform - so not worrying about spam / influence. For the remaining degradation attacks - someone overwhelming the DHT with misbehaving nodes - there's been a bunch of work in this release looking at how to score peers and figure out which ones aren't worth keeping in the DHT.
This becomes hugely problematic the minute you start using IPNS. On one side of the split, the name `foo` can resolve to `bar`, but on the other side, it resolves to `baz`. If you're trying to impersonate someone, then a netsplit would make it easy for you to do so (barring an out-of-band content authentication protocol, of course).
> You've left the attack you're worried about pretty vague. IPFS itself doesn't need to tackle many of the sybil-related issues by being content addressable (so only worrying about availability - not integrity) and not being a discovery platform - so not worrying about spam / influence.
On the contrary, a Sybil node operator can censor arbitrary content in a DHT by inserting their own nodes into the network that are all "closer" to the targeted key range in the key space than the honest nodes. This can be done by crawling the DHT, identifying the honest nodes that route to the targeted key range, and generating node IDs that correspond to key ranges closer than them. Honest nodes will (correctly) proceed to direct lookup requests to the attacker nodes, thereby leading to content censorship.
Honest nodes can employ countermeasures to probe the network in order to try and see if/when this is happening, but an attacker node can be adapted to behave like an honest node when another honest node is talking to it.
> For the remaining degradation attacks - someone overwhelming the DHT with misbehaving nodes - there's been a bunch of work in this release looking at how to score peers and figure out which ones aren't worth keeping in the DHT.
Sure, and while this is a good thing, it's ultimately an arms race between the IPFS developers and network attackers who can fool the automated countermeasures. I'm not confident in its long-term ability to fend off attacks on the routing system.
Check out our most recent research discussion for some thoughts about how we might scale up Sybil resistance in p2p networks: https://www.youtube.com/watch?v=L4SJzoKHKPk
IPFS has been a real pain to work with in the past (the node would just consume all RAM and CPU and had to be restarted a lot), but it's been getting better, which is great to see.
I really hope it gets good enough to run on everyone's desktop machine, since that's the way IPFS is meant to be deployed (rather than just on gateways). It seems that it doesn't take up too much RAM or CPU now, but it looks like it might be a problem bandwidth-wise, if you host some popular content.
Still, great news overall.
The improvements announced are substantial.
Edit : makin jokes about plagiarism people.
Edit: The way I see filecoin working is anyone can post a reward for a file and once the file is provided, the reward is paid. In other words, it's bit like a brokerage that connects downloaders with uploaders. The difficultly is that this brokerage needs to be distributed and resilient.
If that is true, it means that a node can only be profitable if you are freeloading. And if you are freeloading, any price you get will be good which means that it tends to go even further down, perhaps even below the commodity cost. I may be missing something, but I really don't see this going beyond techies with spare disks playing around and definitely no way to run a Filecoin node on a VPS profitably.
Right now if I wanted to compete with Amazon, I could buy 2 or 3 petabytes of storage space and some bandwidth. But nobody would trust me not to lose their data. This is the differentiation that makes cloud storage less of a commodity than it could be. The goal with filecoin is to make it so you don't have to trust me, just some general guarantees about the filecoin network. If amazon for some reason was selling storage on this network, I could compete with them on equal footing (and whoever sold it more cheaply would win).
Many commodities are in close-to-perfect competition. For instance, I don't care if grain comes from England or France, just that it's cheap. While it is very hard to make an economic profit selling grain, many people do farm grain and make enough money to support themselves (The money you pay your workers to survive is one of the costs of producing grain. That's true even if the only worker is you)
If you could make a profit by running a filecoin node on a VPS, everyone would do that. By competing with each other you'd all bring the price down and down until it was no longer more profitable than selling any other commodity (and maybe even lower). So you're right to not expect to be able to do that.
1) Accounting profit: revenue - expenses. This is colloquially what most people mean when they say "profit". To break even you just need to make enough money to cover your explicit expenses. In your example, you would need to make more in Filecoin income than you spent in disks/electricity/etc.
2) "Normal profit": this is a technical term in microeconomics. Basically this takes into account opportunity cost; the cost of not doing some other profitable thing with your time/money. By these standards, to break even you'd not only have to cover the cost of disks/electricity but also the "opportunity costs" of whatever else you could have been doing with those disks/electricity (like mining Bitcoin), or whatever you could have been doing if you never even purchased those disks to begin with (like investing in the S&P or whatever).
3) "Excess" or "economic" profits. These are extra profits on top of whatever you earn in #2. This is the part that goes to 0 for commodities like milk/gasoline/Filecoin but is non-zero for differentiated products like iphones.
For Filecoin specifically, I would expect that normal profits would probably be above the raw costs of disks and electricity, since it takes some effort to purchase and administer all that hardware, but probably below the cost of renting a VPS, since a VPS is typically used for more profitable things than just sitting around and holding stuff on disk.
How does that work in real life?
Of course, in the shorter term there are many different scenarios where the market is slower to adjust. The STORJ network has been subsidizing their node operators with VC money for example, leading to some excellent unit economics for the early adopters.
> The primary focus of this release was on improving content routing. That is, advertising and finding content. To that end, this release heavily focuses on improving the DHT.
> A secondary focus in this release was improving content transfer, our data exchange protocols
Other improvements to data store, libp2p, etc.
source: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md
Wow. Thank you IPFS community.
- Pin management is far to simple for any real use case.
- GC is a complete GC on an arbitrary threshold.
- API is based on the CLI and really weird (ex GET /api/v0/cp?arg={src}&arg={dest}, why don't the parameters have names? Why is it GET?)
- They don't prove a way to upload a directory with references to existing IPFS objects, so you need to manually encode the data.
- The manual encoding uses go-style names, they seem to have let their language choice leak into the protocol.
- Lots of minor issues, such as the API implementing block size limits by simply cutting off the input data at the size limit, this is even before encoding it into the format where the size limit should be calculated and cause data corruption in some cases.
- Their one abstract API for creating directories (MFS) has a number of issues.
- It is incredibly slow for some reason.
- It kindof but not really automatically pins all content recursively references.
I ended up sinking a ton of time into https://gitlab.com/kevincox/archlinux-ipfs-mirror/ and in the end the number of small quirks were incredibly frustrating. I might go back and implement their hash-sharding logic to get the mirror working again but at this point I don't really want to interact with go-ipfs again.
I thought it would be an interesting project to get involved in, and I have a lot of expertise that would be valuable however it appears that it is all a bit of a mess which is a real shame.
Maybe with more success it can be re-written in cleaner fashion, as I said most of the issues are with the implementation not the protocol so there is definitely hope. I do honestly wish all of the success to the project.
Things like distribution of apt packages becomes much more exciting when each computer can choose to redistribute the packages it got to others in the LAN or area, even offline.
It's also very interesting to me how all the visitors to your website become servers as well, so content can never be "hugged to death" or links can never go stale, as long as at least one person has the content somewhere on their node.
That, to me, is huge, as links now go stale with some regularity. Think of all the Geocities sites (and all versions of them) just existing for ever, regardless if Geocities decided to shut down.
For example, here's my site on IPFS:
https://ipfs.eternum.io/ipfs/QmVW6JejQkjLnBJacR8qcZi88WNTMwi...
That can now "never" be lost, as long as someone cares enough about it to visit.
What new thing does ipfs make possible that would make users install the software to use it?
You visit a site, get the index and it references two more files with two URLs:
"./jquery-3.5.0.min.js" "https://cdn.com/4kVideo.mkv"
Your browsers get's those files, the first from the site, the other from a CDN. Your site scales according to the band-width that "cdn.com" is able to provide to you.
In IPFS, you visit a site, get the index and it references two more files with two URLS:
"ipfs://QmWYudWcbX6skKub5wg1Ga3LFh3vbW2k7PWfdqHtDYvAdp" (for fun I used the right address here) "ipfs://QmT9qk3CRYbFDWpDFYeAv8T8H1gnongwKhh5J68NLkLir6"
Your browser gets those files, the first is found in about a billion places. A lot of sites use jQuery so almost everyone has the file's content available, and because you'd already had visited another site with that content-hash, IPFS knew it could just use that one (a perfect safe cross-domain cache-hit). The second is found in less places, but it follows the same logic as a torrent, with data coming from both a IPFS cdn the site uses, along with a few people who've also seen the video, so it loads faster from a few sources than one.
Hope this makes sense. IPFS is really just a decent way to implement websites like a torrent and enjoy the benefits that brings. It's not grand or out of this world, just decent space-saving data-management.
If disagree please explain why.
Basically I host your file and you host my file. With some algorithm to make sure number of seeder for each block never go to 0
Rust needs to start edging out Go for new tools. Kubernetes, Envoy, IPFS, etc. would have benefitted from it.
It might not have been time four years ago, but it's time now.
Rust is a great language, but right now I can start a project in Go and have someone else start a project in Go and have something up and working in a week. Rust still takes significantly longer to learn and/or find skilled developers for. Go is up there with Python in terms of "Get something out now".
This is useful for p2p systems in which you are connected to thousands of peers multiplexing several p2p protocols with each concurrently. You want to be able to switch threads as fast as possible.
The concurrency model is a bit easier to reason about too.
It would be good to see how Rust and others do. Other languages excel at other things (i.e. embedded devices ).
They're very sensible about it, and advocate using the right tool for the job.
https://blog.discord.com/why-discord-is-switching-from-go-to...
A big reason why C is still widely used. A library that's written in C can be used in any other language.
go-ipfs is where the majority of new development happens right now as the desktop/server implementation (compared to js-ipfs=browser & rust-ipfs=IoT) - but if you want to prove that Rust is clearly better/faster _in general_ - have at it: https://github.com/ipfs-rust/rust-ipfs ;)
I think the thing jbenet was selecting for back in 2013 was concurrency support & modularity, and golang is still a decent choice for that. Rust 1.0 didn't happen until 2015 after the go-ipfs alpha was already out - but agree it's made awesome progress since then!
But fair point :)
And do IPNS entries finally last more than 30 seconds? It would be nice to not have to constantly keep a node up just to have an IPNS entry.
Here's what's new for IPNS in the release: https://github.com/ipfs/go-ipfs/blob/master/CHANGELOG.md#ipn...
Like this https://blog.ipfs.io/2020-02-14-improved-bitswap-for-contain...
Note that this is also similar to Kraken[1] from Uber, and Dragonfly[2] from Alibaba. Facebook also does container and artifact distribution using BitTorrent, but I can't find a good reference to it.
the py-ipfs-http-client library is also actively maintained (but I think needs some small changes to work with IPFS 0.5): https://github .com/ipfs-shipyard/py-ipfs-http-client/
The path to upgrade the dependencies is probably to run `go get -u <dep>` for the direct dependencies you're including, and then fixing errors that pop up from doing so.
Usually take go-ipfs's go.mod as a guide on what versions to use.
I already ported the Go code to iOS once, and it wasn't that painful, but functionally it would be a lot more useful as a C++ code base. I don't build browser apps - I feel there is still a need for native apps especially in this particular space.