undefined | Better HN

0 pointstptacek6y ago0 comments

They ate up disk and CPU. It was the nature of NNTP, at least at the serious provider levels, that you had to be able to keep up in real time with a certain high level of responsiveness, or you'd miss your window for posts and there'd be holes in your binaries (because Usenet is perhaps the dumbest imaginable way to transmit a large binary, a Usenet binary was a chain of separate posts, all of which were needed to reassemble the binary).

You could run a Usenet server without binaries, just to host the discussions and source code sharing and whatnot. But customers would have none of it: if you didn't have binaries, you weren't serving real Usenet, and they'd go to ISPs that did, and when ISPs generally stopped serving Usenet because it had become the world's lamest warez and porn distribution network, they moved to centralized NNTP services.

0 comments

5 comments · 1 top-level

webmaven6y ago· 4 in thread

> (because Usenet is perhaps the dumbest imaginable way to transmit a large binary, a Usenet binary was a chain of separate posts, all of which were needed to reassemble the binary)

I just realized that this is superficially similar to how bittorrent works...

tptacekOP6y ago

Not at all. BitTorrent is forward error corrected. It splits binaries up into chunks as an optimization, and ensures that you don't need a precise sequence of chunks to reassemble the file; an error correcting code ensures that you only need k of n chunks. Usenet binaries were literally just raw binaries, uuencoded(!), and split up into n chunks, of which you need all n chunks. Those streams of chunks were then broadcast to every node on the network, despite the fact that only a subset of nodes ever wanted any particular binary.

It was and remains the most batshit file transfer mechanism ever devised.

lvh6y ago

Not that I'd know: but at some point after the nineties Usenet figured out PAR files too, so you wouldn't quite need all the chunks because NNTP servers would drop some of them occasionally -- though to your point, yes, every NNTP server would de facto mirror everything.

300bps6y ago

They actually fixed that problem around 2001! It's been over 15 years since I've done anything with Usenet binaries so I'm shocked I was able to recall the PAR file technology. Essentially, if you posted 50 parts of your encoded binary, you would generally post an additional 5 or 6 PAR files. Downloaders could then utilize a PAR file on a one-to-one basis to replace literally any of the 50 original files. It's been a long time but I used them extensively and it was a good way to correct errors or missing files.

https://en.wikipedia.org/wiki/Parchive

cpach6y ago

Another weird thing is that in the early 10’s, people where still downloading warez over Usenet. Maybe it’s still a thing even today.

1 more reply

j / k navigate · click thread line to collapse