undefined | Better HN

0 pointsTeMPOraL8mo ago0 comments

> I read somewhere, but cannot find the source anymore, that all written text prior to this century was approx 50MB. (Might be misquoted as don't have source anymore).

50 MB feels too low, unless the quote meant text up until the 20th century, in which case it feels much more believable. In terms of text production and publishing, we're still riding an exponent, so a couple orders of magnitude increase between 1899 and 2025 is not surprising.

(Talking about S-curves is all the hotness these days, but I feel it's usually a way to avoid understanding what exponential growth means - if one assumes we're past the inflection point, one can wave their hands and pretend the change is linear, and continue to not understand it.)

0 comments

ben_w8mo ago

Even by the start of the 20th century, 50 MB is definitely far too low.

Any given English translation of Bible is by itself something like 3-5 megabytes of ASCII; the complete works of Shakespeare are about 5 megabytes; and I think (back of the envelope estimate) you'd get about the same again for what Arthur Conan Doyle wrote before 1900.

I can just about believe there might have been only ten thousand Bible-or-Shakespeare sized books (plus all the court documents, newspapers, etc. that add up to that) worldwide by 1900, but not ten.

Edit: I forgot about encyclopaedias, by 1900 the Encyclopædia Britannica was almost certainly more than 50 MB all by itself.

TeMPOraLOP8mo ago

You and 'jerf make a fair point. Assuming you both are right, let's take jerf's estimate (which I now feel is right):

> 50MB feels like "all the 'ancient' text we have" maybe, as measured by the size of the original content and not counting copies

and yours - counting up court documents, newspapers, encyclopaedias, and I guess I'd add various letters to it (quite a lot survived to this day), and science[0], let's give it 1000x my estimate, so 50GB.

For the present, comments upthread give estimates that are in hundreds of terabytes to petabyte range. I'd say that, including deduplication, 50TB would be a conservative value. That's still 1000x of what you estimate for year 1900!

The exponent is going strong.

Thanks both of you for giving me a better picture of it.

[0] - I entirely forgot about https://en.wikipedia.org/wiki/Royal_Society!

jerf8mo ago

50MB feels like "all the 'ancient' text we have" maybe, as measured by the size of the original content and not counting copies. A quick check at Alice in Wonderland puts it at 163kB in plain text. About 300 of those gets us to 50MB. There's way more than 300 books of similar size from the 19th century. They may not all be digitized and freely available, but you can fill libraries with even existing 19th century texts, let alone what may be lost by now.

Or it may just be someone bloviating and just being wrong... I think even ancient texts could exceed that number, though perhaps not by an order of magnitude.

j / k navigate · click thread line to collapse