undefined | Better HN

0 pointssamstave6y ago0 comments

If my calculations are correct;

A typical sms is 140 bytes...

At 40B messages

Thats 57 terabytes a day

But imessage isnt limited to 160 characters

So lets double that.

And about 100 TB a day as of 2014 - so lets triple that.

And assume 300 TB per day in messaging

And lets divide that in two for between iMessage and sms

But since android tops ios in global qty of devices..

Lets double it. And where in the past the Philippines was known to be the top users of sms, but with ayala scaling up over the last years... lets quadruple that number and so i would estimate 600 TB a day in just iMessage and sms alone.

0 comments

airstrike6y ago

I love this. Reminded me of one time some 15+ years ago I did some back-of-the-envelope math like yours but on people using colored characters on every message on IRC w/ scripts and how much meaningless data that generated... it's astonishing how quickly you can get to the PB scale

samstaveOP6y ago

So the question is compression ratio...

So lets assume that amazons cia/nsa op datacenter is receiving these 1PB a day messages.

Lets assume they use a tool to remove certain very common words and an algorithm whic can replace the hidden words later if they want to restore the messages.

Text compresses really well.

And if you have a replacement dictionary, even better.

The most commonly used words:

https://en.m.wikipedia.org/wiki/Most_common_words_in_English

So you have a simple key-value store....

And compress that text even further.

So what can we get a PB down to if we use that thinking...

—-

Hey has anyone noticed that palantir Has stopped their employees from wearing their swag on bart and disappeared from reddit and HN?

Oh and all the nlp competitors went silent??

Yeah, deep state data processing market is booming.

samstaveOP6y ago

Ok so there are 8 trillion messges ent per N

Im building a formula for that cost... gimme a sec

samstaveOP6y ago

I need help with some math... any stats/bi/experts out there willing to converse on this?

saagarjha6y ago

iMessage does file transfer too!

samstaveOP6y ago

Yeah id say its likely closer to a PB a day globally.

j / k navigate · click thread line to collapse

0 pointssamstave6y ago0 comments

If my calculations are correct;

A typical sms is 140 bytes...

At 40B messages

Thats 57 terabytes a day

But imessage isnt limited to 160 characters

So lets double that.

And about 100 TB a day as of 2014 - so lets triple that.

And assume 300 TB per day in messaging

And lets divide that in two for between iMessage and sms

But since android tops ios in global qty of devices..

0 comments

airstrike6y ago

samstaveOP6y ago

So the question is compression ratio...

So lets assume that amazons cia/nsa op datacenter is receiving these 1PB a day messages.

Lets assume they use a tool to remove certain very common words and an algorithm whic can replace the hidden words later if they want to restore the messages.

Text compresses really well.

And if you have a replacement dictionary, even better.

The most commonly used words:

https://en.m.wikipedia.org/wiki/Most_common_words_in_English

So you have a simple key-value store....

And compress that text even further.

So what can we get a PB down to if we use that thinking...

—-

Hey has anyone noticed that palantir Has stopped their employees from wearing their swag on bart and disappeared from reddit and HN?

Oh and all the nlp competitors went silent??

Yeah, deep state data processing market is booming.

samstaveOP6y ago