I've always thought RoR's tight coupling with traditional SQL would be a serious scaling issue. RDBMSs in the traditional sense just dont scale easily. Google figured this out a while ago (hence BigTable). And RoR reads and writes soo much that you are left with a burden on the DB that might have been better mitigated by doing more of the processing client side and persisting with a simplified cloud-based service rather than a single, central RDBMS.
Not to stir up a hornets nest, but I really think this is a limitation of RoR that should be addressed in the future. Abstract out the SQL so other methods of persistence are possible.
EDIT: I am not trolling RoR. I am simply proposing that it might be better to be able to use it with something other than traditional SQL for persistence of data, so that traditional RDBMS can be replaced with a simplified cloud-based service such as SimpleDB.
I speculate this might be possible now using Ruby -> JRuby -> JPA -> SimpleJPA (or any other MySQL/SimpleDB wrapper) -> SimpleDB. I dont want to even get into how much work that would be though. It needs to be easier I think.
Startup idea maybe?
edit: their best bet is to try to find someone who is burned out from working on wall street applications.
Having said that, it would still be fun to rewrite Twitter in Erlang... :)
Which is exactly what they're doing:
One of our Oracle databases sits on a 24 core itanium box
That sort of hardware + Oracle can munch through some serious work - perhaps that is the database solution, but I believe it doesn't come cheap!
The new rewrite will give the new architect time to vest, until it fails, like big rewrites usually do. If you're not sure how to fix things, the big rewrite is a perfect solution, really.
I hate it when non-programmers write about programming! It's a giant festival of stupid.
It doesn't matter what he says. All that matters is that you notice.
After 30 minutes, you are the #1 response to the #2 thread.
So people click through.
Cha-ching.
Since their requirements seem like they're pretty static now, I would think writing a backend in C wouldn't be out of the question.
i am really surprised how facebook, so quikly found the skillset to build a 10,000 server site and how come, twitter didnt! just why ...
its the skillset, but again why was facebook successful in aquiring the skillset when twitter wasnt! ... why is twitter considering a rewrite and facebook didnt ... they just grew !!
With their recent fundraising, they might even be able to run a couple parallel fixup projects in competition, and let the best win.
The reason you'd ditch Java or PHP is because they are lacking a lot of higher-order constructs. (PHP epsecially. Statically-scoped variables? First class functions? An object system that isn't a hilarious joke? Unicode support? PHP doesn't have 'em.)
When I wonder why Facebook doesn't have problem scaling their chat? Oh, erlang + flat files.
I can't wait for Twitter to move to Java or PHP so people finally STFU about Rails.
Twitter moving away from rails doens't make rails look bad, it makes everyone at Twitter look like the biggest jackasses in history.
"We can't get this wrench to hit this nail hard enough, so lets buy another brand of wrench instead of a hammer and see if that works."
A quick Google of "erlang facebook chat flat file" returns your initial post, would love to read more about that if have anything.
All tools tend to look like hammers when all problems are seen as a subset of nails.
Sounds to me like Mr Arrington has an axe to grind and is putting that ahead of objective journalism.
Daniel
Has ror become a cult or something? It's just a language, and if it doesn't fit well with the application, best to move on and try something else.
What they should do instead is to slowly rework their code, instead of doing a massive rewrite.
http://twitter.com/ev/statuses/801530348
(Ironically, Twitter was down when I first tried to get the URL for this.)
(With an HTTP Status header being dumped to the screen instead of, you know, sent as a header.)
You could probably rewrite the front end in applescript and you'd see little difference, and if you did just throw more servers at it. The problem is probably that theres a DB at the heart of all the message crazyness, you can't do that and expect to scale.
Surely they are just rewriting the slower core areas of the API in something faster/more distributed. There is no reason they'd have to rewrite the entire thing from scratch.
"This is exactly what makes Rails a ghetto. A bunch of half-trained former PHP morons who never bother to sit down and really learn the computer science they were too good to study in college. [...]
After this, I started thinking, since it was the first realization that I picked the absolute last wrong community to make my living in. They were all pricks, morons, assholes, and arrogant fucks who didn’t care about the art or the craft.
They were all a bunch of little ghetto fabulous princesses trying to make it in this tiny little Rails world and not enough brains between them all to make it happen."
and about the framework
"Now, DHH tells me that he’s got 400 restarts a mother fucking day. That’s 1 restart about ever 4 minutes bitches. These restarts went away after I exposed bugs in the GC and Threads which Mentalguy fixed with fastthread (like a Ninja, Mentalguy is awesome).
If anyone had known Rails was that unstable they would have laughed in his face. Think about it further, this means that the creator of Rails in his flagship products could not keep them running for longer than 4 minutes on average."
Although he does add that
" [...] Rails is a pretty nice idea, and it demolished the Java world I hated so much."
For Twitter, I think some parts of the API need to be in C. But the whole service? I doubt it.
They say most of their traffic is API requests, and I would expect that they would want to optimize that porition of their system. Rails is composed of many components, many of which you don't need for a REST service that talks json/xml. Blaine and other twitters have talked about their experimentments: other app servers (Thin), queueing systems (before building starling), ...
