David Rice - Latest Comments in The Magical Fail Whale

Re: The Magical Fail Whale

David Rice — Tue, 24 Jun 2008 14:20:00 -0000

Stephen, I don't think you're on crack. Agile Methodologies are great but when you don't apply good Domain Modeling to the problem at the same time you end up with a solution that works... until you realise there's a problem, in a lot of cases it may already be too late. In that vein I like to think for a good while of the simplest AND most sensible solution.

I don't think there are any other rails apps with scaling problems as big as twitter, all of 37signals products are "naturally sharded" by account/user structure. Campfire did require a part to be swapped out of Rails, the backend that responds to the Javascript polling, the (simply) rewrote it in C.

Re: The Magical Fail Whale

Paul Campbell — Tue, 24 Jun 2008 14:11:00 -0000

Stephen,—

The Twitter problem could be described as being as a result of an agile approach, but I can't really agree with is being as a weakness.

From what I've heard, the original app was built and launched in 6 weeks. Now that's pretty agile.

The scaling issues are twofold:
Scaling anything requires two things:
1) The NEED to scale
2) a scaleable architecture
3) $$$ to pay for that architecture

Twitter was a fast, cheap job that took off, and took off big. The risk was minimal. A thousand Twitters could have launched on the same day and if they flopped, not a lot would have been lost.

Twitter did take off, and very soon, as Dave rightly says, they found themselves with a platform that wasn't designed to do what Twitter users ultimately started doing: sending messages, having conversations.

I'm willing to bet, that had they invested in scaleable architecture first, and built their system from the ground up as a distributed messaging platform, then none of us would ever have got the service, they probably still wouldn't be launched and they wouldn't stand a chance.

Re: The Magical Fail Whale

mj — Tue, 24 Jun 2008 13:58:00 -0000

Twitter folk were admitted that they architected Twitter like a content management system when it should have been a messaging system.

As I have to work with Rendezvous every day and see millions of messages passing, I have to wonder....

Re: The Magical Fail Whale

Stephen — Tue, 24 Jun 2008 13:54:00 -0000

I wonder if the problems Twitter has (and some 37Signals apps) are more related to weaknesses in current implementations of Agile Methodologies than anything else?

JF said that Campfire is AJAX based and not Comet because they just wanted "to get things done", or be Agile. And a similar situation seems to be at the root of Twitter's problems.

Or maybe I'm on crack?