So once again I've tried to pre-post things and once again the system lost them after a couple days.  Which means I have no automated posts to put up.  So this is it for me until the 9th (again, I'm on Blog-cation).  But there was one thing I'd been thinking of lately so I figured I'd throw out a quick post on why I don't use Friendfeed or Twitter.   Before I get started let me just say that this is a "me" thing.  There are a lot of things I say here that I think should apply to everyone but this isn't one of them.  If you love these services than more power to you. 

But you won't catch me using them much (I do have a FriendFeed account just so I can post the occasional comment)

Here's the thing, when you come to this blog (or read the feed) it takes time.  Even if you only skim the entries I'm taking a small part of your time away from you.  So when I write an entry I feel that's something I need to keep in mind.

It's a responsibility as far as I'm concerned.  You've chosen to honor me by giving me a piece of your time so I in turn need to make sure I only make use of that honor when I have something worthwhile to say.  Because I'm, in my own small way, cluttering your life.  I'm putting one more piece of information on your plate that you need to deal with.

So I owe it to you to make sure that piece of information is at least worth the time it takes for you to skim it and move on.  

In my view Twitter and Friendfeed are contrary to that goal.  They are services designed to make it as easy as possible for people to post which in turn encourages people to put very little thought in to what they chose to share.  That's my fear.  If you like this blog enough to follow me to either of these services I would, in my own mind, be taking advantage of the trust you put in me by coming here if I acted in that way. 

Essentially I'd be hijacking even more of your time to tell you something that doesn't have any value to you (like what I had for lunch). 

So that, in a nutshell, is why you won't find me on Twitter or Friendfeed.  I don't have anything against the services and I certainly don't fault anyone else for using them but I personally don't feel comfortable with it. 

That said, if you're curious I just ate a Pastrami Sandwich and am now going to buy a Printer. 

So this is kind of funny...

On February 8th, 2008, Israel of AssetBar.com wrote a really informative post entitled "Twitter Proxy: Any Interest?"  In it he pointed out this fundamental problem with Twitter...

Nothing is as easy as it looks. When Robert Scoble writes a simple “I’m hanging out with…” message, Twitter has about two choices of how they can dispatch that message:

1. PUSH the message to the queue’s of each of his 6,864 followers, or
2. Wait for the 6,864 followers to log in, then PULL the message.

 

The trouble with #2 is that people like Robert also follow 6,800 people. And it’s unacceptable for him to login and then have to wait for the system to open records on 6,800 people (across multiple db shards), then sort the records by date and finally render the data. Users would be hating on the HUGE latency.

This post was later quoted by Dare Obasanjo which led to an angry response from Robert Scoble where he said...

First of all, Twitter doesn’t store my Tweets 25,000 times. It stores them once and then it remixes them. This is like saying that Exchange stores each email once for each user. That’s totally not true and shows a lack of understanding how these things work internally.

Second of all, why can FriendFeed keep up with the ever increasing load? I have 10,945 friends on FriendFeed (all added in the past three months, which is MUCH faster growth than Twitter had) and it’s staying up just fine.

Now, with no offense to Scoble, this is a fundamentally ignorant assessment of what is actually going on.  A fact that almost every technical minded person pointed out to him.  The best of those posts, imho, was from Nick Halstead where he says...

In a recent post Robert Scoble tries to explaining how twitter works by saying that twitter is using some form of ‘pivot table‘ - (my terminology for what he explains) and says that a model that others have put forward (i.e. a de-normalized system of inserting messages into everyones queues) was akin to microsoft exchange, now these two examples are so horribly not connected - and I won’t rant about how BAD exchange is efficiency wise, but please Robert do not get into any technical arguments please.

(Its a great post that the above quote doesn't do justice so everyone should check it out)

This led to the same question being presented to the authors of the official Twitter technology blog.  Their reply was to basically confirm what everyone had been saying...

charles asks if there's anything users can do to lighten our load. The events that hit our system the hardest are generally when "popular" users - that is, users with large numbers of followers and people they're following - perform a number of actions in rapid succession. This usually results in a number of big queries that pile up in our database(s). Not running scripts to follow thousands of users at a time would be a help, but that's behavior we have to limit on our side.

A response that Scoble quickly took offense to saying in his link blog...

“This is total bullshit. Why do I have 11,556 subscribers on FriendFeed, I'm FAR FAR FAR FAR more active on FriendFeed, and yet FriendFeed never has gone down on me? Also, Twitter went down at its first SXSW before I had a ton of followers there. Twitter has major problems, they still don't have a good engineering answer, and so they are blaming their most popular users. Great. We get the message. We'll go someplace where there's a good engineering team. You know, the guys who invented Gmail and Google Maps? They are the ones behind FriendFeed. See ya Twitter!”

So there you have it, a little background on what has become a huge mess for Twitter. 

Honestly, I don't have much of an opinion on the human drama part of this (other than to find it pretty amusing).  Is Scoble being unreasonable?  Absolutely.  But that's somewhat understandable when you consider the fact that everyone is pointing their fingers at him when he essentially did nothing wrong.  The truth is, Twitter loved Scoble signing up 25,000+ followers when it meant drawing users in to the service. 

So for him to be slapped on the hand now for it is a bit obnoxious. 

On FriendFeed, there are two issues there.  One,  the model by which people monitor other people's FriendFeeds is not like the model that Twitter uses which lowers their back end problems and two, FriendFeed has far fewer users.  If they were trying to deal with as many users as Twitter I suspect they'd be having problems as well. 

One final point I'd like to make is regarding the root cause of Twitter's woes.  A lot of people have tried to lay this on the feet of Ruby on Rails which is a bit unfair.  With that being said, Twitter's problems do belong at the feet of the "Rails Philosophy" which was set down by its creators 37Signals.  Here's a quote from their e-book "Getting Real"...

You don't have a scaling problem yet

"Will my app scale when millions of people start using it?"

Ya know what? Wait until that actually happens. If you've got a huge number of people overloading your system then huzzah! That's one swell problem to have. The truth is the overwhelming majority of web apps are never going to reach that stage. And even if you do start to get overloaded it's usually not an allor-nothing issue. You'll have time to adjust and respond to the problem. Plus, you'll have more real-world data and benchmarks after you launch which you can use to figure out the areas that need to be addressed.

I've never thought much of the 37Signals gang and it is quotes like the above one that are the reason why.  Putting off your most difficult technical tasks until later is an utterly stupid thing to do and is, as Twitter is now finding out, disastrous if you can't quickly address the problems.  37Signals wouldn't necessarily know that though because their most popular application has fewer users than a program I wrote at 19.  Yet they still speak as and are treated like they are leading experts in application design. 

They're not. 

Not only that, a lot of their advice is outright bad as this Twitter solution proves.  This is all symptomatic of a larger problem on the web which is bloggers' not questioning who they treat as an authority.  Anyone who realistically looks at 37Signals will see that they are still a fairly small development firm.  That doesn't mean they haven't done some impressive things or that their opinion has no merit at all.  But it needs to be put in context and people don't seem to be doing that. 

Had the Twitter architects done that they wouldn't be in the situation they're in now. 

Addendum:  This occurred to me as I was hitting the publish button of the above post.  When you go to the bank for a new business loan what is the first thing they ask you for?  A 2-Year Plan.  That's because no sane institution lends money to anyone who has no idea where they are going or how they plan to get there.  Given that I ask you, doesn't the same apply to technology issues and in particular scaling?