http://www.machinima.com/film/view&id=42271
One of the common themes in the series is Valve's frequent choice to delay shipping in order to get things right. Gabe Newell says something like, "Your software is late for a while, but it's bad forever." As far as I can tell, this has worked extremely well for them.
Which makes me wonder what the difference is between Valve's games and Gmail. I think it has something to do with their games being about experiential storytelling, and Gmail is a tool to accomplish tasks. I imagine most software falls somewhere on a spectrum between the "tool" and "experience" endpoints and I wonder if other software besides games might be hurt more than helped by "release early and often."
As for Valve's multiplayer games, they do release often. Team Fortress 2 gets sometimes weekly updates (balancing and bug fixes mostly), with large content releases every few months.
On the web, your releases update 100% of your client base instantly.
Though I do agree that depending on your market, "release early and often" can hurt you. Getting a bad reputation early can be hard to shrug off later.
Perhaps that's his point about bugs vs new features. Releasing often works when you are adding more good bits, not so much when fixing broken bits that should never have been released in the first place.
That said, I think there are always techniques for using the early, often, listen approach with private betas, multivariate testing, etc.
Release doesn't necessarily mean releasing to the public, and does not even necessarily mean releasing something user facing.
Those 3 versions of GMail could have been one version that was most useful. There was no time constraint for it since it was a side-project.
The links to nowhere sounds like a good idea except I don't think it indicates interest in the feature being built. A link's text is usually very short and you don't really get any demographics from a click. If you knew 15 to 20 yr olds were clicking a link but the rest of the site was mainly used by 30 to 40 yr olds, you might want to spin out the link as a separate site instead of just a new feature or something. But how do you know how interested people really are? :/
In retrospect, a good number of those complaints came from Reddit users when we were just getting out the door. Our startup was only a few weeks old then and I have to say that the feedback felt overwhelming.
In the end, the decision to remove Flash in the builder was a good one because it resulted in a much simpler interface that was easier to use, but to say that it should have been obvious or that it really mattered, would be overstating it.
So I'll change what I said...It shouldn't have taken you more than 5 min to realize that GNU/Linux users wouldn't have been a huge percentage of your users and you could have gotten away with ignoring their advice on Flash/Javascript.
It's a good case against releasing early and asking for feedback...you have no idea if the market segment using your product is the one you will be targetting in a few months.
My criticism was based on the stuff PB said. All of that functionality could have been done in a single version but PB didn't want to waste company resources unless he could offer up proof that they were being used wisely.
There's nothing wrong with that, but to say that it's a good development model is awful. How long did GMail stay in beta for? It never really shipped.
But it does still sound a bit dodgey, I think because to get accurate feedback, you necessarily need to fool people. For example, if you had a link that stated it was a proposed feature, it would affect the clickrate. Or maybe it wouldn't? Maybe interest would be enough. Perhaps this could be experimentally tested.
(The majority of my hack is not for customers.)
