In my experience, the sort of people who talk about open floor plans for "collaboration" tend to be incompetent managers. Instead of managing, leading, and making decisions, they believe that by manipulating some imaginary rate of collaboration, workers will "collaborate" their way to generating value and wealth. It's a managerial form of cargo cult behavior.
Personally, my best years have been in a shared office with someone whose presence I genuinely enjoyed. It was less about collaboration and more about keeping the mood up throughout the inevitable daily slog through painful work that, truthfully, nobody wants to do.
I'm a fan of 4-8 person teams attacking problems together and I've found flexible cubicle arrangements to be very effective. At one company, we could change around our cubes (albeit not terribly frequently) and it was great to be able to create a multi-person office with a conference table for your team (e.g. PM, designer, developers). We had tight collaboration and our own private space without being distracted by the larger company. Of course, this style of organization also affect project management, etc, but it was quite effective.
In retrospect it's as simple as it's obvious: people should be allowed to setup their offices in a way that works for them.
Interestingly enough this all comes from the original patterns book, the one which concerned itself with actual architecture.
It essentially states that people should create their own space and that structure and order should only be imposed through adherence to shared general principles.
Many of these principles seem so simple and obvious yet are so often forgotten in practice, such as that the wall should be far away enough to give your eyes a chance to relax.
Another one was that the sounds you heard had to be similar to that of your own, which seems strikingly true in my own experience.
This also explains why programmers get horrible treatment, equity-wise.
Does Fog Creek still give developers individual offices? Microsoft? Joel Spolsky wrote about an office design where the walls were slanted and interior windows were present so each office had windows on 3 sides (but you couldn't really see each other). http://www.joelonsoftware.com/articles/BionicOffice.html
I'm also a fan of flexible furniture that can be moved around easily but have never really seen it used effectively. And I'd like to see better incorporation of stand-up positioning (I'm convinced sales people are more effective when standing; feels more powerful).
Surely the open plan is not the answer but who's searching these days? Do any YC (or equivalent) startups have individual offices?
Microsoft: Depends on the team, but mostly no.
I think this is the simplest explanation for the "inexplicable" trend. Companies see other young, hip, growing companies/startups do it, so they feel that they should do the same thing if they want to be a young, hip, growing startup.
Now, we have over 20 employees, 13 of them spread throughout the US and Canada (all programmers). I actually just gave a presentation to new employees based here, "Parse.ly's Distributed Team: Open Source meets Open Plan": http://pixelmonkey.org/pub/distributed-teams -- it discusses some of the pro's and con's of our setup.
I'm a software engineer who has only worked in open plan offices, so maybe I don't know the joy of working in an office, but there is definitely a different feel from the one office where the managers/partners had their own offices and everyone else was open plan vs everyone in the same couple rows of desks.
[1] http://jasonlefkowitz.net/2013/03/how-winners-win-john-boyd-...
Open offices are different. They have been very common in Europe for years. I remember being astonished visiting offices in Switzerland back in the 90's with folks sitting at desks just out in the open. And while I found it abhorrent (I had an office back at HQ, with a window no less), these people did seem to get things done.
It is possible Tech companies in the US are "catching up" to the rest of the world with regard to this stuff. I thought quad cubes at Google were a bit much, (and being that close to three other people can be uncomfortable) but no one was arguing that Google was unproductive or 'bad.'
Headphones are great for shutting out the noise if you want that.
And yes, I invented these shortly after my work area was converted to an open floor plan.
I think it has to do with culture. Socializing during office hours is encouraged as employees tend to be more productive if they are in a good mood.
Cubicles are universally perceived as bad and boring to be in, as you cannot socialize.
Unfortunately, those two are at war with each other. If I can easily and quickly get information from a co-worker, then they can also get it from me. But that easy interruption, which minimizes the disruption to my co-worker's work, also breaks my train of thought.
The private office/cube/open office choice is a tradeoff, balancing communication with deep concentration. What the proper answer is may depend on your team, your project, and maybe even the phase of the moon. However, it does not depend on which book your manager read most recently.
Office space is expensive, labor is cheap. That's the conclusion. It's the economy, stupid.
I seriously doubt that. I see so much wasted programmer time in my daily work. It must be a cheap commodity.
I've complained so much about the open office layout that I've been told point blank to shut up about it.
My suggestion: You can keep the open space but you need to enforce either a quiet area for focus or carve out a specified area for loud and open collaboration.
