The thing that really kills me about this is that 90% of the people who are "doing Agile" (which is not a thing that you can do [1]) really believe that they are an Agile shop. And then I will ask as many as 3 questions to find out what they're doing is waterfall, or mini-waterfall, or code-n-fix. In the old days I could say, "Hey, you should try one of the Agile processes." Now they think they are doing that and doing it well (after all, they have certificates!), so what can I even tell them?
Everybody I talk to who was involved in the Agile movement in the 2000-2004 time frame has this experience now. Common reactions are rage, tears, and philosophical resignation. (I favor quiet rage, but I suspect resignation would be healthier.) That there are some shops doing very well is consolation, but the early Agile people were hoping for more.
For the whippersnappers who are curious about now-ancient history, my theory on how we went wrong is here: http://agilefocus.com/2011/02/21/agiles-second-chasm-and-how... A big lesson I learned was that if you don't have a trademark and use it to keep quality high, your popular term will get buzzworded to death.
[1] Agile is an umbrella for actual processes, like Extreme Programming. There is no process called Agile to do. Saying you are "doing Agile" is like somebody saying their house is in America, but not in DC or any of the states, just America. If you point out that it's the United States of America and they have to be somewhere; they just shrug.
I live on Midway Atoll, you insensitive clod! ;-)
(http://en.wikipedia.org/wiki/Territories_of_the_United_State...)
Agile is not something you can follow to the letter, like you said it's a bunch of processes that help you succeed. Which ones to apply, and more importantly, how to apply each process is highly dependent on the circumstances.
The way we got the term Agile is that a bunch of people in the '90s were experimenting with processes that were less insane that the waterfall approach that had come to predominate. DSDM, FDD, Extreme Programming, Scrum, and probably others that I'm forgetting now. All of those people got together and said, "Hey, we all have something in common. What is it?"
The answer to that was the Agile Manifesto: http://agilemanifesto.org/
Agile is to an actual process what a subdivision is to a house. If you say you live in the subdivision but are not in an actual house, then you're just wandering around camping in back yards and parks.
I agree that a given process is a collection of practices, and that one can compose different practices into a process, sort of the same way a talented chef can walk into a farmer's market, see what's on offer, and compose a balanced, nutritious, and tasty menu on the fly.
However, in practice, what "doing Agile" mainly means is what happens when an energetic but unskilled 7-year-old decides to be helpful and make dinner. Maybe you get a dinner composed entirely of candy. Maybe they use the stove and make something that is a mud-pie version of a gourmet meal.
That isn't to say that people can't create a good process out of raw practices. It's just that doing it is an expert-level activity, and it requires an immense amount of dedication, experimentation, and careful thinking. That's not what I see when I ask after shops that are "doing Agile". That phrase seems to be used exclusively by people who don't want to think hard about process, and just want to cargo-cult and half-ass their way through things because they're focused on getting a project out the door and/or pleasing the HiPPOs.
Agile is not a bunch of processes. Agile is a philosphical orientation that guides how you do work, including selection and evaluation of processes in light of current team and mission.
In situations where agile doesn't make sense, do you think 'fake agile' can be a better way of working than 1990s waterfall?
There are actually reasonable techniques for doing Agile internally if you have clients who believe they need a traditional requirements-based contract approach. So those people should just do Extreme Programming.
For people with huge legacy projects, I think they really need to do the cost-benefit analysis. People like Michael Feathers make a persuasive case that you can slowly clean up legacy code, and that's much more economical than just suffering through expensive and risky changes.
But if somebody is deciding not to do Agile for considered cost-benefit reasons, they should just say, "Yeah, Agile doesn't make sense for us. We're going to do waterfall." Or they could choose a mini-waterfall process, or any one of the pre-Agile processes that suits them. (If those people exist, they should read McConnell's Rapid Development, which catalogs the options pretty well.)
If that's what they want, godspeed.
The people who make me crazy are the ones who don't actually have the discipline to use a real Agile process and also lack the guts to say, "Yeah, fuck that Agile stuff." I get that people would like to be all hip and Agile, but sending somebody out for a 2-day "certification" and then changing a few labels is not actually making better anything internally, and it's making my life worse.
I don't want people to be fake anything. Be real, everybody! Especially managers. Because it's only admitting where you're at that's going to let you get somewhere better.
As an aside, I don't know any serious Agile person who thinks that doing 100% anything by the book is a good idea. I think it was Ron Jeffries, one of XP's inventors, who said something like : "By the book XP is a good place to start, but it probably isn't where you'll end up." Agile processes all have an inspect-and-adapt component. E.g., the weekly retrospectives and continuous tinkering of XP.
His interpretation of this seems to be "Wing it, and if questioned just explain that we're reacting to change which is the agile way.".
Not only can you fail on both sides, but you can be too far on one side and think that you haven't gone far enough.
I've had startups tell me they were doing Scrum, but it had gotten so hard to estimate their work that they had "broken agile" by having "grooming sessions" to talk to stakeholders about upcoming work before it got into sprint. They were shocked to learn this was a required part of Scrum and that many books recommended that being up to 10% of the team's time.
Another company I talked with was so concerned about their developers "continuously delivering" code that they were fighting to be "more Scrum" by changing their QA folks to be in a staggered sprint one week after the developers. They were shocked to learn that this was a widely-tried and widely-panned approach that was philosophically opposed to the roots of Scrum.
Now, I remember hearing examples like these before I had really dived into Scrum, and assuming it was a "no true Scotsman" where, whatever you did, someone would find a way to say it wasn't Scrum. But after some very good books and talking with experienced people (who were involved in the early stages, not bandwagon consultants), I now feel like I actually understand what Scrum is for and when it works, and can quickly spot people who don't understand the philosophy. Has anyone else experienced this? I wonder whether it is real experience or just an illusion my brain has created to give meaning to many months of painful trial and error.
Nowadays I think of scrum as being about holistic product creation, about building a team that can deliver what customers will pay you for over the long-term. That requires breaking down most "process" to get "one-piece flow" on a complete team working as one to deliver new features. But it also includes the team building its own processes to make sure they are building the right thing to a high level of quality.
That vision is anathema to cultures that value technical prowess, or pure speed, or meeting corporate objectives, or anything else other than building a team that can deliver value to customers over the long term. It's a challenge and a priority change for both corporate political environments and built-to-flip startups. And it's surprisingly hard to find a company that wants to pay you to build things their customers like, rather than using some other metric.
Scrum might be a starting set of processes adopted by an agile organization, but an organization that is concerned with "doing Scrum right" is not Agile.
I agree that anyone who is dogmatic about "doing Scrum" for its own sake is missing the point, as is anyone who is "doing Scrum" in order to just make their team more organized or productive. That said, within the constraints of the current business world, Scrum is the best way that I have seen to resolve the necessary tensions of creating software products and let people and interactions take center stage.
But I'd love to hear your experience and advice.
I get the "no true Scotsman" sense myself. What book(s) was most helpful in pinning down what scrum is and isn't?
Thinking about it, the book that actually taught me to understand what the early agile people thought was Christopher Alexander's book "The timeless way of building." The book is about architecture rather than software, but the underlying idea is right on. It starts by addressing what is the point of building? Some people try to optimize speed, or predictability, or how good the building looks. But Alexander says the point of a building is to be lived/worked in, and it should be measured by how it improves the human lives of people around it. This implies doing the design and building in a whole different way, and thinking about the relationships between builders and users in a different way.
The "agile" idea is similar. People who try to "use agile" to accomplish their goal of just making software really fast, or making lots of money, are missing the point. Agile is actually about building software that is good, in a way that is sustainable for users, builders, and a company, and makes everyone involved better. If you use Scrum in the context of those values, you may succeed.
But the vast majority of people who adopt scrum, and perhaps the majority of consultants and explanations trying to "sell" Scrum, just use it to wring every ounce of effort out of your employees, or to ship something crummy fast. That ends up not really working, and in my experience making environments even worse than they were before.
By definition, I think that would have to be the Scrum Guide [1].
"Scrum is defined completely in the Scrum Guide by Ken Schwaber and Jeff Sutherland, the originators of Scrum."
In my view, stories are tokens for conversation. Now when I deal with teams who are writing down more than a few words, I look for barriers to conversation. E.g., not all the right people sitting together, or too much work in process.
However for a lot of small companies it is better than what they had before, regarding consistent direction. Lots of startups change scope or features daily and are in a constant state of panic.
Having a fixed two/one week sprint provides same restspite for developers. But still allowing feature and priorities to be changed after each sprint.
That sounded insane to me at first, but the Lean folks really turned me around on this. Mary Poppendieck's books are great in this regard. One of her best examples is the Empire State Building: they started construction on the lower floors before the top floors were even designed. Nonetheless, the project was completed on schedule and budget. How? As with Agile projects, scope was treated as variable, and a lot of thought was put into maximizing efficiency of the system.
I've also read some great stuff from the Lean Manufacturing community on how shifting the approach to accounting is vital. Rather than having annual plans and budgets and then trying to hit them, you have rolling projections n months out and adjust allocation to achieve real-world effects.
In my personal experience as an enterprise apps director, this is significantly better than pure waterfall but still causes all sorts of stress on the IT side. I think the single highest impact change for the better would be to take a product-centric rather than a project-centric perspective. At least in my company, business "owners" don't pay for KTLO work, only projects, so there's no incentive for anyone to ensure high quality products.
It misses important aspects like constantly verifying the product with customer, and adjusting if required.
If they are small, then as long as they are releasing frequently and paying attention to what's happening in the real world, they tend to converge on an Agile-ish approach. And if they aren't, they are likely to quickly crash and burn, limiting their damage.
Enterprises worry me more, because there feedback loops are often broken. Nothing is pushing people toward sanity, and organizational dynamics often push the other way. Plus, they have plenty of money to hire consultants to prop them up. Which in turn creates a marketplace of very well-paid and well-marketed consultants who only sell half-assed Agile, because their clients wouldn't even want to think that they aren't getting the good stuff. And if there are enough of those companies and consultants, then the impression is that half-assed Agile is actually the norm.
But then, try to suggest that there's a requirement that has changed or come to light and suddenly the "agile is open to new requirements" part of the approach is forgotten as the "progress" that's been made so far is too precious to change.
/rant
definition of Done usually includes things like written code, unit tests, code review, documented etc
I suspect most people doing a agile, don't actually do agile. And think its a term for little structure.
Agile methods, such as XP programming are highly disciplined methods, more so than traditional.
The main difference is that they react to customer change, and don't follow plan than stretches 6 months into the future. The priorities of tasks can change from sprint to sprint. Allowing the team to react to the outside world.
Now I am seeing this from an outside perspective; I'm not a programmer or software engineer but I work directly with them in a weird sort of non-supervisory product manager role. In my opinion, this is no different...in fact, "Agile", which as far as I can tell is nothing more than a hand-wavy underspecified form of Extreme Programming, has so much ambiguity in its definition that you can't ask any two advocates and get the same response as to what it is. And as such, whenever there is a problem with the outcomes of the process, the only apt response is the No-True-Scotsman fallacy...which makes development descend into the phenomenal failure of productivity that all holy wars are.
This isn't unique to Agile. Lean and Six Sigma are also mixed bags of philosophy and process, and they have been mixed bags of occasional success and phenomenal failure.
There are different concrete ways to implement Agile, such as using Scrum, DSDM, or various other methods.
I see parallels in people bickering about whether some element of a service is or is-not RESTful, when maybe 95% of the functionality of the service is acting/quacking like a duck.
I think a similar effect has happened to RESTful architectures; people making them have (inadvertently, I believe) produced very convincing emulations while skipping the parts that effectively distinguish them from RPC systems.
[1] http://www.econtalk.org/archives/2013/12/lant_pritchett.html
http://en.wikipedia.org/wiki/Blind_men_and_an_elephant
When merits of processes are discussed devoid of its purpose (delivering quality software in business relevant time), the argument is similar to the story. The process is just the means of reaching the purpose isn't it? It is not like successful software was delivered only in the last decade, we have been to the Moon and back before that.
Unfortunately, processes are always made in good intention to achieve a purpose, this when transmitted from one person to another becomes a system, as there is no guarantee or way of enforcing 100% knowledge for all people in the chain. This deviates it pretty quickly from the purpose with focus only on the practice and soon the system becomes a religion.
Then surprise! people shed blood over it after that demanding that their book is the only truth :-)
I once had to sit through an 8 hour pre-kickoff meeting detailing in excruciating detail how our team was going to be "agile". It involved at least 400 slides, described a weeks long change request process, and two-month long "sprints". Even as a junior dev I know I am lucky I never had to suffer through the rest of that contract.
We end up creating grandiose plans, but they usually just turn into endless bullet points for strategic road maps rather then actionable items
