Agile, done right, is more about how work is broken down into consumables and deliverables. It's about clear prioritization and freedom to focus on the task at hand. But most development teams would iterate aimlessly, while the organization around them continued doing business as usual.
To successfully embrace Agile, the entire organization has to do so at some level or another. You have to break requirements down into pieces that can be consumed in an iteration. You have to prioritize clearly. And you can't change priorities until the iteration is complete.
Additionally, many software teams misunderstand how they work. I can't tell you how many times a team would insist to me they were using a waterfall process, but when I asked them to describe the previous workday they'd describe end-to-end firefighting. When I would introduce Agile and the controls that it brings, they would react negatively expecting that Agile was some magical organized free-for-all.
Additionally, lots of consultants made money off of the promise of Agile and the cult of Agile. They didn't provide practical solutions, and organizations resisted change. So much of the indigestion that developers have regarding Agile is due to their own experiences with organizations that bought the promise of Agile but didn't do the necessary work to make it successful.
Heh, classic response. Whenever your project fails, your coach tells you it's your fault because you didn't do real Agile. Same with OOP, whenever your code inevitably becomes a tangled mess of classes, you will have all sorts of gurus making metric shit tons of money from books and consulting telling you you didn't do real OOP.
Guys, please, can you just admit there is no recipe for success (rhetorical question, of course you can't, that's where you money comes from, maybe you even believe your own bullshit)? There is no real Agile and OOP and other crap that I forget, they're just buzzwords that, when applied long or hard enough, produce shit. If you had kept you project to an absolute minimal level of Agile and OOP, it would probably still be alive and kicking. But you didn't, because you jumped on the bandwagon, you used religiously whatever was popular.
> Agile, done right ...
Yeah, heard that a thousand times before. Sadly, younger developers fall for this all the time, so shit grows exponentially. I'm aware my opinion has always been in minority but whatever.
I grant that there's no one right way. There are many, and they depend on circumstances. I certainly grant that there are an ocean of idiots selling bullshit. But you're throwing the baby out with the bathwater.
There are better and worse ways to make software. Teams can reflect on how they work and get better over time. Those teams can recognize patterns, and other teams can benefit from learning about those patterns and trying them out in their environment. And that's how things like Extreme Programming came to be: teams experimenting and sharing what they'd learned with other people who wanted to get better. That was exactly what the early Agile movement was.
Piss all over that if you like. But you'll be missing out.
There is absolutely nothing wrong with the Agile Manifesto, but this can not be said of the industry that was created on the back of it.
The original agile manifesto was meant to avoid following plans blindly like that. It's only four lines one of which is "Responding to change over following a plan".
Despite what the kanban people think, Agile isn't supposed to be "small waterfall" where the work and priorities are set in advance instead of discovered as you test hypotheses through trying different technological approaches and doing back and forth with the users.
What you propose means that if in a cycle you discover that the plan was sub optimal and not well prioritized (and when you are iterating, you are in the best position to find that out) you march on towards the dead end and only adjust after the "sprint" is over? That is not agile, it may fall under scrum but really it is just small waterfall.
Good agile as it was originally stated is compatible with and even comes from a similar line of reasoning as the Lean approaches. The core of Lean is that the people closest to the technology should be put in a close relationship with the users to be in the best position possible to resolve what I like to call the crux of innovation. Innovation's most difficult and most crucial part which is to figure out what work has the most value to effort ratio, what has the most risk and needs to be studied early to clear up that risk and ultimately what the team should be working on first. At least for any work that is somewhat leading edge or a bit experimental, priorities cannot be properly assessed without experimentation, user feedback and management of uncertainty and risk trough early experimentation.
Scrum tends to ignore this core problem by positing a product owner that has the answers. This is very reminiscent of waterfall where people disconnected from the technology and who aren't experimenting and putting things in front of users, who are ultimately in a bad position to make plans are making those plans anyways. IMO what you are describing and a lot of the ideas in scrum are the very things that original agile was meant to prevent and also goes against Lean approaches.
Here is the agile manifesto:
"Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan"
Also, instead of having broader estimates in terms of months, you force developers to rush constantly with sprint completion pressure. In case you didn't know, rushing developers is very, very counterproductive. Cost, Time, Scope. Pick 2.
But I do agree with you about "sprint completion pressure". With Kanban you don't have that and you don't have large day long "sprint planning sessions". You have just enough work in the "Dev ready" column to keep the pipeline going.
Of course not having the sprint planning pressure and not having a deadline to release doesn't work without a continuous delivery pipeline and some way to turn features on when ready.
What you are describing is the opposite of iterative. When you iterate, you go over the same territory multiple times, with more insight/detail.
And yet the very definition of the word, "Agility" is supposed to mean moving quickly and nimbly, able to easily change direction.
The side effect of shorter iterations is that requirements have to be broken down to fit inside the smaller iteration, so there is a price to be paid by those managing the backlog and generating requirements if change needs to happen quickly.
Agile methodologies don't enable change as needed, they create a structured way of handling inevitable change. And when the backlog is adjusted to accommodate that change, the consequences of that change to budget and delivery date are easy for everyone to see.
If you don't have controls in place, then there is no understanding of the cost of change. And without understanding the cost, good cost-value decisions can't be made.
It broke every single rule of what I look for when accepting a job. I only took the job with the condition that I would have 100% support, budget and time to set things up right from my future manager.
I laid out my vision....
1. Realistic timelines based on a 40 hour work week. All estimates and deadlines were based on realistic hours, realistic staffing levels, and realistic utilization.
2. I would be given the time and budget to setup a proper CI/CD platform.
3. We would put in all of our estimates time for automated testing and code reviews.
4. Devs would have direct access to the (internal) customer.
5. I would have the budget to contract smart people at market rates.
6. Any dissension from the legacy developers with an attitude of "why do we need to change. That's the way we've always done things" wouldn't be tolerated. If they couldn't get on board, I would hire my own team and let the existing developers support the legacy product.
- automated tests keep the cost of hiring QA down. QA only has to test new functionality. The more automated tests you have -- both unit and integration tests - the less manual regression testing you have to do.
- automated testing also allows us to ramp up with contractors and you don't loose business knowledge when people leave because the business logic is in the automated tests - not a bunch of documents in SharePoint. If a contractor comes in and makes a change and the tests break, they've broken a piece of business logic.
- Once everything is setup, and documented, you don't need dedicated "release engineers" all releases are coded with Powershell and configured with Consul. We treat servers like "cattle not pets".
We also based the project on Mongo and not sql server so we don't have to worry about database deployments. We decorate the fields in our POCOs that are being stored with attributes that tell the individual services at startup to auto create any missing indexes that we need. Schemas are enforced by microservices that are the only way to access Mongo.
That right there is one of my biggest pet peeves in any dev shop. It's astounding how few people see the connection between "the way we've always done things" and "the problems we've always had".
This micro-management of the entire software development process is completely unnecessary and contrary to every bit of real-world evidence that we have about what has made certain software projects successful, and others not. Ambition, drive, and a penchant for excellence is what makes great software, not a bunch of managers pushing along a group of mediocre developers like cattle.
I've been there. I've also been in the opposite, where there really is no technical leadership. I can't say it's better.
For what it's worth, certain agile techniques do provide a structured way to make clear group decisions. Unfortunately those decisions usually revolve around development methodology and not, you know, the design of the actual product.
Point being, micro-management is often a broad reaction to a vacuum of technical leadership. But instead of innovating, with the team makeup in mind, on how to make sure good decisions emerge, a manager will figure they need to personally make those decisions. So they need more structure to make sure they're better informed, and agile techniques tend to get hijacked to that end.
It's as if they had never read books like "Rapid Development" which describes over a dozen different design approaches, with pros and cons.
I dislike Agile because apparently the only important thing is the Toyota Way. They never mention alternatives like the Volvo's Uddevalla plant and humanistic manufacturing.
Eg, quoting from http://gerpisa.org/ancien-gerpisa/actes/9/9-5.pdf "After a slow start Uddevalla took off and in 1991 reached the level of the Gothenburg mass production plant. From the last quarter of 1990 to the last quarter of 1992 Uddevalla cut the assembly time at an average rate of one hour a month. This rapid learning curve surpassed developments at the Gothenburg plant, where lean production techniques were introduced to boost performance."
Yes, such conclusions are contested, but it's still a concrete alternative that's almost ignored by those repeating the chant "Toyota ... Toyota ... Toyota."
I dislike Agile because it's a collection of ill-defined mush. If a group adapts or changes in any way during a project, and the project is successful, then it's counted as an Agile success story, while if it project fails despite flexibility then it must not have been True Agile, or at least Enough Agile.
I strongly feel Agile is a nice fit only when base work is done properly in terms of getting the software architecture and contracts correctly defined.
s/People/Managers/
In many of the cases I have experienced, agile breaks down into an excuse to not plan ("Requirements just slow things down!"), and the elements of it that would drive success are ignored.
So often business leaders just want shit done, and when the technical types warn about reliability or technical debt, there is a lot handwaving. Thus short attention spans see agile as a way to justify shortcuts. When this happens the result can be worse than a poorly executed waterfall model.
Agile works when all the proper elements are used, just like any other endeavor, and it fails for the same contrary reasons. Developers become frustrated when yet another path to accomplishment is blocked.
Disclaimer - Am technical manager. Have watched business peers shortcut to failure. Have fought the battles and lost. Frustration apparent.
I have seen this taken to the extreme-- the management at one company I worked for would not specify ANYTHING in writing. Nothing. Not even bullet points on what a feature would do. They would then micro manage intensely (I had a non-technical VP doing code reviews on a Senior Architect).
Estimates, however, only included development time, never any other tasks, and were always challenged. Deadlines were always hard. In short, we had all the responsibility of Agile but none of the empowerment. Engineering was to blame for everything.
If you think this was some small crazy startup... This companies revenues are about 500m a year. Almost every American has our products in their homes.
This is the crux of the problem with agile. The reason it's embraced so readily by business types is the reason that it fails. With the 'justifying shortcuts' description you hit the nail on the head.
Agile encourages staying flexible and able to change plans on a whim. In practice that tends to mean that people using agile processes don't plan long term. Piss-poor performance is the result. Breaking tasks down into manageable pieces is hard, and requires good long-term planning for any significant project.
Agile in practice is "weeks of work can save you hours of planning".
The other side of rapid iteration and emergent design is building a system with change in mind, not over architecting and living by the principle of YAGNI - You Ain't Gonna Need It.
It's the job of the architect who should have both the people skills and the technical skills to give the customer what they want and not what they ask for.
The nastiest balls of glued together snot I have ever seen were almost all from textbook Scrum shops. The rest were from lowest bidder outsourcers that probably use similar management methods to try to get code from bad or underpaid/unmotivated coders.
The problem is that deep conceptual thought about a design and deep refactoring are hard to capture as a ticket item. Quality is something that perfuses all deliverables but is not itself a deliverable.
Agile is also totally unsuited for innovation. You will never get a Xerox Parc or SRI with agile. You can only make what you already know to make with agile.
People who care passionately about technology and software build great software.
On recent Agile projects I have been involved in, a lot of Agile 'artifacts' were produced (mostly walls covered in Post-it notes). Software seemed to by a by-product and didn't really feature much.
I think this is a result of the 'busywork' Agile created.
If developers dislike agile, you'd expect them to be happy with BDUF (waterfall) and I never hear people saying they're happy with book-sized specification documents.
Isn't this really the difference between computer science and software engineering? As developers we have to straddle the fence.
Edit: In case it wasn't clear, I agree with everything you're saying. However, I do think that good managers that understand their developers can make things work and keep everyone happy.
I eat hamburgers, not bites.
That's certainly not the only way, however, to ensure a common vision of the future.
If your plan, on the other hand, is to have three people develop three separate parts for a year, and then think you are going to stick them together and release in two weeks, people can avoid interpersonal conflicts entirely for a year, after which there is a crisis, which is often survivable by the participants because there are already so many chips in the pot...
make plans. try to gauge costs. change plans when necessary. integrate as soon as is reasonable. use consensus to drive structure/interfaces. maintain a testable artifact.
maybe running a service is different, but variants on the above seem to be the only thing that works for software.
Developer driven:
Mike: "Bob, I'm skipping scrum today."
Bob: "K" (Thinks "He's still working on the gnarly multithreading problem.")
Management driven:
Manager/Scrum Master: "Bob, where's Mike?"
Bob: "He's deep in flow working on that multithreading problem."
Manager: "Go get him, he can work on that later." (Thinks "I have boxes to fill in my spreadsheet. How can I do that without a scrum meeting??")
It was heartbreaking to see something that helped me and my teams a great deal get so throughly misused. It became just another stick to beat developers with. I spent a lot of time apologizing for all the pain people endured.
It's hardly the first time, though. The original paper on Waterfall basically said, "This is a thing that is bad and does not work." But people went and did waterfall anyhow, because a) it conforms to a number of human biases, b) it reinforces the typical top-down corporate power structure, and c) provides endless opportunities for people to blame other people for problems. So in retrospect I'm not shocked at all that we've mostly turned the various Agile methods into meaningless pablum.
To move quickly, you really need just two things: a way to make sure things don’t change unexpectedly, and a way to recreate anything you had before (where the 2nd is a crutch if the 1st fails). To have sanity, you need somebody setting a direction (“we’re doing this”), and somebody deciding what can wait.
For software, this means things like: “for Pete’s sake, use revision control”, “track versions of dependencies”, “back things up”, “write scripts instead of step-by-step instructions”, “test somehow”, and “track problems”. There is no “daily one hour meeting” phase of software development.
However, I mostly witness selfish, political, or misinformed ideas being dictated as "requirements". I learned really fast that stakeholders do not like their ideas being questioned with data points from a scrum research spike.
So who enjoys wasting time when your professional viewpoint won't be heard?
If management or stakeholders are unwilling to listen to the team that solves problems for a living, why should I/we waste energy on Agile or Scrum?
The anti-patterns of Agile are...
- no documentation
- incoherent design
- no tooling
This doesn't have to be Agile, but often is.
Problems arise when it is heavily abused and applied incorrectly by management, in my experience. My first time doing Agile (well, SCRUMM, but bear with me here) was someone using it to manage multiple unrelated projects at once. Predictably, it crashed a burned miserably. The person who decided on it was suckered by the buzzwords but failed to actually understand the purpose of the process.
My current company uses SCRUMM quite effectively, and it's been a wildly different experience. It probably helps immensely that the people in charge are also developers.
I think we need something in the middle. Does anyone have any ideas?
I suspect the fix will be on the type of software app that people are writing rather than the way apps are written. Monolithic apps are hard to write and maintain so we need to think small and generalize software something similar to lego bricks.
http://www.halfarsedagilemanifesto.org
SCNR, it's too beautiful...
Maybe there's some description somewhere with meaning, but it doesn't matter because the meaninglessness has prevailed.
The Greenhopper plugin was a joy to use, 10 years ago.
Jira Agile is an atrocity, especially in an enterprise environment. About 5% of the time you'll actually be using it to help you do your job. You'll spend far more time answering questions and doing risk analysis, etc, etc all at the request of some random person you've never met who needs reporting on some random bit of information that is of no consequence.
It's used as a reporting and tracking tool now, not a tool to handle development flow.
Effective Scrum just means sacrificing code quality sufficiently to make your ticket fit your sprint. /s
