The headline of this post pains me but it’s true. Real agile approaches to software development will never replace the old and outmoded command-and-control approaches based on waterfall. Scrum, Kanban, XP, Lean, etc. will augment and improve waterfall approaches but never replace them. Here are two reasons for my pessimism.
The Safe Choice
Have you ever heard the old saying, “No one ever got fired for buying IBM”? It comes from a time when IBM was the king of the enterprise computing industry (some will argue that they still are). If an IT manager purchased equipment from IBM, he was making a safe decision. Even if the equipment didn’t solve the target problem, it would be assumed by many that no vendor could have provided a solution. After all, the manager bought from the best, right?
Today, companies don’t tend to dominate markets the way IBM did largely due to anti-trust laws. However, the same logic is true when it comes to any market leading vendor such as Apple, Cisco, Google, Microsoft, Symantec, etc. When you buy from the (perceived) best, you get a layer of protection from criticism.
Waterfall approaches to software development are the market leaders. They are widely used and accepted. Managers won’t be criticized for following them. Agile approaches carry more unknowns and more risks. Why would any conservative manager even consider an agile approach?
Protected By Procedure
My second rationale is more complex. It has to do with our penchant for following the rules. Most companies place plenty of rules around their software development efforts. They implement phases, gates, approvals, documents, hierarchies, reviews, standards, etc. to control every aspect of software development. Those controls aren’t inherently bad but become evil when carried to extremes.
The problem is that many people like extremes. Many managers like to tightly control how things get done. Many subordinates are eager to follow the rules. After all, you can’t be fired for following the rules, right?
Lots of rules and procedures provide a safety net. Everyone knows what they are supposed to do and how to do it. If anyone encounters a situation that’s not covered by the rules, they seek assistance so that the management team can make a decision. It’s safe and conservative. Agile approaches have few rules or procedures and relatively flat hierarchies exposing everyone to new risks. Why would any software developer even consider an agile approach?
Culture Trumps Process
These are high hurdles stemming from corporate cultures that demand conformance. If you follow the rules and make conservative decisions, you’re granted a layer of protection. If you bend the rules or make independent decisions, you could be reprimanded — even if you made the right decision.
In some organizations, the pain of the status quo will overpower the need to conform. When the software is consistently late, over-budget and crappy, management will sometimes be willing to adopt a completely new approach. That gives agile development a foot in the door.
Unfortunately, I’ve found that many organizations are willing to put up with a lot of pain to protect the status quo — self-preservation is more important than delivering good software. When the software development process battles with the corporate culture, the culture wins — every time.
Can agile software development ever replace waterfall development as the primary approach for software projects? I’m skeptical. Perhaps we’ve gone as far as we can go and it’s time for the agile community to move in a new direction. Any ideas? Your opinion is as good as mine so please leave a comment.