How much process is enough?
Ask that question of ten corporate managers and you’ll get ten different answers. That’s one of the reasons why converting from waterfall software development to any agile approach is so difficult.
Waterfall development is a command-and-control process filled with checks, balances, phases and gates. It feels right to many managers despite it’s lack of speed and responsiveness.
Agile approaches like Scrum, XP and Kanban offer speed and responsiveness but sacrifice some command-and-control elements. You can’t have it all, right?
When managers think of adopting an agile approach, they tend to err on the side of too much process based on the theory that having too much control is better than not having enough. Getting into an argument about the issue is unlikely to convince anyone of the merits of agile development.
So what’s the answer?
Give a little. You’ll likely be forced into implementing your agile approach with more command-and-control elements than you want or need. That’s okay. Have a plan for removing some of those command-and-control pieces as the team improves. Here are some things to consider.
- Do not merely wrap your waterfall process in an agile wrapper. That is, don’t practice waterfall adding daily meetings, stories and burndown charts to it. You’ll just add more overhead and make matters worse.
- Instead, do the opposite — wrap your agile approach with some waterfall elements. Managers want to see metrics that measure progress. You may have to comply.
- Pick an end date for the project. End dates are good. They establish a time box. An agile project cannot operate on the principle that the software will take whatever time it takes. There needs to be an end.
- Managers want to know how big the project is. Stories are great for that. You don’t need to fully refine every story to it’s sprint-ready state. High-level stories are okay as long as you can estimate their duration — roughly.
- Managers want to see progress. Burndown charts are great for that. They clearly show how many stories are complete and how many are left. Of course, this needs to be based on the initial estimates in item 4 so it’s not a true Scrum burndown chart but it’s good enough for reporting purposes.
- Managers like to see defect reports. You’ll need to maintain a defect backlog and report on it.
- Some managers really like earned value charts (I think they are totally fictitious but more on that another time). A simple metric showing stories completed and cost incurred should do it.
What’s the downside?
Once you start tracking size and cost, any change that impacts size and cost has to be tracked. Change control? Unfortunately, yes, though this is a project management issue not a development one. The key is to only track and report on changes that impact size and cost. The majority of the day-to-day team decisions will not impact either.
If you can get this far, you’re in great shape. Over time, you can work toward eliminating the waterfall wrapper — one element at a time. What do you think?
I liked this post a lot. I felt like you were describing what a vendor tried to do for my (current) client a few years ago. Imagine a project lifecycle being waterfall with exception of the actual product development. (Agile with a Waterfall wrapper) Once it was time for the actual work to get done, the vendor tried to use a more agile approach. Though their implementation failed, primarily due to not having Agile coaches on staff and never letting go of some really messed up earned value practices on both sides (tons of overhead), my client and the vendor both saw a lot of value. What is sad is my customer got jack for actual code delivery but they were elated because it was the most transparency they had every experienced on the program. So, instead of knowing the project was in trouble 11 months in, they knew after the first month. Strange to see what some customers see value in.
Great article.
At my last workplace there was a waterfall style project management process in place. Project management was organized but below that layer the developers worked in a largely unstructured way which over time became more or less unmanageable.
My first approach to change this was to try and convince the higher-ups to let us introduce Scrum. But of course they did not want to let go of their waterfall process. My second approach ressembled what you describe above and it worked wonders.
So while we could not change the project management layer, we could change the approach our small team of developers took. So below the waterfall project management we introduced a lot of Scrum elements (dailies, plannings, retrospectives, stories, burndown charts…) . It helped the developers but did not disrupt the larger process outside of the team with it’s fixed end dates etc.
While it’s good that you’re calling this out as a short-term strategy, I’m not so sure it will really work out that way in the long-run, primarily because it doesn’t give the business any imperative to change – and in fact, by retaining the trappings of waterfall, you’re likely indulging some opportunities for failure that will be inaccurately and unfairly painted by execs as “why agile just doesn’t work here (or anywhere)”.
If the business is resistant, they need more convincing instead of indulging bad practices that end up misaligning the dev teams and the organization.
In my opinion (and experience), it is far better to coach both the business and the team on how to work with agile and get them started with a simple project to see where they bump up against the organization and remedy these issues in the near to long-term to drive momentum for adoption. Otherwise, all it will take is one “failure” and the business will feel quite justified in saying “See? We tried ‘agile’ and it failed.” – When in reality, they did nothing of the sort.
Sound advice here. Launching “Agile” (or any initiative) in a “big bang” can lead to disaster. A more gradual approach, taking several months or even years can be more effective long term.
Chris does, however, make a sensible observation. There does need to be commitment to replace Waterfall – and positive steps to do this – or the risk is it will remain in place. That has longer term consequences, including the potential to create the view Agile has “failed” and provide grounds for political battles later.
It is an entirely pragmatic approach to the problem and worth entering in to with your eyes wide open.