Not Getting the Results You Want? Perhaps You’re Measuring the Wrong Things.

Agile process metrics are a frequent topic of blog postings with the velocity metric most often being discussed. But do agile process metrics really matter? What does the business really care about?

Software development metrics are important to development teams but they mean little, if anything, to the business stakeholders and end users. Business professionals care about results and metrics around sales, profits and productivity. Sure, getting software projects done better and faster impacts business performance, but the relationships between them are nebulous at best.

Look at metrics from a different perspective.

The software we develop should help end users do something better. Customers should have a better buying experience or improved self-help options. Employees should experience better workflows, improved knowledge management, or faster communications. Those are the results that matter.

It often amazes me how managers will believe that a process, a workgroup or a tool is inefficient and wasteful. Yet, when I ask probing questions to determine how the conclusion was drawn, I find that it’s conjecture. There is no hard data to backup the statements. How can any business measure outcomes and improvements without meaningful data?

This places software projects in jeopardy. If the current business situation is misunderstood, the improvement offered by the new software might also be misunderstood. Even worse, the software development team could focus on the solving the wrong problem.

We need better metrics around the problems we’re trying to solve not the process we use to solve them. To those ends, we need to capture before and after metrics around the end user experience. What follows are some questions to ask business people. Hopefully, the answers will provide sufficient data to help determine the improvement provided by the new software and the performance of the software development team.

Food for Thought

  • What is working well in the current environment? How do you know?
  • What’s wrong with the current approach? How do you know?
  • What information is captured today? Where is it kept? How is it used?
  • How is performance tracked and reported? Are there any performance goals today?
  • What are your goals for the new software?
  • What would a successful outcome look like? How would you measure it?
  • What information would you like to have the new software capture?
  • How would you like to see the new metrics reported?
  • How often do want the metrics updated?
  • Do you want to improve customer or worker satisfaction? How would you measure it?

This kind of dialog will help both the business and technical professionals determine the success (or not) of their efforts. Internally-focused software development metrics like velocity are useful but they only help on the technical side. Externally-focused metrics around performance and productivity help everyone.

Updated: April 22, 2012 — 9:34 pm