Don’t Build Bad Business Software With a Good Process

5 Steps for Avoiding the Perils of Building the Wrong Business Software

Building an enterprise-scale software package is an expensive proposition under any circumstances. (The definition of enterprise scale for this post is any software package that requires a shared server and supports multiple users.) The real cost includes upfront payments, server costs, user training, support staff training, maintenance fees, and possible network infrastructure upgrades.

Building the wrong software is even more costly. Agile development approaches will help but you need to follow the steps outlined below to make better choices.

1. Conduct an objectives and issues assessment

You can’t effectively make software decisions when your organization is unclear about its objectives, priorities and constraints. Arrange a planning session to talk through these elements. Take a hard look at the business processes that will leverage the software. Zero in on the activities that you want to automate. Prioritize the capabilities you need using a simple three-level system.

• Must have – Essential requirement. Key stakeholder needs will not be satisfied if this requirement is not delivered.
• Should have – Important requirement. If it is not delivered, there is an acceptable workaround.
• Could have – Nice to have requirement. Can be sacrificed if other requirements take more time than estimated.

2. Consider more than just features and functions

Features and functions are quantitative. Any enterprise system will have many. The qualitative aspects are equally important. Any software system will do some things well and other things poorly. Focus on the capabilities you need. Ignore all the bells and whistles you’ll never use. Focus on quality.

3. Understand the technology foundation

The software should be founded on widely used technologies. This will make it easier for you to find people well versed in the software tools. It will also be simpler to integrate the new software with other systems in your firm.

Beware of proprietary technologies. Proprietary is okay when you need something truly unique and innovative though this is rare. Also be on the lookout for outdated technologies. If a technology has been available for a long time and hasn’t undergone a major overhaul, be suspicious. It may be on its way out. You don’t need to be on the leading edge, you need technology with staying power.

4. Map software features to real business processes

Companies need software that is connected to real business practices. Are you willing to change the way you conduct business in order to get maximum value out of the software investment? Or, do you expect the software to support the way you do things today?

Many companies build good software that is a poor match to how they run their businesses hoping that the effort will drive change. It doesn’t work! Change must be driven by the business with the support of the technology. Software enables change but can’t drive it.

5. Take a broad view of costs

Consider all the costs of buying, installing, configuring and supporting a new application. Will the software run on the computers you have or will upgrades be needed? Can your network handle the additional load placed on it? How much and what type of training will be needed? Will any data conversion be required? How will your business be disrupted during the initial stages of implementation? What additional costs will be incurred due to the disruptions?

Building an enterprise software system is a huge endeavor. Thinking through the issues raised above will help you make better decisions. It will also make you more agile.

Are You Being Agile or Are You Writing Software?

Is your goal to be agile or to write software? Sorry. ‘Both.’ is the wrong answer!

There needs to be a focus. Something has to have priority. You can’t serve two masters. Here’s why.

Let’s say your team has used a waterfall (command-and-control) process for a while and is comfortable with it. The team has heard much about agile software development approaches and wants to give one of them a try. Why not, right?

The team reviews Scrum, Kanban, Lean, XP and maybe a few other approaches to agile development. They decide that Scrum sounds like an approach they can embrace so they adopt Scrum and kickoff a project.

Now the fun begins.

Problems, issues and concerns invariably surface. Choices and trade-offs need to be made. This is where it gets interesting.

If the team’s primary focus is writing software, the team members will naturally fall back to their comfort zones. They’ll perform tasks in ways that are familiar. The end result may generate good code but it won’t be agile.

If the team’s primary focus is being agile, they are more likely to remain in their ‘uncomfort’ zones. They’ll put in the extra effort to get the tasks done in an agile way. The end result may not be the most efficient but it will be a learning experience toward building an agile foundation.

What really matters?

Remember that a key part of agile software development is prioritizing and ordering work. If you don’t know what’s important, you won’t deliver what really matters to the business. When you don’t deliver what really matters, you fail — no matter how good your code is.

Ultimately, the team’s objective is too help the company generate revenue and/or reduce costs. The only viable reason for adopting an agile development approach is to meet one of those objectives. If you don’t believe being agile will help increase revenue or reduce cost, why are you bothering?

If you do believe that agile software development can help, then you need to make it a priority to be agile. If you don’t make being agile a priority, you won’t be.

So, I’ll ask you again — Is it your goal to be agile or is it your goal to write software?

Test-Driven Equals Quality-Driven. Develop ATDD Skills.

Manufacturing industries learned long ago that it’s better to prevent problems than fix them. That’s one of the basic tenets of being lean. Fixing problems is expensive and wasteful. Preventing them isn’t cheap but it’s less costly than fixing.

It seems that many in the software industry haven’t figured this out yet.

Software defects shouldn’t be allowed to reach system integration testing. Defects at the module or user story level need to be found and fixed within iterations or sprints. That’s where Acceptance Test Driven Development (ATDD) shows its value.

Here’s a brief comparison of how software is tested in a traditional (legacy) software project versus a newer agile approach like Scrum, Kanban, Lean or XP. The key differences are not in what gets done, who does it, or where it happens. The keys lie in when the testing gets done and how it gets done.

Here’s how the traditional software implementation and testing process works.

1. Write technical specification(s) for the software component.
2. Obtain specification approval from management.
3. Write the functional source code.
4. Test the functional code using manual techniques (or sometimes not).
5. Modify the functional code and repeat the tests (or not).
6. Submit the functional code to QA for formal testing.
7. Wait for QA to test and report problems at some time before the project ends.
8. Fix the problems reported by QA.
9. Update the specification(s).
10. Re-submit the functional code to QA.
11. Wait for QA to re-test and report problems.
12. Repeat 8-11 as often as needed to pass QA

Here’s how an ATDD process might work.

1. Select a user story and discuss the acceptance test criteria with the team.
2. Write the testing source code first. (Automated testing is the norm not the exception.)
3. Write the functional source code.
4. Run the tests against the functional code.
5. Modify the functional code until it passes the tests.
6. Refactor the functional code as needed. (There is no refactoring in the traditional approach.)
7. Re-run the tests.
8. Handoff the functional code and the test code for final verification during the sprint.

Notes:

• The quality of the acceptance tests is crucial. Discuss them in depth.
• The acceptance tests will focus on user behaviors. The test code needs to cover system behaviors as well.
• Use an automated testing tool to minimize the ‘how do I test the test code’ problem.
• Take time to refactor immediately. Minimize technical debt.
• Large projects will need to aggregate tested components and perform system integration testing prior to deployment.

The Delivery of Potentially Shippable Software Is Being Abused

Here’s one of my big problems with some agile software developers and Scrum in particular — deliver potentially shippable software at the end of every sprint. That mantra puts a lot of pressure on the team to build, build, build. Planning? Designing? There’s no time for those. They need to build something that’s potentially shippable!

No they don’t! Some people are taking the words potentially shippable too literally. The Product Owner and the team always decide what to “ship” — or not.

What ever happened to prototyping? There are times when you need to hack some code to try an idea, experiment with an algorithm, or simply test a new system capability. It might work. It might not. Either way, you win because you’ve learned something and the end product will be better for it.

Don’t treat all sprints the same.

Differentiate between the early sprints in a project and the later ones. Late in the game, stable deliveries are important. As the software nears completion (approaches “doneness“), the users come to expect a level of sophistication and reliability. That’s not the time to be experimenting or doing anything that risks breaking or delaying the final build.

The early sprints are different. The early sprints are a feeding ground for hackers. Try something new. Experiment with an algorithm. Test a new library. Change an approach. If they can’t experiment early in the project, when will there be time to prototype new thinking?

Get the stakeholders and end users involved in trying something new. Their feedback is important in determining if a new approach is viable. It’s all a matter of setting appropriate expectations for every sprint deliverable.

Delivering potentially shippable software at the end of every sprint is an enviable goal. It just not always achievable or even necessary. Apply good judgment when deciding what to deliver and how to deliver it.

Try This Experiment and Find Your Agile Limits

I’d like you to try this experiment. Take a planning activity that you and your software development team perform on a regular or intermittent basis and cut the allotted time to one tenth of what is normally expected. That’s it. Try it!

You read that correctly. You get 10% of the time that the activity would normally take. That means a week shortens to 4 hours. A month comes down to 2 days. Scary thought, isn’t it?

Here are some examples to get you thinking through this experiment:

• Prepare a product vision statement – Expected: 1 week. Allotted: 4 hours.
• Prepare project plan – Expected: 2 weeks. Allotted: 1 day.
• Write a user story – Expected: 30 minutes. Allotted: 3 minutes
• Write a test plan – Expected: 2 days. Allotted: about 90 minutes
• Lay out a UX design – Expected: 3 days. Allotted: about 2 and a half hours
• Estimate the time for a set of stories – Expected: 2 hours. Allotted: 12 minutes

You can do it.

It’s natural to think that you can’t reduce the usual time for an activity by 90% and still end up with anything of value — but you can. The point of the experiment is to change the way you and your team approach planning activities — you need to change thinking habits and establish new patterns.

We all fall into behavior patterns. After we’ve performed an activity multiple times, we naturally assume the activity will take a prescribed amount of time. Based on that, we structure our tasks such that we will finish the activity in the expected time.

There’s nothing wrong with natural behavior patterns — but they limit our ability to be agile. If you’re serious about improving how work gets done, you need to change how people approach their work. If you’re serious about creating better outcomes, you need to change the rules.

Cutting the expected time by 90% may seem overly drastic. 10% of the original time estimate may not be enough to achieve the quality results you need. That’s okay. The team will will learn a lot about what really matters in performing the target activity. Team members will also learn more about each other by undertaking this experiment. You’ll find ways to reduce the expected time — maybe not by 90% but 75%, 50% or even 25% reductions would be huge wins. Think about it.

Give it a try. You have nothing to lose. If you don’t like the result, very little time has been wasted and you can revert back to your normal approach. Please let me know how the experiment works out by leaving a comment.

Don’t Give Product Owners More Credit Than They Deserve

Are Product Owners (PO) all-knowing and all-powerful when it comes to software features and functions? I think they’re sometimes given more authority than they can handle. There needs to be a balance between business needs and technology constraints.

Here’s a specific example to illustrate my point.

Let’s say the agile development team is asked to build a software application to extract new insights from an existing ecommerce website and database. For example, the business wants to know which referral websites send the most buyers, not just browsers — they want to know how the buyers arrived at the website. Armed with this information, they can develop a better strategy for leveraging the referral sites.

The existing website contains information on referrals. Any good web analytics software package tracks these. Also, every time someone buys something, the customer database captures information about them — what they bought, when they bought it, how much they spent, etc.

Unfortunately, in the current website implementation the web analytics aren’t linked to the buyers database. The information repositories are completely separate and independent. They are housed in different places and serve different needs. The development team doesn’t know if it can align buyers and referral sites in order to extract anything useful from the existing data.

Time to write stories and define sprints?

The PO begins to lay out a collection of epics for the project and enters them into the product backlog. He and the team begin the process of breaking down epics into stories and defining the sprint backlog.

The PO wants to start with a software framework based on basic UX elements and lay the groundwork for the search and sort features that will be needed. Sounds good, right? There’s just one problem. The team may not be able to deliver anything of value if it can’t align the underlying web analytics with the buyers data.

Should the team get right to work and hope they can align the data in a later sprint? Should they spend an entire sprint doing data analysis and algorithm refinement before starting to build the application? Should they create two teams to operate in parallel on the algorithm and the application?

Any reasonable approach can work.

There are no obvious or simple answers to these questions. The complexity surrounding the data represents risk. The PO and the development team need to work together to assess the risk and determine how they want to handle it.

The safest approach is to first define a set of stories that will help in analyzing the data and defining an algorithm. The team would only commit to building the application once they find an algorithm that works. The drawback is time. The business will not see any working software until the algorithm and the first post-analysis sprint are complete.

The fastest approach is to form two teams and overlap their activities. The drawback is cost. This approach will cost more but result in faster delivery if it turns out that the problem can be solved using the existing data sets. If the data can’t solve the business problem, new data will need to be collected.

It has to be a team effort.

Without getting too wrapped up in the particulars of this example, the point is that the PO cannot define all the stories nor prioritize the work in isolation. These activities need to be undertaken by the entire team. Risk will be manifested in many ways. Only a coordinated team effort can mange it effectively.

So the next time someone tries to tell you that only the PO can make product decisions, remind him not to give the Product Owner more credit than he deserves.

Open-Source Software Thrives on Community Participation

Many companies use open-source software in some part of their businesses. Reports suggest that up 98% of all enterprise companies use open source to some degree — 98%. Server-based software such as Linux, MySQL and Apache is the most common.

Unfortunately, only a small percentage of those companies will ever give anything back to the open-source community. This limits how fast the software can improve because the only way it gets better is by engaging its users.

It takes more than source code.

Some firms have a general policy of buy, not build. They avoid making changes to the software they use not wanting to maintain the new code. This is a valid choice though contributing source code to an open-source project is not the only way to participate.

They can offer feedback on current features and request new ones; contribute to user documentation; or get involved in beta test cycles. It’s all part of being active in the community.

Participating has its benefits.

Some firms make software changes but don’t see any benefit in contributing the changes back to the community. There are several good reasons to contribute that deserve consideration.

Once the changes are merged into the official code base, patches won’t have to be applied to each new release and tested. Releasing code to the community subjects it to peer review and the opportunity for further enhancement and tuning. This is a good way to avoid unintended effects and potential security problems.

By participating in the process, companies gain influence over the direction of the project. The developers are more likely to listen to an active participant rather than someone who sits on the sidelines and complains about the software.

Occasionally, a company will view the changes it makes as proprietary, believing that they offer a competitive edge. Such claims should be examined carefully as they are rarely defensible. Proprietary innovations should be made outside the open-source software not within it to avoid legal problems.

A few other things to think about.

For companies considering participation in an open-source project, here are several factors to consider.

Open-source communities vary widely. Some have a commercial bias while others are strictly non-commercial. Some are large and well-established; others are small, one-person efforts. There are those that are tightly controlled and those that are open to all.

Review submittal guidelines on the website. Communicate directly with the developer that maintains the code you would like to change. Explain the reason for the change and the impacted components. Not every change will be approved so it’s important to defend your reasoning.

The other major variable is licensing. Some open-source licenses are flexible allowing changes and resale as long as there is proper attribution. Others are restrictive, placing conditions on how changes can be made and preventing resale.

Check the license terms in advance. Be sure you’re using the software according to the license terms before you offer code changes.

Open source is gaining traction and attracting many enterprises. Having more companies participate will make the software better and benefit everyone.

Don’t Make Commitments Until the Time is Right

Don’t make a decision today that you can put off until tomorrow.

Many product manufacturers like to operate with minimal finished product inventory. They follow the principle of postponing the final build process until a customer order is in hand. That’s lean thinking.

This technique enables them to optimize cash flow and minimize the risk that a change in customer direction will leave them with a large quantity of finished goods that few people want. They don’t commit to the final product until the customer has committed. (At one time Dell Computer was the poster child for this approach.)

Agile software development processes can apply a similar technique. Don’t make a decision today that you can put off until tomorrow. That may be a little extreme but the idea is to postpone decisions that are expensive to implement and/or irreversible. Those commitments should be made as late as possible in a lean development process.

It’s not easy to procrastinate.

You can’t postpone every decision until the last sprint. You can’t even postpone most of the major decisions until then. Some things have to be decided early but even then, there are techniques for maximizing the effectiveness of those early decisions.

Get into the habit of asking a few probing questions prior to making  irreversible decisions.

• Does this have to be decided now?
• Is there sufficient information to make this decision now?
• Will new information be available later that may affect this decision?
• Is there added cost or risk in postponing this decision?
• What are the cost and risk of making the wrong decision now?

Armed with the answers to these questions, you’ll be in a better position to make the call. Once a decision is made, turn your attention to the implementation and ask a few “what if” questions.

• What if this decision doesn’t work out as planned?
• What if the customer decides to change the software feature set?
• What if the company decides to go up-market or down-market with this software?

The idea is to get the team thinking about flexible implementations using such things as parameter files, database-driven designs, and software interfaces. Being agile refers to more than the team’s behavior. It also refers to how the software is built.

Anything about the solution that can change, probably will. Whether you’re following Kanban, Scrum or XP, introduce a little lean thinking into the process.

Resistance to Agile Development Is Not Inevitable

Many companies would like to do a better job at developing software — better quality, better delivery times, better cost / benefit. Here’s the problem, getting better means changing and change is hard.

It’s widely perceived that most of us are resistant to change. I’m not convinced of that. I think most people, including me, are resistant to being changed. Most of us want some control over, or at least input into, the change process. We resent forcible change.

Going from a legacy waterfall approach for software development toward an agile approach like Scrum, Kanban, Lean or XP involves lots of changes. As soon as the team hears about agile development, they’ll start searching the web for more information. Inevitably, they’ll read some things they like and some things they don’t. They’ll focus on the latter and so it begins — the resistance.

Resistance is difficult to overcome. You’re best approach is to do everything you can to prevent the resistance from building in the first place. How do you do that?

• Get everyone involved in the initiative to define a new, agile approach to software development. Make changes that make sense.
• Have a senior executive lead the change effort. That person needs to be actively involved not just available to help.
• Focus on values and principles first. Best practices can wait. Everyone needs to understand the reasons for change and the vision behind them.
• Emphasize that the path toward the new approach is not well-defined. It will evolve with time. Let the team determine the pace of change.
• Don’t send everyone to agile training at the beginning of the process. Training in what exactly? There needs to be some structure and definition in place before training can begin.

After getting started, use retrospectives as the primary vehicle to drive change. Retrospectives work because they involve the entire team in assessing what’s working and what’s not. Never stop changing. Make it the way business gets done.

Agile Software Development – You’re Doing It Wrong!

Let’s get this out of the way — agile software development is tough — hard, difficult, complex. (Pick your favorite adjective.)

This applies to Kanban, Scrum, Lean, XP or whatever agile approach you like. Is it tougher than those legacy waterfall approaches? Your guess is as good as mine. Agile development is simpler in some respects. For example, there is less rigor and tension at the beginning of the project. It is also more difficult in some respects. For example, good team communication is essential and we all know that communication issues are rampant in almost every organization.

So, if your organization is looking for a quick and simple fix for its software development problems, I have very bad news. There is none. Maybe you should consider a career change?

If changing careers is not an option, here are a few things to consider.

Agile development really works … if you let it.

Agile software development is not a one-size-fits-all solution. Consider that no two legacy waterfall implementations are identical. Every company adapts waterfall to it’s needs and culture. Sometimes the adaptations improve the process and sometimes they don’t. (It seems that more often than not the adaptations make waterfall worse.)

Similarly, no two agile implementations are identical either. That’s why we have Scrum-fall, water-Scrum-fall, Scrum-ban, Lean-ban — you get the idea. Everyone is trying to make agile development better — or so they believe.

Agile development approaches are meant to be tailored to the team and the environment as long as you preserve the core principles. Neither Kanban, Scrum, Lean, nor XP are prescriptive. While they all have fairly rigorous ideologies, none offer a precise formula for success — as in do this, don’t do that, and your software will be delivered on time, on budget and without major defects.

Here’s the core issue. In attempting to optimize Agile development for the organization, managers usually change the agile ideology to make it fit the company culture. That’s doing it wrong. In fact, the opposite is the optimal approach — change the culture to make it more agile.

Here’s an example. Let’s say the executive management team meets every Monday — a common scenario. And, on many Monday afternoons, the development team receives a new directive in the form of let’s stop doing that or let’s try this. Agile teams know that such ideas should be included in retrospectives and openly discussed. Applying the latest management hypothesis every Monday is not agile!

But changing culture is hard to do and that’s why many agile development implementations struggle.

Changing culture is the most difficult challenge.

What’s the solution? Start by implementing simple, agile concepts that fit the organization. Gently but firmly resist efforts to corrupt those concepts. Demonstrate success by keeping commitments reasonable and meeting them. Add more agile concepts to the process through retrospectives as the team matures. Review the Agile Manifesto regularly and adhere to it.

Agile development is tough. That’s why it works when we do it right! You can deliver better software in less time but it won’t happen overnight. Plan on using an agile approach for at least a year before drawing any conclusions.