User Stories Aren’t Just About Business Users

Some folks criticize agile approaches to software development like Scrum, Kanban and XP, with the claim that there’s too much emphasis on the end user. How can you develop software systems that are robust, fast and secure using stories? End users are not going to write stories defining the technical characteristics of a system.

That last statement is true — they’re not. Would a software developer even want them to define software implementation details? I wouldn’t.

Such nuances aside, the technical or non-functional characteristics of a system are just as important as the features and functions seen by the end user. No one wants to deal with a software system that’s slow, unreliable or insecure.

Covering all aspects of a system’s behavior begins with the 3 C’s — Card, Conversation and Confirmation — so let’s start there.

Card

You’ve heard it many times before — stories need to be short regardless of the role of the user or the nature of the story. They should fit on a 3×5 inch card.

Conversation

I’d prefer to use the term Collaboration but regardless, business people and technical people need to discuss each story and assess its implications. At times, seemingly simple requests turn out to be complex. For example:

As a financial analyst, I want to download all transactions for last quarter into a spreadsheet so that…

What if the number of transactions exceeds one million? The file may take a long time to generate and it will be too big to load into Excel. The request could be batched so the user doesn’t have to wait for completion. The file could be split up into two or more subfiles though there needs to be an algorithm for deciding where to make the splits. Or, perhaps whatever the analyst wants to do could be done by the software internally eliminating the need for downloads.

It’s up to the software development team to drill down and assess the implications of a story and design a solution set that solves the problem without creating new ones. This is only possible when open conversations take place.

Confirmation

This ‘C’ is about testing the story and satisfying the definition of done. Testing any story should cover boundary conditions, performance characteristics, security concerns, etc. These non-functional issues won’t be explicitly addressed in the story but they are just as important.

When discussing a story with the Product Owner, take the time needed to inform and educate. Be sure that the point value assigned to the story covers all the work required to deliver a complete solution.

But wait, there’s more!

You can probably tell that the approach I’m advocating will result in some stories growing in size. You may need to do some story splitting to make the work units fit inside your sprints. That’s okay. As long as everyone is collaborating, there will be no surprises and the end result will be robust.

It’s also okay to have some system stories. For example, your company may have a data warehouse and you’ll be required to extract, translate and load transactions into the warehouse nightly, weekly or monthly. It’s perfectly fine to have a story (or epic) along the lines of:

As a corporate data store, I want the ability to send transactions to the data warehouse each night so that…

The “user” doesn’t have to be a person. It can be another system.

In summary, think about the technical characteristics of the system when evaluating and implementing every story. Add system stories when needed to cover functionality that users may not be aware of but is mandated for other reasons. The end result will be a software system that meets everyone’s needs.

7 Warning Signs of Imminent Project Doom

Is your software project in trouble? Before you try to convince yourself that all is well, you should think carefully about the subtle clues I’m about to give you. Your project could be in much worse shape than you think.

Software projects don’t march merrily along until suddenly one day all hell breaks loose and the project is far behind schedule. I know it seems that way on many projects but, trust me, it’s not. Software projects get into trouble at a slow and steady pace until one day it becomes painfully obvious that the project ship is sinking.

We all need to learn how to spot the following early warning signs so we can take corrective action before it’s too late.

• Stories or feature sets take longer and longer to implement. Things that appear simple turn out to be complex. It seems that every time the team implements a new feature, something else that’s seemingly unrelated breaks. These are signs of too much technical debt. The software has become unwieldy.
• Team members and/or stakeholders lose interest. You can spot this when people are late for meetings or don’t show up at all. Or, they don’t meet their commitments and you have to chase them down. Lost interest generally occurs when people are assigned to multiple projects and yours is not high on their priority list.
• Communication breakdowns become more frequent. If people are forgetting to share information or keep the team informed of events that impact the project, they clearly have more important things on their minds.
• People are working longer hours. If there are people on the team consistently working late or on weekends, something isn’t right. An occasional burst of extra hours to meet a critical deadline is fine. Regular late nights and weekends is not. It will lead to an increase in mistakes and frustrations.
• Members of the team keep getting pulled away to help out on other projects. Some companies will make an “all hands on deck” call to salvage a troubled project. Invariably, this results in more projects in trouble. If you see this pattern repeating, your project is pretty much doomed.
• Ignoring declines in team metrics. Even when teams measure their performance using something like velocity or earned value, they often ignore deteriorating performance. Their attitude is that it’s a temporary glitch and will improve. Really? If you see dropping productivity and missed deadlines, it will only get worse.
• Project scope keeps getting reduced. Missed deadlines and commitments are often met with reductions in scope to keep the timeline intact. That’s not such a bad thing unless it occurs over and over again.

If you spot any of these warning signs, take action — don’t ignore the situation. Hopefully, the organizational environment is such that open and honest communication can take place. You need to assess the situation and take corrective action before the project enters a death spiral.

If your management doesn’t want to hear bad news, before long your project might be on the receiving end of an “all hands on deck” call. Sadly, that won’t solve your problems. It may actually make them worse.

Iterative Waterfall Is Not Scrum

One of the more common anti-patterns in Scrum is treating each sprint as a short waterfall project. While there isn’t anything inherently wrong with doing that, it’s not Scrum. It’s Iterative Waterfall. If it doesn’t work, don’t blame Scrum.

There are similarities between Scrum sprints and waterfall iterations but they are not the same thing — not at all.

Software development teams using either Scrum or Iterative Waterfall can:

• Conduct daily standup meetings
• Timebox their sprints or iterations
• Allow change requests to affect future sprints or iterations but not the current one

These elements won’t operate identically for Scrum and Iterative Waterfall but the basic ideas will be the same.

What are the differences between Scrum and Iterative Waterfall?

1. Scrum teams start out with minimal planning and controls. They do just enough planning to get the development effort started. Once the software begins to take shape, they adjust as needed. Iterative Waterfall teams spend significant up front time analyzing and planning. Iterations don’t begin until the complete plan is in place.
2. The biggest difference is in the workflows. In Scrum, stories are worked from beginning to done. Thus any story may be in the Sprint Backlog, in progress, or done. In Iterative Waterfall, stories (or feature sets) all have the same status during an iteration. That is, they go through analysis, design, development, integration and testing as a group.
3. Team members have cross-functional roles in Scrum. They do whatever it takes to get stories to done. Waterfall imposes predefined roles such as analyst, architect, developer and tester. Documentation and code are passed from one role to another.
4. Scrum teams focus on delivering value to the business users at the end of each sprint. That goal may not be achieved on each and every sprint, but it is always a focal point. Iterative Waterfall teams tend to break up business requirements into chunks of work that are optimal for the development team. The business may not see working software until late in the project.
5. Scrum teams are self-contained and dedicated. They have all the skills needed to deliver the software. Iterative Waterfall teams are usually assembled from multiple corporate departments such as project management, software development, quality assurance, etc. As such, the team members have allegiances beyond the project team.

While I admittedly have a strong bias in favor of Scrum (and other agile approaches), Iterative Waterfall can and does work. It’s more agile than standard waterfall but it doesn’t fit any standard definition of agile software development. So, if you decide to use it, please don’t call it Scrum.

You Can’t Be Agile If You’re Not Empowered

Do you feel empowered in your job? Many people don’t. A big part of agile software development — and business agility, in general — is empowering people to change their work environments. If you can’t change anything, you can’t be agile. You can only go with the flow, and hope for the best.

By definition, being empowered means being permitted or authorized to take action. Consider the questions below. They’ll help determine how empowered you really feel.

• Do you spend too much time waiting for others? Do you believe you can change the workflow to eliminate the delays?
• Is your system or network infrastructure too slow? Do you think anyone would listen if you raised the issue?
• Have you observed problems occurring repeatedly? Have you offered suggestions for preventing them?
• Are many people around you often complaining about the same issues? Does anything ever seem to change?
• Is someone assigned to your project who seems to add no value at all? Does management take any action?

If you answered ‘yes’ then no’ to any of the paired questions above, either you aren’t motivated to take action or you don’t feel empowered. Not being motivated is okay. If you simply want to go to work each day, do your best, and go home at a reasonable time, you’re not the problem. All we can expect of anyone is that they work hard and carry their share of the workload.

However, if problems with company culture, business processes, or management styles prevent you from offering improvement suggestions, there’s a problem. You don’t feel empowered. When people don’t feel empowered, the status quo gets locked in. When the status quo gets locked in, nothing ever changes. When nothing ever changes, businesses crash and burn.

Why would you want to work for a business that will eventually fail?

If your software team wants to adopt agile software development, for example, Scrum, Kanban, Lean or XP, you better feel empowered. The underlying cultural, procedural and managerial systems have to be supportive for agile development to succeed.

If they’re not, maybe you should find another place to work.

In Software, Logical and Reasonable Doesn’t Mean Best

Small software development teams in small companies usually embrace agile practices associated with Scrum, XP, Lean and Kanban more readily than large teams at large companies. That makes sense given the added complications of trying to be agile in a big company accustomed to following a corporate process.

Small teams quickly adapt to shorter sprints, more frequent production deployments, and less big design up front (BDUF). They simply feel more confident in making decisions on the fly and self-correcting as they go.

Large, global teams tend to introduce waterfall characteristics into their agile development efforts. They try to align their corporate practices with agile techniques to achieve the best of both worlds.

Seems logical and reasonable, right?

Yes it does … but no it isn’t. What I’m about to tell you might seem counter-intuitive and difficult to accept but give me a chance to explain.

Large teams should aim for short sprints — 1-2 weeks at most. They should implement continuous integration and deployment practices — not simply deploying at the end of sprints. They should spend less time designing and more time experimenting — forget BDUF entirely.

Like I said, it sounds counter-intuitive but it’s the best way to deal with multiple, diverse, development teams. Rather than get into a philosophical debate, I’ll discuss an example showing why these ideas will work.

A Call Center Example

Let’s say we have a large software development group working on a call center application. To keep this simple, assume the software has three major components:

1. Customer Management
2. Help Ticket Management
3. Call-Center-Agent Workflow Management

Assume we have three development teams and each team is working on one of these system components. Also assume that the teams are geographically dispersed across the United States, Russia and India.

The challenges faced in such a situation are fairly obvious. The three development teams have to be kept in sync. The challenges of dealing with multiple languages, time zones, and cultures will converge into a heavy burden for the teams, regardless of the underlying software development approach.

More Controls Might Make Matters Worse

Many companies react to this kind of situation with more process, more documentation, and more sign-offs. Does this additional administrative overhead really help? In waterfall development, there is simply no way to know the answer until all the code is submitted to SQA for final verification and validation. If problems and discrepancies are discovered, it will likely be too late to address them and stay on schedule.

In a conventional agile approach like Scrum, the teams will likely use 4-week sprints. It will take at least 4 weeks for miscommunications to surface. Once they do, the teams will already be into the next sprint and the issues will go into the backlog to be addressed in a future sprint. It could take 4-8 weeks for an issue to be resolved. That’s not very agile, is it?

The solution is to keep sprints very short — 1-2 weeks. This forces the teams to sync and stabilize often. There’s no time to break the code, re-work it, and repair it. Keeping the build stable becomes a major goal. (Even better, consider abandoning sprints entirely and moving to Kanban.)

Adopt continuous integration and deployment practices. There’s no need to wait until the end of a sprint to release new code into the target build. Get that new code out there as soon as it meets the definition of done. If something breaks, drop everything and fix it. Now! Every team will know exactly where it stands at all times.

Finally, avoid the urge to spend excessive amounts of time researching, planning and reviewing. Instead, write some test code. Try something. Create a working model or a skeletal system. Evolve that test code rapidly and use it to guide future sprints.

Plans and designs are notoriously difficult to manage across global teams. For that reason, keep the plan simple. Keep the software design flexible. And most important, keep the build stable and keep the teams in sync.

Define Audiences, Specify Outcomes, or Failure Will Stalk You

There are two areas of software development that are often overlooked or under emphasized. They are defining your target audience (or end-user personas) and specifying your promised outcomes (or business goals).

We need to be specific about our target audiences and promised outcomes. It comes down to setting expectations. You can deliver what you documented but the project can still fail simply because the business was expecting something else. Don’t let that happen.

Target Audience

Can you describe the target audience for your software? There are an immense number of possibilities and that’s why you need to narrow it down. Here are a few general categories:

• Business User
• Desktop
• Mobile
• Technical User
• Desktop application
• Server application
• Knowledge Level
• Novice
• Expert
• Income Level
• Poor
• Wealthy
• Physical Attributes
• Healthy, no significant medical issues
• Physically impaired (including advanced age)

There are lots of ways to slice and dice your user base. You might be inclined to say that anyone can use your software. It’s a platform open to everyone. That’s the wrong answer and here are two examples showing why.

Facebook and Twitter can claim to have universal appeal. Some people use them for business, others for pleasure. However, both services have a small percentage of hard-core users. Those are the people that spread the word, draw in others, and ultimately increase the popularity of both services. Without the hard-core users, neither service could flourish. Who are they?

Who are your hard-core users? Can you describe them in detail? If not, you’re likely trying to please everyone. You won’t.

Promised Outcomes

What promises are you making to your end users, paying customers, or corporate clients? Software offers many promises. Here are a few examples:

• Faster workflow
• Fewer mistakes
• Simpler work activities
• Automation of manual activities
• Integration with other corporate or cloud services
• Lower costs
• Improved reporting

The people using your software deserve to know precisely what to expect. Don’t simply offer them a smorgasbord of new features, enhancements and bug fixes. If you know your target audience, you know their needs. Tell them how the new software will satisfy those needs.

By doing so, you’ll get people to listen. They’ll want to know more. Excitement will build. They’ll be engaged in the effort.

What promises are you making? Can you articulate them in clear business terms?

Don’t let failure stalk your projects. In this case, the development approach doesn’t matter. Scrum, Kanban, Lean, XP and waterfall can suffer the same fate. If you can’t answer the questions above, find someone in the organization who can and enlist that person’s help today.

Hidden Assumptions Can Take Down Your Project

What are you assuming? Every project plan, user story and task list contains assumptions. An assumption is something that’s taken for granted by the software development team — something presumed to be true. This is in contrast to risks, in that risks are adverse events that we believe are unlikely to occur.

Some assumptions will be obvious and others will be obscure. If an assumption proves to be wrong, the team will be adversely impacted. (That would be a risk.) Here are a few examples of obscure or hidden assumptions:

• The team will remain intact for the duration of the project.
• Computer systems and data will be available to the team when needed.
• Development tools and technology infrastructure will be adequate to support the project.
• People outside the core team will be available when needed.
• Outside parties (e.g. vendors, suppliers, business partners, etc.) will deliver as promised.
• User stories have been properly scoped and estimated.
• Task lists are thorough.
• The definition-of-done covers the needs of every story.
• Any team member will be able to select any story and complete it.
• Team members have adequate knowledge and training to complete the project.

You get the idea. We all make a variety of assumptions when making plans and commitments. Usually, those assumptions prove to be correct but not always. And, the more assumptions the team makes, the greater the likelihood that something won’t go as planned. (A risk event will be realized.)

One reason why so many development projects take longer than planned is simply because one or more assumptions proves wrong and the team has to scramble to address an unplanned issue. For that reason, it’s a good idea to take a moment to think through the development team’s assumptions.

Ask yourself if the project goals and timeline seem reasonable? Is the team ready? Is the environment suitable? Are people outside the core development team engaged? What’s missing?

Projects tend to take more time than we expect simply because most of us are optimists — we want to believe. Don’t give up your optimism. Just take a little time to validate it.

Software Outcomes Matter, Test Results Don’t

How do you determine the success (or not) of a software project? How do you measure the final outcome? How do you know if the project delivered what was expected?

You might be inclined to refer to the number of remaining known defects, the number of test cases, the improvements made in the code, the number of new features added, the feedback from the Product Owner, etc. Sadly, those are all the wrong metrics and don’t measure business outcomes.

What matters is end user perceptions and business value, not internal software metrics.

Real people, conducting real business with the software is the only true measure of outcomes. Here are a few real world examples where software teams thought they did a good job but the users disagreed. (I’ve avoided specifics to protect the guilty parties.)

• The software application functioned perfectly but was too slow to be useful by the large, target group. (Performance testing might have caught this.)
• The software worked properly but required too much manual data entry. (Useability testing might have caught this.)
• The software ran into a conflict with another desktop application meaning some people could not use it. (Additional analysis and testing of the target environments might have caught this.)
• The software worked fine but the new workflow actually took longer to get things done than did the old workflow. (Again, useability testing might have caught this.)

I’ve used the phrase “might have caught this” above because there is no guarantee that any laboratory testing will catch any particular complex failure scenario. Testing will catch the major problems and the expected ones. The subtle environmental issues are another matter.

This leads me to a more central question. If the ultimate measure of successful outcomes is the users’ perception, why aren’t teams doing more user testing? It’s not a simple thing to do. I get that. But, clearly we have to try harder. Here are a few suggestions.

• If you’re doing straight beginning to end waterfall, stop! You need to introduce interim deliverables. Break up the waterfall flow into a series of mini-waterfalls that each deliver useful functionality. Or, even better, dump waterfall entirely and switch to Scrum, Kanban, Lean or XP.
• If you’re already doing waterfall iterations, get the software into the hands of the end users sooner and more often. Draw them into the process. Engage them. Setup a small number of systems where users can stop by anytime and try out the latest build. Have lunch and learn sessions. Get creative. You need real feedback from real people.
• If you’re already using an agile development approach, focus more effort on the needs and concerns of the end users. It’s not about technology. It’s about them. Empathize more and they will respond in kind.

Recognize that software is a tool not an end game. Real people use software to accomplish something that adds value to their job functions or enhances their personal lives. Laboratory test environments can’t replicate every real work condition. There are too many variables.

Team Velocity Is Not as Simple as It Seems

The subject of Scrum team velocity can be confusing and controversial. My intent is not to add fuel to the fire but rather, to contribute to the knowledge base around velocity and what it represents.

The simplest way to do this is via an example.

Let’s say a Scrum team has an average velocity of 20 story points per sprint. Everyone considers that to be a pretty good number. But what does it really mean?

I’d start by asking how many sprints were completed to arrive at that number. In other words, how many sprints are we using to calculate the average? I’d also want to know the smallest velocity and the largest one to get a sense of how widely distributed the velocities are. (Excluding the first 2-3 sprints might be wise as Scrum teams often need a few sprints to sync and stabilize as a functional unit.)

These additional parameters are important. If a small number of sprints were used to calculate the average, the team may not have established a repeatable cadence as yet. If the range of velocities per sprint is very wide, I’d have to wonder if the team is operating well as a whole or simply consists of several developers working near each other.

Now it gets more complicated.

I’d also ask about story points being created during sprints. These new points could be defects or they might represent functionality that no one thought about in advance but is required before the final release. Either way, if the team completes 20 story points and creates 5, their effective velocity is somewhere between 15 and 20.

If the 5 new story points are bugs, should those points be subtracted from the sprint velocity so 20 points of calculated velocity becomes 15 points of effective velocity?

If some of the newly created points represent new functionality, it gets tricky. New functionality gets added to the product or release backlog depending on when the new functions are needed.

If a new function is critically required before the release is complete, the new points need to be added to the release backlog, thus delaying the final release date. If the new function is optional, the new points can be added to the product backlog and ignored for now. (BTW, a release consists of several sprints.)

Inevitable arguments will ensue over whether an issue is a defect or a new function. Let’s not go there.

There is also a more general argument to be made that knowingly adding to technical debt also reduces velocity. So if a developer takes a short cut in the code, he should estimate the story points needed to pay off the debt and that value should be subtracted from the team’s velocity for that sprint.

This involves a bit of a judgement call in that the team might decide not to pay off the debt anytime soon. Regardless, the technical debt exists and won’t simply go away.

Velocity is intended to be a simple metric and in principle it is. Unfortunately, nothing about software engineering for enterprise business users is truly simple. Should you track velocity? Yes. Should you reduce it as I’ve described above? Not for a new Scrum team but be aware of the issues and factor them into your retrospectives.

Oh, and please don’t try to simply compare velocities across multiple Scrum teams. That gets even more complicated!

How Much Project Planning Is Too Much?

Does your organization spend weeks and weeks preparing to launch a project? Does it seem like new projects take forever to get off the ground? Have you seen projects get cancelled after months of planning but no implementation activity?

I’ve seen it all and it can get ugly. As a general rule, planning and analysis should take no more than 20% of the time for the entire project. I’ve see that number go over 50% in some cases. For example, 6 weeks of planning and analysis for a 12-week project. That extra planning time delays the solution implementation and may prove to be very expensive.

I’ve even seen teams put together lengthy and detailed plans simply because they were required to do so. Once the projects got rolling, the plans were ignored and the teams operated day to day. (I don’t recommend that approach.)

The Law of Diminishing Returns

If something truly new and innovative is being done, additional planning time makes sense. Even if it’s not innovative but complex, more planning might help. But if the project is a variation of something that’s been done before, 20% is more than enough time to plan. So, why does all this extra planning occur?

Planning, like any other business effort, succumbs to the law of diminishing returns. All it means is that it’s relatively easy to throw together a high-level plan. Many of us have done it during a one-hour planning meeting. You may have 60-80% of the plan in place in a few hours or a few days at most. Now comes the hard part. The details of the plan. Ironing out the remaining 20-40% of the plan could take weeks. Is it worth it?

Plan less. Deliver more. Gather feedback and repeat. That’s how to be agile. Here are a few guidelines:

• Prioritize. What are the most important goals for the project? What are the high-risk areas? Those items may need more planning.
• Resist the urge to pile items onto the planning list. A long list of planning items doesn’t make a good plan.
• Remember that a complex plan will require more effort to track, maintain and adjust, increasing the project’s administrative overhead.
• Don’t get lost in the planning jungle. The goal is to deliver a business solution not to create a beautiful plan with lots of checkboxes, charts and graphs.
• Fewer people will likely deliver a better plan. Involve people who have unique knowledge or skills.
• If you’re unsure of something and cannot adequately plan ahead, allocate time during the project to adjust the plan.
• Add new requirements, stories, etc, to the backlog as you go. You don’t have to define everything up front.

Planning is a good thing. Just don’t get carried away. You’ll end up changing it anyway.