Focus on Building Complete Software Systems Not Just Software Components

So you’ve adopted an iterative approach to software development. Every ‘N’ weeks your team delivers working software. ‘N’ is usually 2-4 but could be 6-8. The point is that the project is divided into a series of iterations or sprints rather than building everything and delivering all of it in one big bang.

Congratulations! Your software development team is agile — or maybe not. We need to take a closer look at how the work is being broken down into chunks and spread out among the iterations. Let’s begin with a few basic assumptions. If you’re not doing these things, you’re not agile.

1. The team includes a test function (i.e. there isn’t a separate QA group)
2. The team delivers working software at the end of each iteration.
3. There is a backlog of work items to be done.

Sounds good, right? But wait, let’s examine that backlog. Agile software development teams are strongly encouraged to define their work chunks (e.g. user stories) as vertical slices through the software system. This approach helps in growing the system little by little. It invokes the principle of progressive refinement — the software becomes more comprehensive sprint by sprint.

Unfortunately, many teams don’t approach the work effort this way. They begin by defining a set of software components that will be built semi-independently and ultimately joined to construct the system. They then build and test each component during separate iterations and deliver them as completed units.

Here’s the problem. The software won’t function in a useful way until nearly all the components are delivered. Thus the deliverables at the end of each iteration are largely useless to the business. The components don’t accomplish anything individually. They need to be integrated into a whole system to be useful.

This component approach is iterative but it’s NOT agile.

Go back and take a closer look at that backlog. Think about user activities (or business process flows). Those are your work chunks (i.e. stories). Map the activities to your software components. Then, isolate specific functionality within each component that’s needed to support the user activities.

The goal is to build out components little by little and as needed. This way the team will deliver functional software at the end of each iteration and the components will be progressively refined. That’s agile.

photo credit: @Doug88888 via photopin cc

Technical Debt Isn’t the Only Type of Debt You Need to Avoid

Technical debt is a frequent topic in blogs covering agile software development. But it’s not the only type of debt incurred by software development teams. There’s no shortage of ways for software developers to dig deep holes for themselves.

This post explores other forms of debt that lie in wait for the unsuspecting development team. You may knowingly choose to incur short-term debt with the intent to pay it off before it gets too large. Alternatively, you may choose to avoid any indebtedness so you’ll never have to deal with repayment issues. It’s up to you.

If you decide to take on some debt to meet a critical deadline or satisfy a business need, document it and keep track of it. If the product or service you’re supporting is short lived, the debt won’t matter. If you find yourself adding many new features and upgrades, the debt will likely have to be paid off eventually.

Some of the following debt types are subsets of technical debt (i.e. defect, performance and size) while others represent different types of debt (i.e. backlog, competitive, learning, testing and useability). Some are more sinister than others but they are all formidable challenges.

Software Development Debt Types

Backlog debt – The story backlog just keeps growing. The team does a reasonably good job of delivering completed stories but they arrive at an even faster rate.

Recommendation: Either expand the software development team (this may require adding another team) or push back on the product owner and the stakeholders encouraging them to focus on what really matters in the software.

Competitive debt – Your company’s competition gets new products and services to market faster than you and grows its market share.

Recommendation: Your team is too slow. You need to crank it up. Consider adding more (and better) people. Avoid seeking perfection in the product or service. Make sure the software is “good enough” and move on.

Defect debt – The bug count keeps increasing despite the team’s best efforts to fix bugs.

Recommendation: Your team is sloppy. Make good engineering practices a priority. Focus on preventing defects using pair programming, code reviews, test-driven development and related techniques.

Learning debt – New technologies and tools are appearing yet the team continues to use what it knows and lacks the skills it needs to adopt new ones.

Recommendation: Allocate time for training, learning and experimenting. Just do it. Identify technologies and tools that will benefit the company. Make a strong argument for acquiring the skills and push for them.

Performance debt – The software system’s response time keeps increasing. It takes longer and longer to perform routine tasks.

Recommendation: This is similar to defect debt in that good engineering practices will help. Set performance goals, include them in acceptance criteria, and incorporate performance testing into the development process.

Size debt – Memory usage keeps increasing. The software’s footprint keeps growing and as it does, load times get longer.

Recommendation: Memory handling problems can be vexing. They creep up on you and get out of control. Be vigilant. Monitor memory usage during test runs. Identify memory hogging areas of the code and improve them.

Testing debt – The percentage of the system not regression tested keeps increasing. QA has their hands full keeping up with new features.

Recommendation: It’s only natural to focus on testing new features and functions. However, regression testing is almost as important. Hire good testing talent and invest in automated testing tools.

Usability debt – The software becomes more complex and difficult to use. Feature creep produces so many options that many users are overwhelmed.

Recommendation: Usability is often driven by the software developers. Yet, what makes perfect sense to them may be unintelligible to the average consumer or business person. At a minimum, conduct periodic usability reviews and make implementing the feedback a priority.

photo credit: StockMonkeys.com via photopin cc

Don’t Try This On Your Next Project

You hear the words greatness, excellence and leadership quite often in the worlds of business operations and software development. Every person, team, organization and company should strive to be the best — but don’t go too far.

What’s too far? Perfection. You don’t need to be perfect — not even close. The results you produce need to be better than they were last time; better than the competition; better than whatever you’re replacing; better than your stakeholders expect … but not perfect.

Pursuing perfection paralyzes performance.

It goes back to a variation of the over-used 80-20 rule. In this case, you can achieve 80% of perfection in about 20% of the time it would take you to reach true perfection (assuming you could get there at all).

For example, let’s say it takes us 3 months to get to 80% of the ideal goal — perfection. It would take 15 months, five times longer, to reach the ideal. If we’re building a revenue-generating product, we’d have to sacrifice an entire year of revenue to get to 100%. If the software will be used internally to improve the organization’s results, we’d have to sacrifice an entire year of improvement.

Is that a tradeoff worth making?

I know some readers will shrug this off thinking that neither they nor their teams seek perfection. But, perfection can gnaw at us in subtle ways. Consider these:

• Fear of making mistakes
• Unwillingness to take prudent risks
• Refusal to incur any technical debt
• Obsession with grammar and formatting
• Inability to make critical decisions
• Hesitating to try anything new and different

You get the idea. Perfection isn’t only about the whole project. It’s also about the artifacts and components that make up the project — the little things. Spending an inordinate amount of time on any single issue or item can derail the entire project. Before long, a year has gone by and the business is still waiting.

photo credit: becca.peterson26 via photopin cc

Incomplete Software Iterations Put Everything At Risk

Project iterations or sprints are not an excuse to cobble together some software and toss it over the wall to the end users and stakeholders. This happens because the team knows the software isn’t done yet. If it doesn’t work properly or isn’t what the users want, the team will fix it in future iterations. That’s wrong.

This mindset is a derivative of waterfall thinking. When software development teams practice an iterative waterfall approach, they deliver incremental software updates to software quality assurance (SQA) and wait for feedback. The team fixes problems found by SQA and tosses the software back for more testing. That’s the way it works.

Now they decide to make a switch to agile software development in the form of Scrum, Kanban, Lean or XP. Instead of tossing the software to SQA, the team tests the software internally then tosses it to the stakeholders and end users. The development team can always fix it later, right?

There won’t be a later.

The stakeholders, end users and senior managers will quickly become disillusioned with the quality of the software (or lack of it). They will blame the recently adopted agile process. It won’t be long before the team hears that agile development doesn’t work. It’s out of control. There’s not enough documentation. There are no management controls. Development and SQA are too cozy. It’s time to go back to waterfall!

The real problem lies in the fact that the team is clinging to its waterfall values. They have adopted an agile approach to software development but they are not fully invested in it. They have carried over many of the habits and techniques from the old waterfall process. They view iterative software deliverables as snapshots in time — the deliverables might work, they might not. Nothing is complete until the end of the project.

Is it production-ready?

This is why some agile practitioners are so adamant about delivering production-ready software at the end of each iteration or sprint. This technique forces the team to focus on quality and getting stuff done. It doesn’t matter when the project ends. The software is ready for production use at any time.

In my opinion, having each iteration deliver production-ready software is often overly restrictive. I prefer to think about production-ready releases that are composed of iterations. However, teams that are new to agile might benefit from the discipline that production-ready iterations enforce. Once the team settles into a good delivery flow, they can opt to loosen the constraints on iterations and focus on making releases production-ready .

Whichever approach you choose, always strive to deliver only high-quality software to the end users. Anything less puts your project, your team and your company at risk.

Iterative Waterfall Is Not Scrum

One of the more common anti-patterns in Scrum is treating each sprint as a short waterfall project. While there isn’t anything inherently wrong with doing that, it’s not Scrum. It’s Iterative Waterfall. If it doesn’t work, don’t blame Scrum.

There are similarities between Scrum sprints and waterfall iterations but they are not the same thing — not at all.

Software development teams using either Scrum or Iterative Waterfall can:

• Conduct daily standup meetings
• Timebox their sprints or iterations
• Allow change requests to affect future sprints or iterations but not the current one

These elements won’t operate identically for Scrum and Iterative Waterfall but the basic ideas will be the same.

What are the differences between Scrum and Iterative Waterfall?

1. Scrum teams start out with minimal planning and controls. They do just enough planning to get the development effort started. Once the software begins to take shape, they adjust as needed. Iterative Waterfall teams spend significant up front time analyzing and planning. Iterations don’t begin until the complete plan is in place.
2. The biggest difference is in the workflows. In Scrum, stories are worked from beginning to done. Thus any story may be in the Sprint Backlog, in progress, or done. In Iterative Waterfall, stories (or feature sets) all have the same status during an iteration. That is, they go through analysis, design, development, integration and testing as a group.
3. Team members have cross-functional roles in Scrum. They do whatever it takes to get stories to done. Waterfall imposes predefined roles such as analyst, architect, developer and tester. Documentation and code are passed from one role to another.
4. Scrum teams focus on delivering value to the business users at the end of each sprint. That goal may not be achieved on each and every sprint, but it is always a focal point. Iterative Waterfall teams tend to break up business requirements into chunks of work that are optimal for the development team. The business may not see working software until late in the project.
5. Scrum teams are self-contained and dedicated. They have all the skills needed to deliver the software. Iterative Waterfall teams are usually assembled from multiple corporate departments such as project management, software development, quality assurance, etc. As such, the team members have allegiances beyond the project team.

While I admittedly have a strong bias in favor of Scrum (and other agile approaches), Iterative Waterfall can and does work. It’s more agile than standard waterfall but it doesn’t fit any standard definition of agile software development. So, if you decide to use it, please don’t call it Scrum.

Agile Software Development – You’re Doing It Wrong!

Let’s get this out of the way — agile software development is tough — hard, difficult, complex. (Pick your favorite adjective.)

This applies to Kanban, Scrum, Lean, XP or whatever agile approach you like. Is it tougher than those legacy waterfall approaches? Your guess is as good as mine. Agile development is simpler in some respects. For example, there is less rigor and tension at the beginning of the project. It is also more difficult in some respects. For example, good team communication is essential and we all know that communication issues are rampant in almost every organization.

So, if your organization is looking for a quick and simple fix for its software development problems, I have very bad news. There is none. Maybe you should consider a career change?

If changing careers is not an option, here are a few things to consider.

Agile development really works … if you let it.

Agile software development is not a one-size-fits-all solution. Consider that no two legacy waterfall implementations are identical. Every company adapts waterfall to it’s needs and culture. Sometimes the adaptations improve the process and sometimes they don’t. (It seems that more often than not the adaptations make waterfall worse.)

Similarly, no two agile implementations are identical either. That’s why we have Scrum-fall, water-Scrum-fall, Scrum-ban, Lean-ban — you get the idea. Everyone is trying to make agile development better — or so they believe.

Agile development approaches are meant to be tailored to the team and the environment as long as you preserve the core principles. Neither Kanban, Scrum, Lean, nor XP are prescriptive. While they all have fairly rigorous ideologies, none offer a precise formula for success — as in do this, don’t do that, and your software will be delivered on time, on budget and without major defects.

Here’s the core issue. In attempting to optimize Agile development for the organization, managers usually change the agile ideology to make it fit the company culture. That’s doing it wrong. In fact, the opposite is the optimal approach — change the culture to make it more agile.

Here’s an example. Let’s say the executive management team meets every Monday — a common scenario. And, on many Monday afternoons, the development team receives a new directive in the form of let’s stop doing that or let’s try this. Agile teams know that such ideas should be included in retrospectives and openly discussed. Applying the latest management hypothesis every Monday is not agile!

But changing culture is hard to do and that’s why many agile development implementations struggle.

Changing culture is the most difficult challenge.

What’s the solution? Start by implementing simple, agile concepts that fit the organization. Gently but firmly resist efforts to corrupt those concepts. Demonstrate success by keeping commitments reasonable and meeting them. Add more agile concepts to the process through retrospectives as the team matures. Review the Agile Manifesto regularly and adhere to it.

Agile development is tough. That’s why it works when we do it right! You can deliver better software in less time but it won’t happen overnight. Plan on using an agile approach for at least a year before drawing any conclusions.

20 Ways to Tell If Your Project Planning Is Taking Too Long

I see too many organizations spending too much time in project planning. Actually, I take that back. They spend too much time thinking they are planning. If you want to be agile, you need to deliver software. Planning helps get you there but it can also get in the way if you over do it.

With that in mind, here are 20 ways to tell if your company is spending too much time planning the project, and not enough time delivering it. (Warning: A few of these may be slightly over the top — at least I hope they are!)

1. The business stakeholders are asking “Is the software done yet?” and you’re still planning.
2. The requested delivery date for the software passes and you’re still planning.
3. You spend more time in planning meetings than you do with your family.
4. Every time you go to a planning meeting, you meet new people.
5. You have to schedule meetings to plan the planning meetings.
6. You call meetings but no one shows up any more.
7. You’ve revised the planning documents at least 5 times.
8. The planning documents are so complex you create a taxonomy to organize them
9. The planning document set is so large you can’t use email to distribute it.
10. Everyone answers “I’ll get back to you.” to information requests yet no one ever does.
11. Writing the code is expected to take 4 weeks yet the planning has dragged on for 6.
12. People assigned to work on the project are being re-assigned.
13. Writing your risk management plan has become a project in itself.
14. The name of the project has changed at least twice.
15. Your email distribution list is so long you need Constant Contact to manage it.
16. Every time you print your planning documents the printer runs out of toner.
17. The same issues, discussions and debates occur over and over again.
18. The development team spends more time playing video games than writing code.
19. The business, tired of waiting, shows you prototype software they are developing on their own.
20. You’ve been planning for so long that the original project goals are no longer valid.

I’m fairly sure you’ve witnessed at least one of these patterns in at least one of your projects. The cure? Stop. Just stop. Write some software. Test it. Deliver it. Plan a little more — caution, I said a little. That’s how agile teams do it.

Have anything to add to the list?

Agile Development Is a Highway. Where Will It Lead You?

This is the tenth and final post in a series dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The tenth antipattern discussed in the original post is…

10. Believing that agile is strictly a software development issue, not a business one

It’s usually the software development team that decides to try an agile approach like Scrum, Kanban, XP or Lean. On the surface, these approaches seem easy to learn and use — daily stand-ups, short cycles, frequent deliveries — with some adaptation, the development team can rock!

And so it begins — the agile highway to project failure.

Let’s start by examining what we mean by ‘team’. A group of software developers doesn’t constitute a complete agile development team. Firstly, software testing needs to be an integral part of the development effort. A small project may be able to minimize the focus on testing, particularly if the team consists of experienced developers. However, a large project will need people who know how to test and more importantly, how to automate the testing.

If you skimp on testing, those short cycles will result in an ever growing defect backlog and the team may not even be aware of it. Eventually, the defects will be discovered and the stress level will rocket into the stratosphere.

Another key element on an agile development team is active involvement by the business stakeholders usually represented by a Product Owner. The stakeholders are the ultimate decision-makers. They know what the software must do. They know what features are mission-critical versus those that are merely important.

If the business stakeholders are not fully engaged in the development process, those daily stand-ups will turn into mindless status updates. Eventually, everyone will grow tired of reporting useless information.

Lastly, what about the real people that will use the software on a regular basis? They need to be part of the team as well. The developers and stakeholders can deliver useful software but if it doesn’t meet the needs of real people, it will quickly turn into shelfware — if you’re lucky. If you’re not so lucky, all hell will break loose.

Real people need to try out the software on a regular basis to provide feedback on what they like and what they don’t. The earlier this feedback is delivered, the better the final outcome will be.

Software development groups can be agile internally. That is, they can meet daily, establish short cycles, and produce regular builds. I strongly encourage that behavior. However, the only way to fully leverage the Agile Manifesto is to engage every group that will define, develop, build, market, sell, package or use the software.

A software development team can make itself agile but only a complete team can make the business agile. Which route will your company follow?

Don’t Just Build the System Right, Build the Right System

This is the ninth in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The ninth antipattern discussed in the post is…

9. Emphasizing process metrics over actual project outcomes

It’s been my experience that some software development teams like to track metrics — sometimes, lots of them. Other teams track few, if any at all. Which approach is better?

Sadly, neither. When you track many different metrics, you’ll get mixed messages. Some of the measurements will tell you that the team is doing well while others will claim the opposite. On the other hand, if you track few if any metrics, there is no objective measure of the team’s past performance or its future potential.

Yet the biggest problem with most metrics is that they are inwardly focused. The team is measuring its output artifacts with an eye toward improving itself. Yet what matters most is the outwardly-focused, end-user experience.

If the software will be used inside the company by a business group, productivity improvements are a valid expectation. If the software will be sold to paying customers, increased revenue and market share are the most important measures.

How much time do you have?

The other challenge with metrics is that they are only as useful as the time you’re willing to devote to gathering, analyzing, reporting and tracking them. If your data collection approach is haphazard or unstructured, your results will be useless, or worse, misleading.

In the end, team performance is a function of outcomes not metrics. Don’t fall into the trap of forcing the team to meet contrived metrics at the expense of meeting user expectations.

I don’t mean to imply that metrics are a waste of time. They can be useful in your continuous improvement efforts if you use them wisely. Try following these simple guidelines:

1. Identify what is most important to the team and to the business.
2. Define the fewest possible metrics that can be used to evaluate #1.
3. Recognize that metrics are situational; they will change from project to project.

There’s more to software development than delivering high-quality software on time. It has to be the right software. Inwardly-focused metrics can’t tell you if the software is right. Only an outward focus on the end users can do that.

Invest in the People, Not in the Process

This is the eighth in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The eighth antipattern discussed in the post is…

8. Unwillingness to invest in training and coaching

Agile looks easy. All you really need are daily meetings, a status board, and active SQA participation, right? No need to spend a lot of time planning and designing. Just jump in and start writing code. The team will sort things out as they go. What could be simpler?

In fact, it’s so simple, that there’s no need for training, coaching or other expensive services. The team is bright. They’ll figure it out.

Sure they will! To anyone who thinks agile software development is easy, I have bad news for you. It’s not. In fact, getting agile development right can be more complex than getting waterfall development right.

How can that be?

Waterfall development techniques rely heavily upon documentation and their associated review cycles. Writing and reviewing are pretty basic principles that any competent individual can master. Easy? No, but the basic, non-technical skills are readily learned.

Agile development techniques rely heavily on interpersonal exchanges and the ability to fail fast. These behaviors can be difficult to master as they often get people out of their comfort zones. Team-based development and a willingness to try something new are not so readily learned.

That’s where training and coaching can help. Some degree of classroom-based instruction for the entire team can help orient everyone around the formal elements of the agile approach being used (e.g. Scrum, Kanban, XP, Lean, etc.). This helps get everyone on the same page and avoid arguments over how to do this or that.

Following up with an experienced adviser or coach during the first few iterations or sprints can help the team to hit the ground running and build a level of comfort with what they’ve learned. Classrooms and books are fine but there is nothing like doing the work and delivering results.

Even if one or two people on the team have experience with agile development, some level of independent, formal instruction for everyone on the team is valuable. If you need to minimize expenses, this is not the place to do it. Setting up the team for success is more valuable than saving a few bucks.

For the record, I’m neither a trainer nor a coach. I’m not trying to sell you anything. I just like to see software projects succeed.