Don’t Just Build the System Right, Build the Right System

This is the ninth in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The ninth antipattern discussed in the post is…

9. Emphasizing process metrics over actual project outcomes

It’s been my experience that some software development teams like to track metrics — sometimes, lots of them. Other teams track few, if any at all. Which approach is better?

Sadly, neither. When you track many different metrics, you’ll get mixed messages. Some of the measurements will tell you that the team is doing well while others will claim the opposite. On the other hand, if you track few if any metrics, there is no objective measure of the team’s past performance or its future potential.

Yet the biggest problem with most metrics is that they are inwardly focused. The team is measuring its output artifacts with an eye toward improving itself. Yet what matters most is the outwardly-focused, end-user experience.

If the software will be used inside the company by a business group, productivity improvements are a valid expectation. If the software will be sold to paying customers, increased revenue and market share are the most important measures.

How much time do you have?

The other challenge with metrics is that they are only as useful as the time you’re willing to devote to gathering, analyzing, reporting and tracking them. If your data collection approach is haphazard or unstructured, your results will be useless, or worse, misleading.

In the end, team performance is a function of outcomes not metrics. Don’t fall into the trap of forcing the team to meet contrived metrics at the expense of meeting user expectations.

I don’t mean to imply that metrics are a waste of time. They can be useful in your continuous improvement efforts if you use them wisely. Try following these simple guidelines:

1. Identify what is most important to the team and to the business.
2. Define the fewest possible metrics that can be used to evaluate #1.
3. Recognize that metrics are situational; they will change from project to project.

There’s more to software development than delivering high-quality software on time. It has to be the right software. Inwardly-focused metrics can’t tell you if the software is right. Only an outward focus on the end users can do that.

Invest in the People, Not in the Process

This is the eighth in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The eighth antipattern discussed in the post is…

8. Unwillingness to invest in training and coaching

Agile looks easy. All you really need are daily meetings, a status board, and active SQA participation, right? No need to spend a lot of time planning and designing. Just jump in and start writing code. The team will sort things out as they go. What could be simpler?

In fact, it’s so simple, that there’s no need for training, coaching or other expensive services. The team is bright. They’ll figure it out.

Sure they will! To anyone who thinks agile software development is easy, I have bad news for you. It’s not. In fact, getting agile development right can be more complex than getting waterfall development right.

How can that be?

Waterfall development techniques rely heavily upon documentation and their associated review cycles. Writing and reviewing are pretty basic principles that any competent individual can master. Easy? No, but the basic, non-technical skills are readily learned.

Agile development techniques rely heavily on interpersonal exchanges and the ability to fail fast. These behaviors can be difficult to master as they often get people out of their comfort zones. Team-based development and a willingness to try something new are not so readily learned.

That’s where training and coaching can help. Some degree of classroom-based instruction for the entire team can help orient everyone around the formal elements of the agile approach being used (e.g. Scrum, Kanban, XP, Lean, etc.). This helps get everyone on the same page and avoid arguments over how to do this or that.

Following up with an experienced adviser or coach during the first few iterations or sprints can help the team to hit the ground running and build a level of comfort with what they’ve learned. Classrooms and books are fine but there is nothing like doing the work and delivering results.

Even if one or two people on the team have experience with agile development, some level of independent, formal instruction for everyone on the team is valuable. If you need to minimize expenses, this is not the place to do it. Setting up the team for success is more valuable than saving a few bucks.

For the record, I’m neither a trainer nor a coach. I’m not trying to sell you anything. I just like to see software projects succeed.

Cramming Works in College But Not in Agile Projects

This is the seventh in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The seventh antipattern discussed in the post is…

7. Cramming as much as possible into every iteration to ensure that the team is kept busy

You’ve undoubtedly heard of Parkinson’s Law — “Work expands so as to fill the time available for its completion.” Many managers strongly believe it. It causes them to cram more work into a given time period than can be done. This provides some level of comfort in knowing that everyone will be kept busy. Of course, the target completion date cannot be met by definition — there’s too much work to do. Project failure, as measured by meeting end dates, is guaranteed.

There are other negative repercussions from this law as well. When a project team is under intense pressure to over-deliver, they will cut corners. Software quality, code maintainability and system performance will have to suffer the consequences. The team may react by padding their task estimates to give themselves more time. All that does is mask the problem and damage the team’s ability to meet business needs.

This suicidal management behavior can be even more damaging on agile development projects. Short iterations or sprints provide many opportunities to invoke Parkinson’s Law. As each iteration under-delivers, the team either falls further behind the business or incurs massive technical debt. Either way, the team and the business are in big trouble.

The cure lies in following agile principles.

Short deadlines and peer pressure are powerful tools. Every administrative manager, project manager, Scrum Master, and project leader should master them.

Keep tasks small and make the end results measurable. This approach provides frequent checkpoints and many opportunities for corrective action. Don’t wait several weeks (or months) for an activity to be completed only to find out that it’s late or deficient. Break up the activity into tasks that are no more than 30 hours of work (I’m assuming a 40-hour workweek with 75% efficiency.) Ideally, shoot for average task duration of 6-12 hours (i.e. 1-2 days).

Encourage teamwork, swarming, and active business participation. When the team is collaborating openly and swarming to help a team member in need, everyone will make the extra effort to get more done while meeting quality metrics. When the business stakeholders and end users are engaged in the process, additional positive pressure will encourage the team to over deliver.

Don’t force the team to accept more work than can be done. Lead them to doing more by creating many opportunities to succeed, and many situations to collaborate.

If the Code’s Good But Not Perfect Code, Plan to Refactor

This is the sixth in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The sixth antipattern discussed in the post is…

6. Postponing refactoring until later, usually after a release

Software refactoring is one of the most misunderstood and controversial aspects of agile development. Perhaps it’s simply because if we wrote perfect source code from the beginning, there wouldn’t be a need for refactoring. I can’t speak for you but I know that I’ve never written perfect code in a single pass and I don’t know anyone who has.

So is refactoring a euphemism for fixing mistakes? Yes and no. This is where it gets blurry.

Fixing reported defects is not refactoring. If the code is wrong, it needs to be corrected. However, the definition of defect is often complex. At times, what one person observes as a defect, another observes as a feature. Regardless, if the Product Owner decides it’s a defect, just fix it.

If the software is too slow in responding to user inputs, is that a defect? What if the software consumes too much memory space? Those are tough questions to answer and tend to be situational. It’s up to the Product Owner and the team to decide what action to take, if any.

We still don’t have a clear definition of refactoring. Let’s try this simple perspective:

Refactoring affects the internal operation of the software. It does not affect the externally observed behavior.

Here’s an analogy. I could upgrade the processor, the memory or the hard drive in my computer without any changes to the way I interact with it. In a sense, I’d be refactoring my system — making it better.

However, if I patch the operating system to correct a problem, upgrade the keyboard for better ergonomics, or buy a larger display, I’m not refactoring. The first fixes a defect and the other two change the behavior of the system.

Should you try to avoid refactoring?

No! In fact, do the opposite. Plan to refactor. Allocate time for it. Don’t aim for perfection on the initial implementation of any code module. Get the software up and running quickly, even going as far as cutting a few corners. Once you know that the implementation is what the users want and need, refactor to make the code lightning fast, efficient and maintainable.

Refactoring is an important part of the team’s frequent delivery and continuous improvement efforts. Embrace it.

What Problem Are You Trying to Solve?

This is the fifth in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The fifth antipattern discussed in the post is…

5. Expecting the software to be built better, faster AND cheaper

Software projects have a reputation for delivering systems that are late, costly and buggy. Having worked in the field for many years, I feel the pain that many companies experience. They seek out alternative approaches to software development hoping to solve these problems.

Agile forms of software development are a hot topic today. Scrum, Kanban, XP, Lean, etc. get a lot of attention. Go to Amazon.com and do a paperback book search for “agile software development”. You’ll get over 1,000 hits. There’s a lot of agile information available so it’s not surprising that many companies have adopted an agile approach or are interested in doing so.

Not so fast!

If your company is considering agile development or has recently adopted an agile approach, ask a simple question. “What problem are we trying to solve?” If the answer is along the lines of delivering better software (quality issue), on time (speed issue), and on budget (cost issue), the responder is trying to solve three complex, interrelated, problems. An agile approach may be able to achieve these goals but only after a lengthy trial and error period.

Most agile approaches focus on delivering a higher quality solution to the end user. This is achieved by engaging the end users in the process sooner and delivering incremental updates to them frequently.

Will this result in an earlier final delivery? It might, but finishing the project sooner is not an agile principle. The project may finish sooner if the business requirements are captured better and if less rework is needed. Agile development can help get you there but it won’t be easy.

Will agile development result in a lower project cost? It might, but reducing cost is not an agile principle either. The cost may be lower if mistakes are caught sooner and if the team can react to business changes quicker. Again, agile development can help but it’s no magic bullet.

Scrum, Kanban, XP, Lean, etc. often “fail” simply because expectations are too high and/or the investment to make the effort successful is too low. Don’t try to solve too many problems at once and don’t expect major improvement overnight.

I strongly encourage every software development team to embrace agility using an agile development approach. Simply remember a few simple rules:

• Know what you’re getting into.
• Know what problem you’re trying to solve.
• Don’t try to do too much, too soon.

Think ‘Continuous Delivery’ Not Fixed Feature Deadlines

This is the fourth in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The fourth antipattern discussed in the post is…

4. Adherence to strict feature-set goals per iteration even if extra time is needed

The old command-and-control mentality demands that software development teams establish plans and march to them. So if a team decides to implement 10 stories during a sprint and they run out of time, they must keep marching until those 10 stories are complete. At some level, this seems to be a reasonable mandate.

The team committed to delivering a set of functionality as defined by a group of stories. That commitment was reported to the business stakeholders. Commitments need to be kept, right?

What about time?

The original commitment won’t be met by allocating more time. There was a time element included in the commitment. The feature set would be delivered by an end date. Moving the end date (a commonly used tactic in software scheduling), doesn’t make the team successful.

One of the major advantages of any agile development approach is that software is delivered early and often. This is a critically important notion. The team can always add more features to the software, however, they can never reclaim lost time.

If the team misses a trade show deadline, the damage is irreparable. If the software is not ready in time for a major customer demonstration, the sale is lost. If a marketplace window is missed, competitive advantage is gone.

Think ‘Continuous Delivery’

It’s not about delivering faster and faster. It’s about making reasonable commitments and meeting them. Being agile requires timeliness. In turn, timeliness requires discipline. The team has to learn to accurately estimate the work to be done and deliver on time. If all the estimated work can’t be delivered in the time allocated, the team must assess the situation and take corrective action.

The estimates might be overly optimistic. There may be obstacles within the team that are slowing everyone down. There may be external factors beyond the team’s control that are causing delays. It’s important for the team to re-group at the end of each iteration or cycle and identify areas for continuous improvement.

In my experience, marketing, sales and customers are willing to accept fewer software features in exchange for timely delivery and a commitment to keep delivering until all the requested features are included. Time is at least as important as features, and almost always more so.

Identify the mission-critical stories and focus on them. Don’t get dragged down by superfluous features early on. There will be plenty of time for the little stuff later — if you deliver the big stuff sooner.

By the way, I don’t mean to ignore or slight quality. My operating assumption is that quality must always be kept high. You can cut corners on features. You can cheat on time. But if you compromise on quality, you’ll lose.

When You’re Trying to Be Agile, Less Is More

This is the third in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The third antipattern is…

3. Insisting that every decision be documented in writing

I’m sure you’ve heard someone say something to the effect of “If it’s not in writing, it never happened.” There are good reasons for that, particularly if the work is being done for a paying client. The problem is that the concept is often carried to extremes.

If you need to hire someone to keep up with documentation changes, you’ve gone too far. There is a sinister cascade effect that can bog down development teams. It goes something like this:

A change to the requirements forces,

a change to the functional specification forcing,

a change to the technical specification forcing,

a change to the test plan forcing,

a change to the user documentation.

Another sinister problem lies in creating lots of pretty documents. It’s not enough to merely capture information. It has to conform to a documentation standard and be formatted properly. It can easily take more time to pretty-up the information than it did to capture it.

Don’t go crazy.

Consider longevity. Much of the information software development team generate has a limited lifespan. It may only be useful for the duration of the project. If so, why spend much time on it? If the information is expected to have a long lifespan, then it makes sense to invest the time to make it clear, concise and appealing.

Focus on content, not appearance. There are many ways to capture content from simple camera snapshots to elaborate and formal documents. Always aim for the simplest information capture approach. When you’re trying to be agile, less is more.

If your client, stakeholder or manager demands comprehensive and beautifully formatted documentation, you’ll have to comply. Let him know what the impact on time and cost will be and plan accordingly. In all other situations, don’t go crazy.

Managers Manage. Teams Self-Organize. Conflicts Follow.

This is the second in a series of posts on dealing with the impediments raised in “Agile Antipatterns Are Easy to Spot, Hard to Change”. The second antipattern is…

2. Preserving the command and control hierarchy rather than letting the team self-organize

Managers are paid to manage. It’s hard for them to relax their grip and allow an agile development team to self-organize. Yet, self-organization is one of the key tenets of agile development.

Firstly, what does it mean for a team to self-organize? It simply means that the team accepts its share of the responsibility and accountability for delivering the software. This in turn means that the team must be an integral part of the project planning process and must take control of its internal workflow.

If the team can do all that, can we just get rid of the managers? No! The team still needs to rely on managers to establish direction, define projects, set priorities, create work environments, and manage external workflows.

So here’s the problem. Some managers cannot let go of the command and control mentality. They try to control too much. This antipattern may emerge from the manager’s insecurity or it may descend from the top. It is often a symptom that the corporate hierarchy has not accepted agile development. Senior management may be aware that the development team is following an agile approach but they, the senior managers, still insist on following the same old rules and tracking the same old metrics.

This leaves the front-line managers caught in the middle. They want to let the team self-organize yet they are being held accountable for every decision that’s made and every outcome generated. If you were being held accountable for every decision, wouldn’t you want to maintain command and control?

Find a mutually beneficial approach.

Effectively dealing with this antipattern means creating a mutually beneficial work process. The Product Owner (or lead stakeholder), Scrum Master (or project manager) and the offending manager(s) need to meet and arrive at a consensus.

What decisions does the team need control over in order to be successful? What decisions does the manager need to make or have veto power over?

What metrics does the team need to track internally? What metrics does the manager need to report up the chain of command?

Find a middle ground. The team may have to relent a bit and violate a few agile principles. The manager(s) may have to give a little and let the team have more control over its destiny. This give-and-take effort gets easier once the team can demonstrate success via positive stakeholder feedback. Start simple, and perhaps, not as agile as you’d like. Once you get the effort rolling, negotiate for more autonomy.

Always remember that management has the ultimate authority to redirect the team or cancel the project. Self-organization doesn’t mean the team can do whatever it wants, however it wants, whenever it wants. That’s anarchy.

Don’t Put the Organization Ahead of the Project Team

In my recent post called “Agile Antipatterns Are Easy to Spot, Hard to Change”, I listed ten impediments that prevent agile development from working well in a corporate setting. In a series of posts, I’ll offer suggestions for dealing with these impediments starting with the first antipattern.

1. Protecting the organizational structure over the team structure

Agile teams should include all the skills needed to build and deliver the software. That typically includes experts in the business domain, user experience, software design, software construction, software testing and system packaging. Outside experts can be brought in to help but the challenge is often one of timing. The experts may not be available when the team needs them and that wrecks havoc with project timing. For that reason, the use of experts outside the team is discouraged.

While many managers will agree on the need for multi-disciplined teams in principle, they insist on maintaining administrative control over their people. So for example, the test engineers work on the project team but they must report status to their manager. They are also held accountable for following corporate procedures and reporting standard corporate metrics.

This means that team members must serve two masters. Administratively, they report to an organizational structure. Project-wise, they report to the team (including the Scrum Master and the Product Owner on a Scrum project). This can create an awkward situation and place team members under undo stress.

What do you do?

First, educate! The admin managers need to understand how the team intends to operate and what agile metrics the team intends to track. Don’t assume anything. Help them understand what will happen and why. Negotiate for a mutually agreeable way to operate.

Next, keep the admin managers informed. Invite them to the daily standups. They may not have the time to attend but it’s important to extend the invitation. If they do attend, allocate time after the standups (not during) to answer questions and get feedback.

If they cannot attend, send them status updates even if it’s simply a picture of the status board. Consider setting up a 30-minute meeting with the admin managers every two weeks to provide information and solicit feedback as you continue to educate.

Lastly, plan ahead. Whether the above measures work or not, a team member is likely to be pulled off the project at some point to address a crisis situation on another project. This is an inherent risk in matrixed organizations. Accept the risk and develop contingency plans for managing it.

Don’t put the organization ahead of the project team but don’t ignore the organization either. Seek out a win-win situation.

Agile Antipatterns Are Easy to Spot, Hard to Change

There are some tell-tale signs that an organization is not ready to be agile. There may be lots of discussion about Scrum, Kanban, XP, Lean or some other agile development variation but the organizational commitment is lacking.

Adopting an agile approach to software development is not easy or quick. Doing so while protecting and preserving the current environment and culture is a recipe for stagnation. Don’t even think about it.

If you see any of the antipatterns that follow, you’re in trouble.

1. Protecting the organizational structure over the team structure
2. Preserving the command and control hierarchy rather than letting the team self-organize
3. Insisting that every decision be documented in writing
4. Adherence to strict feature-set goals per iteration even if extra time is needed
5. Expecting the software to be built better, faster AND cheaper
6. Postponing refactoring until later, usually after a release
7. Cramming as much as possible into every iteration to ensure that the team is kept busy
8. Unwillingness to invest in training and coaching
9. Emphasizing process metrics over actual project outcomes
10. Believing that agile is strictly a software development issue, not a business one