Distributed Agile Works Better

Let’s be honest — managing geographically distributed teams is tough. Here’s a simple breakdown of how teams might be distributed from least distributed to most:

1. Shared space: This is the simplest. Everyone is in close proximity in a shared office area.
2. Separated shared space: Everyone is in the same building but in different areas/floors.
3. Different buildings: The team is housed in different buildings not within walking distance.
4. Long Distance: The team is separated by many miles but in the same time zone.
5. Time Difference: The team is in different time zones but only a few hours apart.
6. Day Difference: The team is many hours apart and normal work hours barely overlap if at all.

As you go down this list, managing the team becomes more complex. Can agile software development work even at #6? If so, will agile techniques make the process better?

I think the answer to both questions is yes, with a few qualifications. Agile works best when teams can collaborate constantly. Questions should be asked and answered immediately. Issues should be resolved or backlogged right away. Being widely distributed adds complexity.

Teams often fall back on waterfall in these situations. The feeling is that if you write everything down in excruciating detail you minimize the need for collaboration. That is so wrong!

Bulky documents are hard to understand under the best of circumstances. Add in language barriers, just to make things interesting, and you have a failure waiting to happen. Do not try this at work!

Instead, adopt some or all of the following agile measures:

• Automate. A physical status board (e.g. Scrum or Kanban board) will not work. It must be electronic and shared by everyone.
• Settle on good collaboration tools. Email is not enough. You’ll need some form of instant messaging for conversations and a private Twitter-like tool such as Salesforce.com’s Chatter will also help. Invest in video conferencing if you can.
• Keep your iterations or sprints short. 4 weeks is an absolute upper limit. 1-2 weeks is better. You’ll want frequent checkpoints to validate that everyone is in sync.
• Daily meetings are tough when many hours separate the team. Be creative. If the meeting is truly inconvenient for some, can you offer something in exchange for making the effort to attend? Can the meeting time change periodically to spread the inconvenience? Would meeting 1, 2 or 3 times per week be adequate? (If your sprints are only 1 week long, maybe you don’t need a daily meeting.)

Distributed software development is an area where agile techniques shine. Agile reduces risk and improves team dynamics.

Agile Development Is Not All Or Nothing

The transition from chaotic software development to agile has got to be easier than the transition from waterfall to agile. If a team is not following any process — just winging it — adopting a structure, any structure, can only help.

In my experience, people don’t like chaos — complete lack of any process — nor do they like a rigid, tightly-controlled process. Agile software development presents a comfortable middle ground to teams in chaos.

On the other hand, teams that are using the waterfall approach already have some structure. The process may have holes and weak points but at least it offers rules and guidance.

Waterfall teams turn to agile development for one of two reasons:

1. Too many projects are late, over-budget, and/or buggy. The company needs better results.
2. The competition is getting products and services to market faster. The software development team needs to pick up the pace.

Often, out of frustration, management decides to try something new and agile is a hot buzzword. The team is likely to feel stressed and likely to be resistant to any new approach.

A better way to go about this might be to introduce agile techniques while still using waterfall. Consider the following approach as an example:

• Use planning poker to prepare your development estimates. Forget points. When estimating features, modules, tasks, etc., get the team together and compare estimates. Discuss and arrive at consensus. Use the basic concepts of planning poker to get the team used to the idea of generating estimates together.
• Hold daily 15-minute meetings. Get the team into the habit of making daily commitments and sharing information.
• Hold retrospectives at the end of each project phase and after major deliverables are achieved. Seek continuous improvement.
• Have some developers try pair programming and report back to the group on their experiences.

By adopting some agile techniques and getting the team used to the concepts, full-scale agile adoption will be easier and more likely to succeed.

Oh, and please go through with full-scale agile adoption. Things may improve by simply doing what I describe above but you need to commit to following through to get all the benfits agile software development offers.

Business and Technology Worlds Collide

Dealing with the business users of a software application can be a real challenge. It’s rare that you find a user group that embraces agile software development and wants to actively participate in the process.

There are two predominant types of business groups that you’ll encounter in agile development. The first is a business group that just wants you to go off and build the software. They’ll be happy to answer questions but are too busy to participate in the development process in a meaningful way.

Try negotiating for a business representative. This should be someone who can commit to attending daily meetings and will make the effort to test sprint deliverables. It could be anyone as long as the business owners trust their representative’s judgment in making software decisions. That’s probably the best you can achieve.

Another type of business group is one that is constantly changing features, functions, goals, process and anything else they can. It’s a major challenge to get such a group to settle down. This behavior is likely to be a reflection of how they operate in general. They won’t change.

The only thing the development team can do is anticipate change. You can’t predict what will change next but you can take some extra time to make the system flexible — not easy to do, but you can minimize the effort needed to make software changes.

Getting business participation on an agile project can be tough. Take what you can get and make the best of it.

Scrum in Trouble? No, It’s the Engineering.

There seems to be a small but loud group of agilists proclaiming that Scrum is dead or dying. I’d guess that many of them declared that waterfall is dead at an earlier time. They were wrong then and they are wrong now.

I think many of the Scrum critics have an agenda. They either have a vested interest in another approach like RUP or Kanban, or they are “experts” looking for media attention.

Let’s start with waterfall software development. It is alive and well, particularly within major corporations. I say ‘well’ in the sense that waterfall is widely used and even succeeds often enough to remain viable.

Sadly, one of the primary reasons for its ‘success’ is low expectations. Many have come to accept that software development is slow, late, defective and expensive. With expectations so low, it is relatively easy to succeed.

But I digress. The point is that waterfall lives on and even prospers in a sense.

As for Scrum, it has met with considerable success. It is relatively simple to understand and adopt. It is also easy to customize and that’s where the problems begin.

Many teams using what they call Scrum have modified waterfall to make it Scrum-like. They may call their software development approach, Scrum, but it’s not. The inevitable clash between agile and waterfall occurs and Scrum gets the blame for being hard to do well.

The truth is that software development is difficult regardless of the methodology used. Scrum is not in trouble. Software engineering is in trouble.

In the end, people make the difference. People succeed or fail — not projects and not processes.

Be Agile: Build a House Iteratively

People often struggle with how to apply agile techniques, particularly iterations or sprints, to their situation. They are so entrenched in the ways of waterfall that they cannot envision another way.

Let’s consider a simple example — building a house. The traditional approach looks something like this (simplified):

1. Design the house.
2. Dig a big hole.
3. Pour the foundation.
4. Frame the structure, walls, floors, ceilings, and roof.
5. Run the plumbing, heating and electrical systems.
6. Seal up the structure with the appropriate materials, wallboard, flooring, roofing, etc.
7. Install cabinets, appliances, fixtures, etc.
8. Paint, wallpaper, carpet, etc. to make it feel like a home.

Many people have a hard time looking at the design of a house and envisioning what it will look and feel like. They often end up disappointed in some respects.

What if you hate disappointment and want to build the house in sections so that you can try it out before completing the project? It’s hard to envision how this could be done using agile techniques. How do you break this down into “stories” where you can build vertical slices?

Think in terms of modular construction. To start, you have must a kitchen, bathroom and bedroom. The rest can be added. You build components and tie them together something like this:

1. Design the first components.
2. Dig a small hole.
3. Pour a foundation for the kitchen, bathroom and bedroom (one story).
4. Frame the structure.
5. Run plumbing, etc.
6. Seal it up.
7. Install cabinets, etc.
8. Paint, etc.

You can move in and try it out. Want more space? You have two options, add a second floor or enlarge the foundation.

To add a second floor, repeat the above skipping steps 2 and 3. (Of course, you’ll need to remove the roof!) To enlarge the foundation, do not skip steps 2 and 3. (Of course, you’ll need to remove a wall or make a doorway!)

If you used pre-built modular housing sections, this approach would be simpler and faster though your options would be more limited.

Of course, it’s unlikely you’d build a house this way simply due to the inconvenience. But, it could be done and the final result is much more likely to be exactly what you want.

There is some re-work involved but be honest — every project contains re-work. The amount of re-work varies but it is inevitable on all but the simplest of projects. Admit that it happens and plan for it.

Software is easier to “enlarge” than a house so the approach is more practical in the software business. Agile software development really works — you just need to wrap your head around the concepts.

The Scrum/Kanban Board: Physical or Electronic?

There are two kinds of agile teams — those that use a physical Scrum/Kanban Board and those that use an electronic one. Which is better?

Without a doubt, the answer is … whichever one works for your team! Neither approach is fundamentally better than the other though the electronic approach has a major downside that you must work around.

Have you ever been in a meeting where the organizer is using a projector to show a document or spreadsheet? She makes updates as you watch and is in total control of the whole process. At times, you just want to grab the keyboard and change something yourself!

The danger of using an electronic Scrum/Kanban Board is that the person holding the keyboard will dominate the daily stand-up meetings and control the outcomes.

The nice thing about a physical Board is that anyone can walk up to it and make a change. It is fully visible at all times and presents the project as an “open book”.

I think an electronic solution can still work. You could use a specialized solution like Jira/GreenHopper or simply use a spreadsheet. If you go the electronic route, here are a few guidelines:

• Provide software licenses to everyone on the team (chickens and pigs)
• Allow anyone to update the Board (it has to be network accessible)
• During meetings, setup a team computer and allow everyone to make changes

Remember, the team’s goals are building good software and managing risk. The Board is simply a tool.

Agile Adoption Requires Adaptation

A blog post at the Harvard Business Review, “Why Best Practices Are Hard to Practice”, is worth reading for anyone trying to introduce agile practices into a waterfall shop. The two key ideas in the article are:

1. Introducing a new process into an organization requires adaptation. Just because an approach worked in another firm doesn’t mean it will work in yours.
2. The introduction of a new process requires full adoption. The management team needs to buy in and actively support the approach.

Adaptation is the need to fine tune a new process so that it fits the organization. Company culture, management style, development tools, and even office space constraints will impact how an agile methodology is implemented.

Don’t get into a fight with the establishment. Try to adapt your techniques to fit into the organizational norms. For example, if a group manager assists on attending the daily Scrum meetings, let him. Try negotiating how the person should behave at the meeting. Set aside time to discuss the results of each day’s meeting so the manager feels comfortable.

Adoption reflects the need to get support from managers across the organization. They don’t have to be staunch supporters. At a minimum, they need to agree to give agile a chance — to wait and see.

Project success requires that everyone on the team is engaged and focused. If anyone feels like they are being pulled away or distracted, success becomes more difficult.

So be sure you adapt to the situation and be sure the organization adopts agile for real. There are no guarantees but your odds will improve — and there will be a lot less stress.

Get Out of the Way

The one word that best encapsulates the spirit of an agile project is teamwork. You cannot be truly agile without teamwork.

You may want to be. You may think you are. You may even use agile techniques. But if your “team” members are favoring their individual agendas; if they are more interested in personal goals; if they lack focus and commitment to the team…you cannot be agile.

Most people want to be part of a team — preferably a successful team. To get there, begin with the following steps:

1. Define clear goals and objectives for the project.
2. Explain why the goals and objectives are important and to whom.
3. Be sure that every team member understands his or her role.
4. Assign clear individual goals and deliverables (using your definition of  ‘done’).
5. Give everyone a chance to be heard and contribute beyond their area of expertise.
6. Monitor workloads carefully. Seek a balance across the team.
7. Recognize successes with lots of small celebrations.

Good teamwork can make up for many shortcomings in the process and the tools. Good teams will find a way to succeed if you let them.

Lay the groundwork. Get out of the way. And let the team be agile.

Big Organizations Need Your Help to Be Agile

Organizational structure can be a major roadblock to agile adoption. Consider that software development, quality assurance, business development, product marketing, etc. are separate organizations. Then it gets worse.

Even within a functional group, organizational silos evolve. For example, within software development there may be a database group, a java apps group, a .Net apps group, etc. They likely use different development methodologies or variations of waterfall.

Your mission is to get them all to adopt something agile like Scrum. Mission impossible, right?

Almost. Such an organization isn’t going to change quickly or easily. Corporate behaviors tend to be deeply entrenched and highly resistant to change.

How fast you can drive a change agenda depends on the degree of senior management support you receive. The more such support you have, the faster you can go.

Let’s assume you don’t have much support — a common scenario. You likely won’t be able to change too much, too fast. Plan on small steps — incremental changes.

You may have to derive stories from a formal business requirements document. You may have to file weekly progress reports in addition to tracking a burndown chart. You may have to deal with a QA organization that does not want to be part of the dev team.

None of this is ideal but if you can live with it and demonstrate success, you’ll be in a better negotiating position for the next project. The key is to be successful and use success as leverage to drive your agile agenda.

Don’t get discouraged and don’t lose sight of your agile goals.

Keep It Simple and Reduce Risk Via Agile

There are two things that agile software development does really well. Unfortunately, they are widely misunderstood or even ignored.

1. Keep things simple.
2. Manage risk.

Let’s examine these a bit. Simpler is better because people can only remember so much. If you have to refer to the rule book every time a decision needs to be made, you’re doomed. People won’t take the time to refer to the rule book. They’ll wing it. Chaos will result.

I was an advocate for the Rational (now IBM) Unified Process some years ago. There were many things I liked about it though I have always maintained that it was too complex. It presented a process intricate enough to handle the world’s largest software project. That’s fine, but my projects tended to be fairly small forcing me to select the aspects of RUP that I thought were appropriate. Too much work.

RUP was intimidating and hard to master. We need systems that are simple and consistent. The rules should not change from project to project. Everyone should be able to master the rules quickly.

The second point is more complex. Risk is rather nebulous. It’s a bit like buying life insurance. You know there is a risk of dying but it’s hard to relate to it.

Likewise, you know that many traps and pitfalls await any software development team but it’s hard to visualize them until risk turns into consequence. By then, it is too late.

Agile techniques teach us to leverage the experience of the entire team not only individuals. Just as many hands make light work, many minds make better decisions. Better decisions result in less risk.

We also learn to postpone decisions until we need to make them. The goal is to have as much information as possible by the time the decision must be made. Having more information drives better decisions. See a pattern?

We need to admit that we don’t have all the answers up front. And even if we did, the situation will change before all the decisions can be implemented.

Agile development makes the software development process simpler and less risky — at the expense of senior management command and control. Seems like a worthwhile trade-off to me.