8 Lessons in Being Agile from the Open-Source Community

The open-source community can teach us a few things about being agile whether you follow Scrum, Kanban or any other agile approach.

1. Open-source teams tend to be decentralized with a minimal hierarchy. This enables fast decision-making. Multiple levels of approval aren’t needed to get things done.

This level of autonomy can be difficult to accept but the productivity gains make it worthwhile. Let your teams define their own structure and reward system. Define clear guidelines and success parameters then let them go.

2. Open-source teams value intellectual contributions. The people that make the most valued contributions are the ones with the most influence and control. No one just goes along for the ride.

This approach follows the agile principle that results matter more than anything else. People who complain and criticize carry no weight with the team unless they can offer a better idea and help others implement it. Forget rank and job title. Only results matter.

3. Open-source teams collaborate. They use forums, email, instant messaging, and other tools to stay in contact and share information. They don’t write lengthy documents as a means of sharing information.

Resist the corporate call to generate extensive paperwork. Offer the minimal level of documentation that corporate watchdogs require. If they want more, offer them software artifacts instead such as source code, use cases, architecture drawings, etc. Avoid creating new documents just for outside review.

4. Transparency matters to open-source developers. Source code and documentation are readily available for anyone to review. Ideas aren’t hidden or protected. They’re shared.

Provide access to all project documents including source code to any employee that wants it. Don’t hold back. Withholding any information will only invite closer scrutiny and criticism.

5. Open-source teams actively involve end users throughout the development cycle. Early and frequent feedback provides a self-correcting mechanism for the team. Many small corrections are easier to manage and integrate than a few major ones near the end of the project.

Work directly with the user community. Don’t rely on marketing or some other group to act as a liaison. First hand feedback is always the best.

6. Peer review is one of the stalwarts of open-source development. Team members look forward to having their work reviewed by their peers. This is one of the key ways that they learn from each other and develop their skills.

Peer review must be an assessment by other developers not professional reviewers who no longer write software. The team must respect the reviewer for this to work.

7. The open-source focus is on continuous improvement by incremental, not monumental, change. Small, frequent changes allow for rapid development and feedback cycles. Both quality and productivity improve.

Keep project timelines short. If many new features are being demanded, group them into a series of short projects or iterations. Insist on user feedback after each iteration.

8. Time is not rigidly controlled on open-source projects. Quality and stability are valued more than getting the next major release done by a predefined date.

This works because there are multiple mini-releases of the software along the way. The major defects are corrected and many people install the application before the final release is delivered.

Energize your teams by challenging them to produce results not documents. Give your teams the tools they need to be successful. And most importantly, get out of their way and let them succeed.

Best Practices Are Guidelines Not Absolutes

We hear a lot about “best practices”. They apply to every endeavor from home repairs to business strategies. The phrase is also often used when examining agile software development approaches such as Kanban, Lean, Scrum and XP.

What are best practices? They’re really just methods, processes, rules or techniques that have been successful in certain situations. Once a practice delivers a successful outcome, it’s immediately a candidate for “best practice” status. Multiple successful outcomes make it appear to be a sure-fire approach.

Do best practices represent the best we can do?

Of course not. A best practice, even if it’s optimal, is merely a snapshot in time. It’s the best we can do today, given the present situation. It says nothing about tomorrow.

There is another downside to the idea of best practices. They often serve as a crutch; a kind of safe haven. Because a practice is generally accepted as a best practice, it must be the right thing to do — a no-brainer. Thusly, there is no need to analyze and assess the present situation. All you need to do is adopt best practices.

Take daily standup meetings, for example. These represent the most widely accepted element in any form of agile software development. But, are they an essential characteristic of being agile — a best practice?

Consider these two divergent situations.

Team One consists of senior developers and testers. They have been through many agile projects together and work well as a unit. Do they need to meet everyday? It won’t hurt but it’s likely to be unnecessary. The team should decide what works best for them. Perhaps meeting 2-3 times a week will suffice. Or, they might decide to meet weekly for longer than the prescribed 15 minutes.

Team Two consists of a senior-level contributor along with a staff of mostly junior members. This is the first agile project for most of them. Do they need to meet everyday? Yes! In this situation, the best practice recommendation of daily meetings should be followed. A junior team needs additional oversight and more opportunities to self-correct.

Any particular best practice may not be suitable to your situation. If you conclude that a best practice cannot work or will be too difficult to implement, ask yourself how the practice can be adjusted to make it work for your team.

Managing Expectations Leads to Successful Outcomes

I’ve written about context (Without Context, You Can’t Make an Informed Decision) and how important it is when making a point. For example, if your work involves IT consulting to the U.S. Government, your context is completely different from someone working in a software start-up targeting consumers. The ground rules are totally different.

In a similar vein, setting expectations is critically important. Few of us manage them well. Set them too high and you’ll likely disappoint your customers, even if you deliver as specified.

Software development teams, regardless of approach (Kanban, Lean, Scrum, Waterfall, etc.), spend lots of time on business requirements, stories, minimum viable products, and other ways of defining what the business needs. While these are good concepts, they often mean almost nothing to our stakeholders and end users.

Expectations mean much more.

Consider that expectations go far deeper than requirements. Expectations have an emotional component. They represent a kind of vision or mental image of a future state. When system changes are introduced, they will be mentally compared to expectations. If the changes align to expectations, you have satisfied customers. If not, you don’t (even if you delivered on the requirements as specified).

Most people don’t know what to expect from a new or revised software system. We need to educate them in terms they can relate to. For example, telling a group that the software will run inside Safari on the Apple iPad will set high expectations with some people and mean nothing to others. Keep it simple and use examples wherever possible.

Expectations are usually set by a combination of factors. The requirements, stories, etc. play a role, but often a secondary role. Why? Simply because they tend to be too long and complex for most people to grasp.

That presentation given to the business stakeholders may have had the biggest impact on expectations. The words used and the enthusiasm in the presenter’s voice may have set expectations higher than intended.

Listen and learn.

To gauge expectations, it’s important to listen carefully to what the stakeholders and customers say. Their comments help us understand the level of excitement they have. Is it too much? Their questions can help us direct their expectations appropriately by offering appropriate answers.

People are more likely to let us manage their expectations once they get to know and trust us. Would you put your faith in a complete stranger? Get to know your key stakeholders and power users. Be open and build credibility.

Remember that setting expectations is not a one time exercise. We need to monitor and redirect them during the course of every project as new information arrives. Try spending a little more time managing expectations. You might have more successful outcomes.

Adding Managers to a Late Project Only Makes It Later

I’ll bet it’s happened to you. You’re working on a software development project that runs into trouble. The project could be on a long, slow decline or some unexpected event may have blindsided the team. Either way, the project is late and your team is in trouble. Now it gets worse.

The news spreads up the management chain like an explosion at the bottom of an elevator shaft. Middle, senior and even executive managers are eager to solve the problem. Have you ever heard the phrase, “I’m from corporate and I’m here to help.”? Just what you need.

I’m not suggesting that management intervention is always bad. In fact, I’ve witnessed many good managers help teams by clearing technical roadblocks and administrative obstacles. These actions enable the team to focus on solving the problem(s) and not get slowed down by ancillary tasks.

Unfortunately, I’ve also witnessed many poor managers. They schedule weekly or daily status meetings that only take up valuable time. Or, they do nothing more than demand weekly or daily status reports. Not only does the team need to solve the problem(s), they also need to allocate time for preparing status updates. These steps simply apply pressure, create more work, and cause anxiety.

When this happens to one of your projects (and, eventually, it surely will), here are a few agile ideas to help you get through it:

1. Establish control. Your management team is not going away. If they sense that the project or the problem is out of control, they will establish a command-and-control structure along with rules and procedures you’ll have to follow. Don’t let that happen.

It’s important for you and the team to define the steps toward a solution such as:

• Understand the problem and assess the situation.
• Explore courses of action and decide on an approach.
• Implement a sequence of actions to solve the problem.

You get the idea. The details will vary with the situation but the general technique is the same.

2. Ask for help. Your management team wants to help but they may not know what to do. Be clear about what the team needs and what your management team can do to assist.

• Do you have the right tools and equipment? If not, ask for them.
• Do you have the right people? If not, clearly state what skills are needed.
• Are there normal business processes that you’d like to circumvent temporarily? If so, lay it out.

If the problem is relatively small and localized, you may not need anything and even if you do, you may not get it. If the problem is large and widespread, be assertive. Get what you need.

3. Assign a team representative. Have one person report status for the team. Make it clear that the rest of the team will be focused on solving the problem and cannot be disturbed.

• The spokesperson should be a senior member of the team or the team’s manager.
• Daily (internal) team meetings (like Scrum standups) are likely to be a good idea.
• A team status board (like a Scrum board) showing tasks, assignments and current state may also help.

Be sure that the spokesperson communicates openly and honestly. There is a tendency to sugar-coat management reports. Don’t. Tell it like it is and maintain control.

There is no better time to be agile than during a time of crisis. Do you have any ideas to add?

Applying Agile Development to the Real World

You may have heard about Everett Rogers’ book, “Diffusion of Innovations”, Geoffrey A. Moore’s book, “Crossing the Chasm”, or Eric Ries’ concept of the Minimum Viable Product (MVP). They each present intriguing ideas that fit nicely into developing software using an agile approach.

Take a look at the graphs shown here. They come from WikiPedia and show groups of people adopting a new idea, product, or technology (blue line) and the growing market penetration over time (yellow line). The groups are loosely described as follows:

Innovators – They have a high-risk tolerance. They tend to be younger and better educated. They also tend to have higher incomes.

Early Adopters – These folks are like Innovators except that they tend to have a lower risk tolerance and they are viewed as social leaders.

Early Majority – These folks tend to be more deliberate and thoughtful. They are socially active but not viewed as social leaders.

Late Majority – These are the skeptics. They wait and see. Their socio-economic status is usually lower but their group size makes them a force to be recognized.

Laggards – These traditionalists are usually the last to adopt. They tend to be older, risk-averse, and reluctant to spend money.

(Bear in mind that these are generalizations and there will be exceptions to these rules. For a detailed analysis of these groups, you should read the books.)

These groupings make agile development simpler.

Having the target users grouped, enables the team to focus on stories (or requirements, features, use cases, etc.) that benefit each group. In the simplest case, the development team can plan five major releases aligned with the five groups above.

This approach helps with prioritizing stories and obtaining valuable user feedback sooner rather than later.

This also works when the target audience consists of internal workgroups. There will always be Innovators, Early adopters, etc. It helps to identify workers in each group and deliver functional software to them as appropriate.

You’ll target the Innovators first. They tend to be less demanding and more forgiving — just don’t abuse them. As the software matures and becomes more feature-rich, work your way through the other adopter groups.

This concept fits well with the idea of giving your users personas. Get to know the personas and assign them to adopter groups. Prioritize and order user stories based on the groups. Make sense?

How Not to Adopt Enterprise Agile Development

Let’s say that your enterprise decides to give agile software development a try. They select Scrum as the preferred agile approach (though it could also be Kanban, Lean or XP). They decide to try a pilot project rather than changing the entire company at once.

A pilot project is selected; not just any project but one that seems most likely to succeed using Scrum. A team is hand picked and assembled to work on the project. Similarly, the team consists of people most likely to succeed using Scrum.

Do you see where this is going?

The project will be successful by definition. The team will be nurtured, obstacles will be cleared, and acceptance criteria will be adjusted as needed. The company concludes that Scrum really works! But does it? Would any software development approach have worked just as well under such circumstances?

Now things will start to go terribly wrong. The company decides that because the pilot was so successful they should roll out Scrum more broadly. Of course, they will need to make some adjustments so that Scrum fits the way the company does business. They decide to implement Scrum — but — tailor it to their needs. (In other words, make Scrum more like waterfall.)

They define a hybrid development approach. For example, teams will create business requirements documents, functional specifications, class diagrams, and other development artifacts. Then, they’ll use Scrum to write and unit test the code followed by a formal software quality assurance period at the end — water-scrum-but-fall — the worst of all worlds.

Don’t stack the deck.

Adopting Scrum (or any agile approach) successfully, demands organizational and cultural change. These types of fundamental transitions are very hard to achieve. Success depends on active participation by senior and executive managers. Agile development really should be applied to a software product line or software-based service, not to a single project.

Trying to force agile from the bottom up won’t work. There will be formidable resistance to change. #fail

Modifying agile to make it fit the environment won’t work either. You’ll end up with a hybrid development approach that no one will be able to fully understand and follow. #fail

Enterprise agile is a mindset. It takes commitment and dedication. Training, coaching and mentoring will also help. Know what you are getting into and do it right — or don’t bother.

Even Simple Software Applications Can Be Complicated

If I asked you to write a simple software application to calculate loan payments and remaining balances over time, how would you approach it? Here are a few possibilities:

1. Ask questions to clarify the requirements and document the answers before continuing.
2. Write a formal requirements document and obtain sign off before continuing.
3. Produce a user interface mockup or wireframe and use it as a way to clarify the requirements.
4. Develop a software prototype and use it to finalize the requirements and the design.
5. Locate a few existing loan payment applications and use them as examples to define detailed requirements.
6. Write the application as you believe it needs to be done and deliver the final version.

How would you approach the problem?

None of these approaches is necessarily better or worse than any other — as long as the customer is happy with the end result. There are, of course, several complications. Did I mention…

• …the software must handle balloon payment loans.
• …the software must handle interest-only payments during the first few years of the loan.
• …the software must export XLS or CSV files.

 Would these complications make you approach the solution differently?

#1 works well if you know what questions to ask. Do you?

#2 isn’t much better than #1.

#3 might work well … if … the mockup or wireframe is detailed enough.

#4 will ferret out the details though it may take several iterations.

#5 may help enormously … if … you can find existing applications that handle most of the issues.

#6 will only work if you are intimately familiar with the customer and the solution space.

I prefer approaches that are agile and offer multiple opportunities for customer feedback. Thus, #3 and #4 are most appealing to me. What would you do?

10 Reasons Why Projects Cascade Into Chaos

Projects fail for many reasons. Sometimes it seems that the team can do everything right, or at least to the best of their ability, and failure occurs anyway. There is simply no way to guarantee success but there are some key indicators that can alert you to impending failure.

1. Lack of Sponsorship

Executive sponsorship is critical to the success of every major project. Without it, the team won’t be adequately funded or staffed. If your management team is not solidly behind you, drop the project and find something else to do.

2. Poor or Optimistic Planning

“Poor planning” — how many times have you heard that? Is it more or less than “overly optimistic”? The most frustrating part of this problem is that companies make these mistakes over and over again. If you recognize poor or optimistic planning, speak up but don’t criticize. Explain where the plan is lacking and recommend ways to improve it.

3. Failing to Prioritize Requirements

I’ve found that many stakeholders are reluctant to prioritize requirements. I think they fear that if a feature doesn’t make the final target release, it will never be seen or heard from again. Fixing this problem demands that we instill confidence in the stakeholders. They must believe that we will say what we do and do what we say. Solve the credibility problem and this issue will go away.

4. Weak or Wrong Personnel

Companies often assign whoever is available to a project. Of course, that person is usually working on a few other projects already. Big mistake. A good developer assigned to the wrong situation will slow everyone down. If this must be done, be sure to plan accordingly — leave extra time and invest in training or coaching.

5. Getting Off to a Slow Start

I’ve seen many projects get the green light only to meander aimlessly for a few weeks, or even months — under staffed, under funded, and under prioritized. Eventually, someone decides that the project is important. Great! But now the team is so far behind schedule that failure is assured. Don’t commit to a project until everyone is ready to focus on it.

6. Inadequate Design

This can stem from poor planning or weak personnel. Either way, the team gets far along in the project only to discover that the software design cannot handle some aspect of the solution. Code needs to be tossed out and re-implemented. (This isn’t re-factoring. This is re-designing.) It’s another problem that can’t be completely avoided but it’s always a good idea to tackle the tough implementation issues in early iterations.

7. Cutting QA Time and Effort

When time is tight, and it always is, quality assurance testing and other quality-driven activities are the first to get cut. It’s far better to deliver less, high-quality functionality than more of the low-quality variety. Think about it. Less is more.

8. Creeping Scope

During any project, items will arise that weren’t built into the schedule. Because we all want to be helpful (and liked), we often agree to “see what we can do”. It’s a good idea to build some extra time into the schedule to handle ad-hoc requests. However, if a request is more than a few hours of work, it belongs as a line item on the schedule or backlog.

9. Cutting Corners Under Pressure

When a project runs into trouble, there is a strong tendency to cut corners so as to get back on track. Those shortcuts result in technical debt that will need to be repaid, if you’re lucky. If you’re not so lucky, the shortcuts will cause even more problems and the project will cascade into chaos. Do what you have to do in meeting commitments. Just be smart and keep it visible.

10. Insufficient Risk Management

Too many projects ignore risk management. Ignorance is bliss, right? Why tempt fate? If something goes wrong, you’ll deal with it when it happens. While you can’t plan for every possible contingency, you have to plan for some disruption. If your plan assumes that the planets will align perfectly, you’re already late and over-budget.

If Your Users Lack Personas, You Might Be a Drug Dealer

You’ve likely seen the line that says ‘Only drug dealers and software developers call their customers users’. It’s funny but also carries an important message.

You see, drug dealers don’t care about their customers. All they care about is the money. Some software developers exhibit a similar characteristic in that all they care about is the technology.

Creating really good software requires more than good technology, minimal defects, and decent performance. The software has to behave in a manner consistent with the person running it; let’s call him or her the ‘client’. The only way to achieve such a result is to get inside the head of the client.

That’s where personas fit in.

Whether you’re doing waterfall or agile software development, User Stories and Use Cases are great tools for elaborating on client activities and behaviors. They allow us to describe client actions in ways that software developers can dissect and turn into executable code. But there’s a step missing!

We need to know a few things about our clients before we dive into their behaviors. Here’s a simple example. Let’s say I have the following user story:

As a user, I want to send Twitter messages so that I can participate in its global community.

Simple right? Now what if I told you that the ‘user’ has severe arthritis and cannot type or use a touchscreen? That changes everything.

Here’s an even simpler example:

As a user, I want to generate an inventory aging report and export it to Excel so that I can manage inventory levels and costs.

You might think that this report will be run monthly by a finance staff member. What if I told you that the user is a highly-impatient, senior executive who runs this report hourly and uses it to adjust product pricing dynamically?

Get the point? It’s critically important to know your clients.

There are many aspects to knowing a client. Here are a few ideas to ponder. Determine which are important to your situation and create personas around them.

• How much on-the-job experience does the client have? Or, how savvy is she as a consumer or buyer?
• What are the client’s job responsibilities? Or, where does he fit within his household?
• What does a typical day (or evening) look like for the client?
• What is the client’s attitude toward technology and computers?
• What tasks does the client perform that are the most critical or repetitive?
• What is the client’s personality like (e.g. mellow and even-tempered versus impatient and hot-headed)?
• Does the client sit still and complete a task in one place or is the client highly mobile, starting and pausing tasks often?
• Does the client have any physical limitations affecting mobility, sight, hearing, etc.?
• Is English the preferred language of the client?
• Does the client abide by the same rules and regulations as other clients (e.g. state and national laws vary widely)?

You get the idea. Develop a small number of client personas for your next project. Give them names so the software development team can relate to them. How many you will need depends upon the nature and complexity of your software.

When you write user stories or use cases, direct them at one of the personas you’ve defined. Then go one step further. Apply a different persona and ask the team, “How does this change the client behaviors?”.

How to Capture Non-Functional Requirements

Non-Functional Requirements (NFRs) can be difficult to define and track. We often treat them as after-thoughts and only near the end of the project.

We usually focus on functional requirements. These represent system behaviors. They define what a system is supposed to do and how the inputs and outputs will appear.

NFRs are more varied. They define characteristics of systems rather than behaviors. Here are a few of the more common types of NFRs:

• Availability (e.g. 24x7x365)
• Capacity (planned)
• Compliance (legal and/or regulatory)
• Efficiency (resource consumption)
• Interoperability (inter-system data exchanges)
• Performance (response time)
• Portability (cross-system functionality)
• Resource usage (processor, memory, disk space, network bandwidth, etc.)
• Scalability (user count and data volume growth)
• Security (e.g. login, encryption)

Traditional business requirements documents have a section devoted to NFRs. They may even be captured in a separate document. Either way, it can be a bit awkward in that the business often doesn’t specify NFRs. They tend to focus on what a system must do making the implied assumption that the system will be able to handle the workload thrown at it in a secure and robust manner.

Agile software development teams often have a difficult time with NFRs too. How should they be captured? What do they look like? Here are a few ideas.

Capture NFRs as user stories. While most NFRs don’t fit nicely into the story metaphor, capturing them this way makes them highly visible and easy to manage. They are just like any other requirement. An NFR story might read something like…

As an impatient and over-worked data analyst, I want to process 10,000 records in under 60 seconds so I can get more done in less time.

As a claims adjuster handling sensitive information, I want my data to be password protected and encrypted so that I meet compliance obligations.

As a marketing executive, I want the software to operate in Windows, OSX and Ubuntu so that the company maximizes its revenue potential.

Capture NFRs as technical stories. These stories are different from user stories in that they don’t originate from the business community. Thus, they don’t have the visibility and discussion associated with user stories. There is nothing wrong with this approach though I’d argue that sacrificing visibility and discussion does not lead to positive outcomes.

Capture NFRs in the acceptance criteria for each story. This is a little more work in that it forces the team to think about NFRs while implementing each story. This may make some of the acceptance criteria repetitive. In addition, some NFRs may be difficult to test story by story. It may be best to only reference an NFR in the acceptance criteria if it is essential to the implementation of the story.

Write a Non-Functional Requirements Specification. Such a document would serve as an umbrella or wrapper over the project. This could work but using two methods for capturing requirements might be confusing. There will always be gray areas. For example, if I want to run the software on a touchscreen, is that functional or non-functional? It’s both because of the many issues and nuances introduced.

There really isn’t a right-way or wrong-way to define NFRs. Pick an approach or a combination of approaches that works for your team. The most important point is conditioning the software development team and the business community to recognize the importance of NFRs and document them.