An Agile Dilemma – Stop Talking. Start Delivering.

A challenge awaits any IT group switching from a waterfall process to agile software development. It is often overlooked. It has to do with the mindset of the company’s business community and how they perceive IT.

The business stakeholders and end users have historically complained about issues like the following:

• It’s difficult to get IT to focus attention on their problems.
• Once they have IT’s attention, it takes a long time to get new or updated software delivered.
• IT won’t deliver everything they request, so they have to request everything they can think of.
• Once IT delivers (if they deliver), it will be a long time before they see any updates.

Ouch! How stinging is that! (Yes, these are real complaints I’ve heard over the years at multiple companies.) Does it describe your IT shop?

These complaints are a direct result of the waterfall process coupled with change control (better described as change avoidance). Waterfall approaches don’t always take longer than agile ones but it feels that way. There’s long lead time between user requests and tangible software. It feels like an eternity.

An Agile Dilemma

Now, let’s say IT decides to switch to some variation of Scrum. They seek collaboration with the business community. What would you expect them to encounter? Skepticism, right?

Scrum offers solutions to the above complaints in the form of:

• A dedicated software development team
• Frequent software deliveries
• Incremental improvements until all requests are filled
• Change acceptance

If you were on the receiving end of these promises, what would you think? How would you react? (I know I’d be extremely skeptical.)

Here’s my suggestion. Actions speak louder. You already have a backlog of user requests, right? There must be plenty of requirements that didn’t make the final list or were dropped during the project. There must also be defects that didn’t get fixed or weren’t fixed to the satisfaction of the users. Start with all these backlog items and software defects.

Get to work. Fix something. Add something. Re-work something. Just do something. Deliver it. Now repeat!

Engage the business stakeholders and end users in productive conversations about each delivery. Explain to them that this is how agile development works. It’s a whole new approach. You’re showing them, not just telling them. They’ll get the idea and become engaged in the new process almost immediately.

Stop talking. Start delivering.

Retrospectives Work When They Are Impersonal and Impartial

Retrospectives are a terrific way to continuously improve any software development process. As long as the organization is committed to improving itself, rather than protecting the status quo, there is nothing better than regular retrospectives.

So why are project retrospectives often ignored or under used?

There are many articles across the Web describing how to do retrospectives. They are not hard, though there is a serious pitfall that can derail the best of intentions. Retrospectives must be impersonal and impartial.

That’s harder than it sounds. It seems that many of us are conditioned to place blame. At least here in the U.S.A., we see plenty of people not taking responsibility for their actions. They want to blame someone else every time there’s a problem.

We also tend to easily become defensive. Even if a problem is being discussed in general terms and no names are mentioned, some people will jump to the conclusion that they are being targeted. This usually prompts them to retaliate. The session can turn ugly, quickly.

If anyone comes out of a retrospective feeling beat up, picked on, or abused, the event was a failure. Even worse, no one will want to attend future retrospectives out of fear that they too will be abused.

Let’s examine a few situations that cause such behaviors and how you might handle them.

1. There is one person on the team that is not carrying his weight. He is letting down the team and causing extra work for others. Is this a retrospective topic? No, of course not. This is a personnel issue and should be discussed with the team member in private.
2. One of the processes followed by the team isn’t working. It turns out that the process is always executed by the same team member but she is doing it faithfully as defined. The process needs to change. Retrospective topic? Yes, but! The issue must be discussed with the team member in advance. Explain that the process is flawed and ask for suggestions that can be aired at the meeting.
3. There is a process followed by most members of the team. But, a couple of folks take shortcuts resulting in unintended consequences. Retrospective topic? Maybe. There are two ways to handle this. a) Speak privately to those taking shortcuts and get them to stop; or b) modify the process during a retrospective so it works for everyone.
4. The Product Owner or Scrum Master is not fulfilling the requirements of the job or meeting the needs of the team. Retrospective topic? No! Again, these are personnel issues and should be discussed in private. If the PO or SM wants to seek feedback from the team, that’s fine, but only after a private discussion.
5. An end user, customer or client complains constantly and is difficult to handle. Retrospective topic? Yes, indirectly. The PO should engage the accused party in a conversation to uncover the real issue(s) and explore solutions. Meanwhile, the team can discuss how to better meet the needs of all people using the software.

The key points to take away are:

• No surprises. A retrospective is not an ambush.
• No confrontations. A retrospective is not a street fight.

Always keep Norm Kerth’s Retrospective Prime Directive in mind.

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.

Walking into a retrospective and becoming the target of someone’s complaints is not fun. Retrospectives are for continuous improvement of the software development process. People improvement is valid topic as long as the improvements apply to everyone. Individual performance is off limits. Take it outside.

What Are the Characteristics of Successful Software Projects?

We spend a lot of time examining software project failures but it’s equally important to understand why some projects succeed. Here’s a short list.

1. Commitment. The business stakeholders and the technologists are committed to the project — not just at the outset but throughout. This takes intense collaboration. There is also an executive sponsor or champion — someone with the authority to make things happen. (If there is a client-consultant relationship, this is doubly important.)
2. People. The right team is assembled. There is an appropriate mix of senior, mid-level and junior people. The skill sets are also mixed and complimentary. Everyone understand their role and how it fits into the project. Most importantly, there is a core group of exceptional people who are capable of leading the project both administratively and technically.
3. Goals. The project has clear goals that everyone understands and accepts. This includes the critical dates that the team has to hit. The scope of the project is narrow enough for everyone to comprehend and embrace yet wide enough to deliver value to the business. The constraints placed on the project are reasonable and realistic.
4. Communication. Frequent and open communication is encouraged. Everyone is willing to share information and thus everyone knows what’s going on. Whenever the team reaches a milestone or achieves a major successful outcome, everyone celebrates.
5. Focus. The team is focused on getting the project done. They are not distracted by cultural, hierarchical and bureaucratic barriers. They use informal contacts and relationships to make things happen.
6. Learning. Everyone has the opportunity to learn and grow during the course of the project. They are encouraged to test and experiment. When mistakes are made, they are leveraged as learning opportunities.
7. Change. The team deals with change effectively. That means they don’t try to block change but they don’t throw the doors wide open and allow anything to change any time. They find a middle ground and accept change as an opportunity to learn and improve the final result.
8. Environment. The team has the right environment for getting the job done. This covers everything from office space to desks and chairs to software development tools.

Every situation is unique, of course, but these traits are common to most successful software projects whether they follow a prescriptive approach like waterfall or an agile one like Scrum.

As a Product Owner, I Want to Avoid Writing Stories So That I Can Do My Job

Writing user stories for agile software development can be a challenge for any Product Owner. Stories are supposed to deliver business value. What’s that? Stories are supposed to be vertical slices. Slices of what exactly?

At times, we get overly bogged down in words and their definitions. We take things too literally and get stuck. User stories are a simple concept. Let’s keep it that way. Take this example.

Sort a Random Collection of Data

Let’s say your software application gathers an assortment of textual data (let’s call it motley data) that might be interesting to end users based on some specified criteria. It works and the users have asked for the ability to sort the data.

As a data analyst, I want to sort the motley data so that I can more readily find items of interest to me.

Sounds simple, right? Not so fast. Let’s say the data consists of text, numbers and dates. (Forget pictures, sound and video. That would be over-the-top difficult.) Here are a few problems to ponder.

1. Text – Should upper and lower case letters be sorted together or separately?
2. Numbers – Are 01 and 1 the same number? What about 0.56 and .56? Should numbers be sorted before, after or along with text?
3. Dates – Should 6/1/12, 06/01/2012 and June 1, 2012 be sorted as the same value?

What appeared to be a simple sort algorithm could become multiple complex algorithms — far too much to be considered a single user story. In addition, the user interface needed to control the sort will present its own challenges.

Many agile teams would be tempted to break out the UI as a story, perhaps using it as a way to drive the implementation. But, a UI that does nothing has no business value. If the UI is implemented but only a few things actually work, it will confuse and frustrate the users. If algorithms are implemented without a UI, how will the users control the sorts?

See that! Stories don’t work! Not everything of value can be delivered within a sprint!

Hold on. The original user story grew. It became an epic. Epics consist of multiple stories. Our goal is to deliver complete stories within each sprint. Epics may take several sprints. We can do this.

Here’s one way to proceed (not the only way, but this could work):

Define several user stories that will make up the epic. For example,

Epic: As a data analyst, I want to sort the motley data so that I can more readily find items of interest to me.

Story: As a data analyst, I want to define criteria for sorting the motley data so that I can more readily find TEXT of interest to me.

Story: As a data analyst, I want to sort the motley data so that I can more readily find TEXT of interest to me.

Story: As a data analyst, I want to define criteria for sorting the motley data so that I can more readily find NUMBERS of interest to me.

Story: As a data analyst, I want to sort the motley data so that I can more readily find NUMBERS of interest to me.

Story: As a data analyst, I want to define criteria for sorting the motley data so that I can more readily find DATES of interest to me.

Story: As a data analyst, I want to sort the motley data so that I can more readily find DATES of interest to me.

Now we have six user stories. From an implementation perspective, three focus on the UI and three focus on the sort algorithms. From a functional perspective, there are three pairs of stories and each pair focuses on a data type. (You might choose to break this up differently and/or add more stories. That’s okay too.)

Depending on the size of the team and other factors, these six stories could be implemented in one, two or three sprints. Each sprint would deliver real business functions (a.k.a. value).

Also note that you might decide to build out the entire UX in the first sprint. If so, simply hide UI elements that are not yet functional. You can expose those elements as the underlying algorithms mature.

Bottom line: Think of stories as descriptions of software features that taken together as epics provide meaningful business functionality. Don’t get too caught up in the individual stories. These will often be too small to be meaningful. Also, don’t just think in terms of sprints. Consider using releases as the delivery vehicles of tangible business value.

10 Tips for Enhancing Waterfall Software Development

So your company uses a waterfall approach to software development and you’re stuck with it. Your management doesn’t understand agile development and doesn’t want to. They like how work gets done even if projects often take longer than they’d like. Now what?

Other than finding another job, your options are limited. Here are ten suggestions for enhancing standard waterfall development making it less bureaucratic and more fun.

1. Iterate. Instead of a single design-code-test cycle, introduce multiple cycles. These are essentially mini-waterfalls, not agile sprints, but they still allow you to get partially completed software into the hands of real users more often. The feedback will be priceless.
2. Accelerate. Use Rapid Application Design sessions. Kick-start your projects in a few days, not weeks. Assemble a multi-disciplined team to plan and estimate quickly. Rough out scope, schedule and architecture. Don’t seek perfection — just give it your best shot. Repeat as needed in future cycles. The team should include both business and technical people with in-depth experience in the target environments.
3. Visualize. Use walls, whiteboards, flip-charts, projectors and even sticky notes to help everyone comprehend the problem and solution spaces. Timelines, assignments, risks, features, open issues, etc. are candidates for visual communications. Allocate a dedicated Team Room (or Project Room) for this purpose.
4. Face it. Promote and encourage face-to-face interaction. Email should be used to document results not raise and resolve issues. When someone asks you a question via email (and copies everyone they can think of), walk down the hall and answer the question. Then, ask the person to email the answer to everyone copied on the original message.
5. Participate. Set an expectation of continuous stakeholder participation. Establish participation guidelines. In the Team Room, make it clearly visible who is actively participating and who is not.
6. Automate. Invest in automated test development. Include SQA in planning and analysis meetings. Give SQA a voice in establishing the timeline and quality metrics. Software quality will improve markedly.
7. Demonstrate. Show the software to key stakeholders and end users early and often. It’s the best way to uncover implied assumptions and hidden risk factors. There’s no substitute for timely, targeted feedback.
8. Test. Start SQA testing early and don’t stop. The sooner the developers get detailed feedback on the quality of the code they’ve written, the better. Bugs will get fixed faster.
9. Learn. Invest in continuous learning and training for the entire team. Software tools and technologies, development and business processes, people skills, etc. need continual reinforcement and advancement.
10. Finish. Develop a good ‘definition of done’. This rule applies as much to waterfall teams as agile ones. Everyone needs to know what it means to be ‘done’. Hold them to it.

Doing all of the above won’t convert your software development team into an agile one but you’ll see improvements. Pick a few items on the list and get going. Adopt more over time if you can. These tips will enhance the waterfall development process you’re using and result in better software.

This Test Will Help You Determine How Agile You Really Are

We sometimes talk (or write) as if agile software development and waterfall software development are two completely separate and different ways of building software. Yet, in reality they have much in common and it can be hard to slap a label on the process followed by any particular team.

With that in mind, here are seven key criteria that are often different among software development teams. Score your team in these areas and see where you end up. It will help determine if the process you’re using is more agile-like or waterfall-like.

The score for each criteria runs from one to ten where one is waterfall and ten is agile. Most answers will be likely be in-between the end points. That’s okay. No team is likely to be a perfect 7 or a perfect 70. Be honest.

At the end you’ll find a range of scores to help you determine how your team stacks up.

1. Requirements and Scope Definition

1 – Requirements and scope are well-defined and stable. Stakeholders and end users communicate them clearly and in writing.

10 – Requirements are poorly defined, uncertain and/or subject to change at any time.

2. Stakeholder and End User Involvement

 1 – Stakeholder and end user involvement are intermittent.

10 – Stakeholders and end users are available when needed.

3. Team and Process Experience

 1 – Team, technology and business process are mature. Project methodology is established and documented.

10 – Team likes new challenges and embraces change. The technology or business process is new and unproven.

4. Team Communication

 1 – Information exchange is formal and in writing.

10 – Information exchange is informal, open, and usually face-to-face.

5. Team Structure

 1 – Departmental hierarchies are important and management is top-down.

10 – Teams are cross-functional and self-organizing.

6. Planning Approach

 1 – Project planning is front-end loaded and detailed.

10 – Project planning is spread throughout the project and adjusted as needed.

7. Process Governance

 1 – Development process is structured, complex and audited.

10 – Development process is fluid, analytical and adaptive.

Scoring Summary

• 56-70: It’s an Agile approach.
• 42-55: It’s an Agile approach that needs work.
• 28-41: It’s mostly a Waterfall approach with some improvements.
•  7-27: It’s a Waterfall approach.

Remember, this is a single data sample and a simple one at that. I’m sure some will disagree with one or more of the criteria I’ve chosen or wonder why I left something out. These are the criteria that I view as most important. If you’d like to offer an additional item or change something above, your constructive comments are welcome.

There Are Only Five Software Risks Worth Considering

Agile software development teams face many types of risks but only a few really matter.

There are countless risk possibilities to consider on any software development project. Everything from under-estimating the project scope to the local nuclear power plant suffering a meltdown. It can be mind-boggling.

I think many teams don’t even bother to assess risks because they feel that risks fall into one of two categories. They are either self-evident or ambiguous. Here are a few examples:

Self-Evident Risks

• Inadequate test coverage
• Staffing issues
• Unforeseen problems with tools and technologies

Ambiguous Risks

• Fire in the data center
• Reductions in the project budget or staffing level
• Unexpected competitive announcements

The self-evident and ambiguous risks are probably not worth spending time on — unless the team is inexperienced or a particular risk factor has a high probability of impacting the project.

There are five risk factors worth discussing among the software developers and the business stakeholders. Every project is bitten by at least of one the following:

1. Missing Stories – Some “Must have” stories may be missing in the backlog because no one has thought of them yet. When they do, a scramble will ensue.
2. Under-Estimated Stories – Some epics/stories in the backlog may require more work than estimated. This may be offset by other stories that were over-estimated but there’s no way to know until implementation time.
3. Under-Valued Stories – Some lower priority stories may actually be “Must haves”. This is common. As the software takes form and word gets out, more interested parties will emerge and priorities may change.
4. Not-”Done” Stories – Some stories thought to be “Done” may not be. This can happen to anyone particularly if a story is unusual in some way. The standard “definition of done” may not cover every situation.
5. Lack of Participation – Stakeholders and end users may not be engaged. This is also a common problem. It results in a flurry of changes late in the project increasing tensions and putting commitments at risk.

At a minimum, always do your best to document, discuss, and mitigate these five risks. If you don’t consider any other risk factors, you’ll at least have the major gotchas covered.

If you’re aware of other commonly-occurring risk factors that may be peculiar to your organization, cover those as well. For example, software consulting firms usually experience a number of client risk factors from failing to approve deliverables to not paying bills on time. These usually fall into the category of business risks but can still wreak havoc with your project.

The Future Is Here and Living in Your Smartphone

Apple sent out a few loud and clear messages at their Worldwide Developers Conference this week. They announced some great new products and the media is fixated on them. But the products are not the message. The capabilities of those products and the future direction they establish are what’s interesting.

Here’s my interpretation of the announcements.

1. Social Media technologies are the future. You can poke fun at Twitter, Facebook, Google+, LinkedIn and all the other social media services. But, there’s no denying that social media capabilities are in demand and growing fast.

I don’t know if companies like Twitter and Facebook will survive long-term but I know that the foundations they’re pouring will endure. People want to be connected. We want to share, be informed and meet other people.

What does this mean for your company? Are you using any social media tools for business? Can your employees openly share information? Can they subscribe to internal information threads that interest them?

Here’s an idea: Stop sending spam (a.k.a. email messages) to your employees with all kinds of mundane corporate announcements. Instead, let them subscribe to information channels and get the information they want. Once they get a taste of how convenient this is, they’ll come up with all kinds of ideas for improving productivity.

2. Geolocation services are a big thing. We want to know exactly where we are in relation to places and people we know. We want to know how to get where we’re going simply and safely.

Obvious, right? How long have humans been using maps? Over 12,000 years.

Think about what this means for your company. If you have many people that travel, you need to provide them with geolocation tools. Whether they travel locally or around the globe, helping them travel safely and quickly can be a big win.

How many buildings does your company occupy? How big are those structures? Do people often visit multiple facilities? Forget paper maps. Distribute all your facility maps electronically. You can even track equipment and people (with their approval) inside your building to help them get around and find others. Think of the efficiency improvements.

3. Voice input has come of age. Our computer systems have become too complex. Even the diminutive smartphone has become so complex that many of us have trouble understanding all the features. Even when we do, finding apps and information becomes a huge hurdle amidst all the messages and icons.

The solution doesn’t lie in improved point-and-click methods — been there, done that. The solution lies in a combination of voice input and semantic search technology. Simply ask the system a question and it will either provide the answer or give you a list of possible answers.

If your company develops software, even if only for internal use, you need to be thinking about voice input and semantic search. Too many hours are being wasted simply because your employees can’t find what they’re looking for. No amount of training will solve the problem. Training only reduces the aggravation level.

Being agile is more than a set of principles or even a mindset.

People need to feel empowered and they need tools that help them feel connected and in control. The technologies mentioned here are changing the way we work and play. Apple gets it. Does your company?

Agile Development Teams Follow This Simple Principle

Have you ever heard of the principle known as Occam’s Razor (or Ockham’s Razor)? I like to think of it as the law of simplicity — simpler explanations are better — all things being equal. Another way to think about it is that you should always start with what you know in trying to understand a situation, outcome or behavior. Only add complexity when you conclude that your simple approach isn’t good enough.

Every business and every software development team should embrace Occam’s Razor. Start simple. Add complexity as you go. The principle applies both to the software being developed and to the process being used to develop it. Good agile development teams understand this.

Start Simple.

Keeping the software system simple at the outset and adding complexity over time makes good sense. It’s easier to envision, build and test a simple system. As the system becomes more complex, the testing effort benefits from all the work that’s already been done.

It helps the stakeholders and end users too. It’s much easier for them to comprehend and evaluate a simple system than a complex one at the outset. Their feedback will be more focused and constructive.

Add Complexity as Needed.

Keeping the development process simple also makes sense. Why over-complicate the process with rules, procedures, documents and other artifacts that may add little or no value? Only add complexity and artifacts to the process when you’ve identified clear needs.

That’s why retrospectives are so important to the continuous improvement of software development. Don’t try to change too much, too quickly. Make frequent small changes to the process and evolve it rapidly.

If you’d like to be more agile, embrace Occam’s Razor. When in doubt, opt for the simpler approach or solution. The team will make faster progress and it will be much easier to apply corrections and/or re-group. Add complexity if needed, but only if needed.

For a more detailed explanation of Occam’s Razor, read this Wikipedia entry.

Failure to Collaborate Threatens All Large Software Projects

There’s a famous quotation from the classic movie Cool Hand Luke that applies to many business situations; “What we’ve got here, is failure to communicate.” In many post-mortem evaluations of failed projects, inadequate communication is cited as a primary reason for team turmoil.

Companies have responded by making it easier to communicate. Many of us carry smartphones enabling us to receive and reply to e-mail messages anytime, anywhere — even on vacation. We also have instant messaging and text messaging that allow informal conversations with anyone regardless of location.

Now that we’ve solved the communication problem, many people argue that over-communication has become a major burden. In particular, managers are overwhelmed by the non-stop blizzard of messages that rage 24 hours a day, seven days a week. There simply isn’t enough time to think through and understand every message.

The real problem is no longer failure to communicate. It’s failure to collaborate.

Teams and workgroups need more than the ability to send and receive messages. Exchanging messages certainly qualifies as communicating but it’s not exactly collaborative. High-performance teams must be able to share knowledge while using it to generate new ideas.

I’m a big fan of software tools. Technology has created the information overload problem and technology can solve it.

There’s an abundance of software tools available that enable collaboration. They fall under the three broad categories of document, content and knowledge management tools. The terms are often used interchangeably however there are differences among the three.

Types of collaboration tools

Document management is a structured means of storing, locating, and tracking business documents. The files can be word processing documents, spreadsheets, images, audio or video recordings, or any other file type. Multiple revisions of the files can be saved showing how the information evolved. In addition, security controls define who has read or write access to the documents.

Content management is focused on creating documents and controlling changes to their contents. This solution is used when two or more people are creating or modifying a single file such as a word processing document. Additions and changes are controlled by the software to avoid conflicts.

Knowledge management is about capturing worker experiences and making the information available to others. Most companies do this by creating word processing files or spreadsheets. While effective in capturing information, this approach leaves much to be desired in sharing it. A knowledge management system stores information in a database making it much easier and faster to search and retrieve.

Collaboration tools solve two large and growing problems. The first problem is information overload. Most of us are nearly drowning in a vast sea of computer data. If it’s not managed, it’s useless.

The second problem is information sharing. It’s great if one of your employees seems to have all the answers but what about everyone else? What happens if that gifted employee leaves the company?

Communicating has gotten much easier making failure to communicate less of an issue. Of course, progress brings new problems. Now we have to deal with failure to collaborate.

Don’t let it happen to your team!

While there are a variety of commercial vendors offering software products, check out OpenSourceCMS.com for information about open-source content management systems.