Research Is Not the Same as Software Design

Does agile development suffer from lack of architecture and design? Are agile solutions inferior to those developed using non-agile approaches?

Twenty years ago the answer would have been ‘yes’. Twenty years ago the software profession was less mature than it is today. Software engineering lacked standardization back then.

Patterns Rule

Today, we have patterns — architecture patters, design patterns, solution patterns, even workflow patterns. Whatever problem your team is trying to solve, it’s likely that someone has already solved a similar problem and published a pattern for the solution.

Thus, software professionals can get more done in less time by adopting a pattern and customizing it to their needs.

Solutions at the speed of change

Some will argue that every problem is unique requiring analysis and design time to craft the optimal solution. I would counter that in today’s fast-paced business world there just isn’t time to craft the ideal solution. Furthermore, business models and technology implementations are changing so fast that solutions are outdated shortly after they are delivered.

The lack-of-design argument against agile just doesn’t hold up any longer — with one exception. If your team is blazing a new trail, attempting to solve a problem that has never been solved before, you will need significant analysis and design time before development can begin.

In this situation, I’d argue that you are conducting research, not building software. Once the research is complete, why not proceed using an agile approach?

If you want a good source of software patterns, check out the Hillside Group Patterns Library.

Do you have anything to add?

Scrum’s Deep Dark Secret

Scrum has a deep, dark secret. It all started when Scrum was originally defined. One of the key goals was to empower the software developers to self-organize and take control of their destiny. This is often viewed as anti-management but I don’t believe that was the intent.

Self-organization demands that the team take responsibility for it’s work. This in turn means that the team will be held accountable for the decisions it makes and the results it delivers.

Here’s the problem. Scrum defines two powerful roles on the team — Product Owner (PO) and Scrum Master (SM). Filling these roles with high-quality people becomes a critical success factor for the team. That’s the deep, dark secret.

The Scrum Product Owner Is Powerful

The PO has qualities like the following:

• Actively involved in the project and readily available
• Knowledgeable about the software and the needs of the end users
• Empowered to make business decisions
• Collaborative, willing to discuss and negotiate
• Decisive and able to make on-the-spot-decisions

These qualities comprise a lot to ask of a single individual. They place enormous responsibility on the PO and make the PO an easy target if the project gets into trouble.

The Scrum Master Is Benevolent

Similarly, the SM has a tough role to fill:

• Responsible for adherence to Scrum principles
• Humble in the sense that the SM serves the team not vice versa
• Encourages collaboration among team members
• Able to discuss business and technical issues
• Influential without being controlling

Again, we have enormous responsibility placed on a single individual. Another easy target.

One could surmise that Scrum was designed to protect the software developers while setting up two non-developers as scapegoats should the project get into trouble. That’s not self-organizing and it’s not teamwork.

Honestly, I don’t believe that was the original intent of Scrum at all. I think the original intent has been distorted by a minority of developers trying to avoid accountability.

Scrum needs to get back to it’s roots. PO and SM are just two of the many roles on a team. The PO and SM have no more to do with the success or failure of the project than anyone else. That’s agile!

What do you think? Is there too much emphasis on the PO and SM roles?

Enterprise Solutions Don’t Have to Take Forever

Have you ever worked on an enterprise-scale project in a big company? By ‘big’, I mean one with an IT organization of several hundred people — or more. If you have, you’ve likely had the frustrating experience of sitting in a room with 10-20 people (or more), some of them senior managers from several IT departments, trying to agree on an approach to solving a problem.

By definition, those managers will have differing agendas. Their priorities reflect the functional areas that they oversee, not the needs of the project.

Oftentimes, the group reluctantly agrees to meet periodically as they talk through the issues and try to arrive at a consensus. This is guaranteed to take forever (well, it will seem that way) and is anti-agile from any perspective.

Here’s a more agile approach.

1. Start the meeting by offering an overview of the project with its key goals and priorities.
2. Quickly focus the meeting on the problem at hand — do not get pulled into tangential issues. (If pressed, agree to follow up on those tangential issues later.)
3. Give everyone a brief opportunity to ask questions and/or express an opinion.
4. Ask for help in creating an acceptable solution. (Asking for help is important in winning support for the effort.)
5. Suggest forming a core team of experts to fully assess the problem and recommend a solution. (The core team will likely have to include at least one member from each IT department represented at the meeting.)
6. Request to have the core team meet two or three times a week. This will emphasize the importance of the issue and send the message that being on the core team is a commitment.
7. Let the whole group know how you will keep them informed and when you will meet again.

This approach will result in a better solution in less time as long as you have a strong leader on the core team. Be agile, give it a try, and send me some feedback.

Take a Simple Quiz, Learn About Being Agile

James Shore and Shane Warden developed a self-assessment quiz for software development teams. It is included in their book, The Art Of Agile Development.

Most of us seem to like surveys, quizzes, and questionnaires. They often help us gather our thoughts and explore topics in greater depth. The downside is their superficial nature. It’s hard to improve your understanding of a complex topic just by answering some questions.

Negatives aside, I like having complex topics boiled down to a series of questions. They make me think and often encourage me to research a topic in more depth.

According to Shore:

“The goal of the quiz isn’t to tell you if you’re ‘agile’ or not — it’s hard enough for a human to do that, let alone something printed on dead trees — but to give you an idea of where you could improve, and what your biggest sources of risk are.”

Nicely stated.The quiz is a series of about 50 yes/no questions. After each response, try asking yourself “why” or “why not”.

Sebastian Hermida has gone to the trouble of posting the quiz online in a simple easy to use format. Just click on over to abetterteam.org. You might get more out of the quiz by having the development team take it together. The ensuing discussion might be the best thing to come out of the effort.

Slow Down; You Move Too Fast!

There’s a common misconception floating around in enterprise companies. They believe that software release/upgrade cycles have to be long. At one time, Microsoft was criticized for releasing major Windows updates too fast. (I know, hard to believe, isn’t it?) Why do they feel that way?

• It’s expensive and time consuming to roll out new software releases.
• It’s disruptive — user training, equipment upgrades, process changes, etc.
• The user community doesn’t like it — too much hassle.

Clearly, trying to introduce agile software development into such a company will be a steep uphill climb — but there is hope.

Despite these widespread sentiments, there are large companies that have adopted regular, frequent, upgrade cycles and done it with minimal cost, disruption, and hassle. They do it by controlling the number of changes in each release.

Rather than two or three year release cycles that result in drastic changes in the software, they focus on release cycles of a few months. The magnitude of the changes is much smaller and thus easily managed.

A good example of this approach is Salesforce.com.

Background: Salesforce.com is an enterprise, cloud-computing company. They started out with a CRM solution and have diversified into custom, cloud-based applications. In 2010, Salesforce.com generated over $1.6B in revenue. They have over 92,000 customers worldwide.

When Salesforce releases a software upgrade, it’s a big deal. They have 23 production and sandbox instances. Each instance consists of multiple servers. Tens of thousands of users are impacted.

If Salesforce operated by conventional wisdom, they’d do a major release about every 5-10 years. I think they’d be out of business well before then. Their actual release cycle is 3 to 6 months — and they do it with minimal problems and little disruption to customers.

Fast doesn’t have to be disruptive.

There’s more to it than just keeping the number and scope of changes small. Here are some other things Salesforce does to make frequent releases a positive experience:

• Maintain backward compatibility at the API level. Don’t break user customizations.
• Make major new features optional. Don’t force customers to take something they don’t want.
• Conduct beta testing for significant changes. Let customers try out a feature and offer feedback.
• Provide plenty of training and documentation.

Of course, there are many other enterprise companies that operate with rapid release cycles. Google is certainly the most notable among them.

There is no point in adopting agile development techniques if your IT department is on a mission to slow down the pace of change. Work with them to identify pain points and find ways to address their concerns. Being enterprise agile is more than just daily stand-ups and short sprints.

How do you feel about rapid change?

7 Steps to Bug-Free Software

No bugs! Sounds good! Is it feasible?

Can agile software developers create software with no bugs? Image if they could. Think of all the time that would be saved testing, re-testing and testing some more. All that time spent repairing things that weren’t done correctly becomes time to spend on making the software better and faster.

It is a feasible goal.

Achieving it means thinking about the process of writing software differently. Some agile teams are proponents of pair programming believing in the theory that two heads are better than one. Pair programming can help but it alone won’t get you to zero defects.

To go all the way, consider adopting all of the following techniques:

1. Pair programming or at a minimum peer review. Some form of community participation will always result in better code than can be produced by an isolated developer.
2. Test-driven development. You’ve heard it before — write the tests before the code. You need to go one step further though. Someone other than the person writing the code should write the tests. Don’t isolate them. Have them collaborate and develop a common understanding of what the code must do.
3. Define coding standards and enforce them. We all take shortcuts when we’re rushed (and we are always in a rush). Coding standards will help ensure that the code is written properly.
4. Get your customer actively involved. Misunderstandings around requirements, stories and features are common. The only way to prevent them from becoming defects is to engage the customer.
5. Allocate time to pay down technical debt. Debt is inevitable. Defects are not. The more debt you build up, the greater the likelihood that bugs will infest the code.
6. Track the source of defects that make it through. Look for patterns or common sources of mistakes. When you identify them, take steps to prevent future occurrences.
7. Lastly, automate your regression testing. A simpler and faster testing cycle will result in more testing which will uncover more bugs before the software leaves the development team. This is a last resort and the number of bugs found at this stage should be minimal.

The most important goal is to prevent bugs from reaching the customer. It won’t be easy, particularly if you are dealing with a large base of legacy code, but it can be done. Care to comment?

It’s Not Scrum Vs. Kanban; It’s Scrum And Kanban

Scrum and Kanban have a lot in common — far more similarities than differences. Scrum is more structured and disciplined than Kanban. Kanban is more adaptive and flexible than Scrum. But the major difference that I see is simply that Scrum relies on time boxes, Kanban relies on workflow control.

Scrum Time Boxes

A time box is a fixed time period. Scrum often uses 2-4 week time boxes though shorter and longer periods are also in use. One of the major tenets of Scrum is that the time box cannot be violated. At the outset, backlog items are selected such that they can be completed within the time box. Anything that is not completed moves to the next time box or may even return to the backlog.

Time boxes establish a cadence. The team and the user community become accustomed to regular software deliveries at the end of each time box. Face it, we are all creatures of habit. We like events to occur on a regular schedule.

Kanban Workflow Control

Kanban controls workflow by limiting work-in-progress (WIP). The general idea is that stories are either waiting in the backlog (inventory), being developed (in-progress), or completed. Having too much work-in-progress leads to waste, confusion and bottlenecks.

Limiting WIP provides some flexibility. It is easier to change the workflow, add new stories or re-prioritize them using Kanban than it is with Scrum. Thus, Kanban may work better in environments that are in flux — hard to predict.

Challenges

The challenge with time-boxes is that while they sound good on paper, maintaining a regular rhythm in software development is not easy. Interruptions, unforeseen problems, new requests, emergencies, etc. are all lurking nearby just waiting to disrupt the time box.

The challenge with limiting WIP is the constant attention needed to maintain control. For example, if code development gets too far ahead of system test, adjustments need to be made to restore balance. Also, the team may feel like they never get a break as the work items just keep coming.

Which is best? I like Kanban although I have to admit that we can never get away from hard deadlines — customer demos, trade shows, industry events, financial reporting, etc. Kanban doesn’t do well with these kinds of time-boxed efforts. Maybe we need Scrumban — borrowing the best elements of each.

We could employ a lengthy time box, say 8-12 weeks, and use Kanban within the time box switching to Scrum near the end in order to synchronize the final deliverables. Our ultimate goal is to satisfy the needs of the business. Don’t ever forget that.

If you want to read more about the similarities and differences between Scrum and Kanban, take a look at the PDF download called “Kanban and Scrum – Making the most of both“. Your thoughts?

Mozilla Wants to Be Agile. Can They Do It?

The Mozilla Foundation is exploring moving from a long, browser development cycle to a much shorter one. If you’ve followed the life cycle of the original Mozilla browser which evolved into Firefox you know that the release cycles have been unbearably long at times.

For example, Firefox 4.0 has been in development for over a year. In the fast-paced world of browsers and web technologies, that’s an eternity.

Google’s Chrome browser undergoes much faster release cycles, sometimes only weeks apart. Can Mozilla successfully speed up it’s development life cycle and go head to head with Google?

Yes…but…it won’t be easy.

Mozilla will need to overcome cultural, organizational and infrastructure issues to get there. The cultural issue is obvious. Everyone is accustomed to the long, methodical, development process. Many will question the need for change. Mozilla will need to sell the new process internally and sell it hard!

Organizational issues are less obvious but no less difficult. Shorter cycles will mandate overlapping development teams to accommodate short and long term needs. QA will need to be more tightly integrated with software development. The current organizational structure is unlikely to support accelerated release cycles.

Finally, infrastructure issues are last but far from least. Everything from the layout of the office space to the software tools used by the development teams will need to be updated or overhauled. These types of changes will be controversial and, possibly, expensive (luckily, they are an open-source organization).

The extent to which the Mozilla Foundation faces these challenges and commits to change will determine their long-term success. It won’t be simple and it won’t happen within a single release cycle. I hope they make it happen. I’m a long-time Firefox advocate, though I have to admit that Chrome is also on my laptop.

P.S. Ars Technica has published an overview of Mozilla’s proposed development cycle that is worth reading.

Don’t Get Stuck on a Definition of Done

What does it mean to be ‘done’? Many agile software development teams adopting Scrum, XP or Kanban struggle with their ‘definition of done’.

They usually try to derive a set of measurable criteria that will enable them to verify that a work product is fully complete and ready to be deployed to the user community. Most definitions of done include things like:

• Story written and estimated
• Story acceptance criteria defined
• Code written
• Code unit tested
• Code integrated into the build
• etc.

The problem with kind of universal done definition is that it treats all aspects of the project the same. Stories should not follow the same done criteria as sprints. They have unique characteristics and thus their definitions of done will be different. Trying to lump them altogether creates a problem that quickly becomes too complex to solve.

Create three definitions of done

The three major elements of a major software project deserve their own definitions of done:

• Stories and the associated code
• Sprints
• Releases

This will (hopefully) make it easier to create the definitions because they will be focused rather than encompassing.

There are always exceptions

If you want to take this a step further, consider implementing an exception process. That is, not all done criteria will apply to every project artifact.

For example, some code modules will require performance testing due to their complexity or data volume requirements. Other modules will not because they are accessed infrequently or used in low stress situations. You could include performance testing as a done criterion and define a process for granting an exception if warranted. Usually, the Scrum Master and Product Owner would have to approve the exception.

Don’t get stuck defining done. Keep your definition(s) simple and expend your energy on delivering results that add value. Make sense?

Some Waterfall Project Phases Have To Go

One of the big problems I have with the waterfall approach to software development is the laser-like focus on the beginning and end phases of the project. It starts with requirements gathering and ends with testing, testing and more testing.

All of the intense activity and complex analysis in the middle gets lost. At the very least, let’s get rid of the following “phases” of waterfall software development.

Requirements Gathering: For any fast-paced or rapidly-growing organization, there is simply no way to fully define software requirements up front. It cannot be done. You merely end up with numerous change requests which just adds overhead. (Of course, you could inflate your estimates so much that your team can accommodate any reasonable request for change but that would be unethical, wouldn’t it?)

Requirements need to be gathered and analyzed throughout the software development process. Some of the obvious requirements gathered up front will be deferred to make way for late arrivals. This is a huge problem area that agile approaches like Scrum, XP and Kanban are designed to solve.

Alpha Test: This is supposed to be the project phase when the application is not yet ready for customers to see but can be used by insiders or carefully screened outsiders. What an antiquated concept! Ultimately, the only opinion that matters is the customer’s.

Focusing your team’s attention on testers that are too close to the project or too far removed to give a damn will confuse and frustrate everyone.

Beta Test: We’ve all participated in beta tests. The software is ready for the customers and prospective customers to use but remains incomplete and possibly buggy. Incomplete and buggy…that describes every software application I’ve ever used!

Forget beta! Focus on automating a business process, then another and another. The software will never be done. It will evolve and improve over time. And one more thing, don’t rely on customers to find bugs!

User Acceptance Test: This is my favorite. Acceptance of what? If the users ‘accept’ the software, what happens? Do they get stuck with it bugs and all? Do they have to use it everyday whether they like it or not? What if they reject it? Does the project start over?

It’s about user feedback — early and often. Admittedly, consulting firms have a legal and financial interest in ‘acceptance’. Fine. For the rest of us, hope that the users reject the software so you can keep making it better!

Waterfall is deeply entrenched and is not going away anytime soon. Can we at least get rid of these antiquated phases?

What do you think?