Big Changes May Need Special Handling

If it won’t fit within a sprint, get creative.

One area that causes concern among software developers using Scrum is the short sprint time boxes. Specifically, how can they develop complex algorithms in short bursts?

We’ve all heard that stories should be kept small so that each one can be fully implemented and tested in a single sprint. 4-week sprints will make it easier to meet that goal than 1-week sprints. However, 1-week sprints make managing the project simpler because there are more frequent checkpoints and fewer opportunities for the team to get off track.

Some teams adopt 6-, 8- or 12-week sprints so they have plenty of time to implement complex stories. This may seem like a reasonable approach but be aware that there are many ways for the team to get itself into trouble over such long sprint durations. Thus, managing the project will be more difficult for the Scrum Master and the Product Owner.

There are occasions when a major chunk of work must be done and there is simply no way to accomplish it within a sprint. This usually happens when something is being upgraded or replaced within the software. Here are a few examples:

• Rewriting a complex search and sort algorithm to replace an existing one. You could implement search first then sort but you may have to take away the sort function for a sprint or two. Not good.
• Changing a database model. If the existing model needs major revamping, making the changes, updating existing code, and migrating the data is unlikely to be completed in a single sprint.
• Converting to a new library or toolset. If you’ve ever been through such a conversion, you know that it can be quite an ordeal. Some deficiencies get fixed and new ones are introduced.

What can you do?

Several options exist for dealing with big changes and the one you should select depends upon your situation. Regardless of the approach, normal sprints should continue even if the number of developers is reduced. If you stop sending updates to the business users, you risk losing their interest, or worse, damaging the team’s credibility. Here are a few approaches to consider.

Create a Feature Branch

Branching the code is an obvious choice. The team makes all the necessary changes to the branch and checks the new code into the trunk upon completion. This means having what are essentially parallel development teams. One group keeps working on backlog items within sprints while the other tends to the big change.

Ideally, code in the feature branch should be checked into the main branch often to maintain build integrity. The undone code can be hidden or disabled (see below). The point is to periodically verify that the build is not broken rather than to wait until the end only to discover the new code cannot be merged into the production build.

Hide or Disable Features

New code can be hidden from the user or completely disabled using options within the build or a configuration file read by the software at startup. User interface elements can be hidden or grayed out. Functions (or classes) can be configured to return a null result or dummy data so that other parts of the system are not impacted.

In a similar vein, the team could implement the user interface for a feature (or whatever triggers the feature) after the logic is built and tested. Thus the feature is implemented but there is no way to access it within the software until the team is ready.

Write a Wrapper Layer

If the team is replacing an older code module with a newer one, they could add a layer of abstraction such that the existing software calls the wrapper and the wrapper decides whether to invoke the old or the new code. The wrapper can be controlled via build or configuration file switches.

This approach means writing some throwaway code but it may be better than keeping the business waiting for updates or taking the risk of breaking the software.

Options are good but consistency is better.

It’s nice to have a set of options for dealing with big changes. Having said that, there is no substitute for short stories, short sprints and continuous integration. Use the approaches above if you must but only after you’ve explained why accomplishing the work within the sprint time boxes is not feasible. I’ll bet that more often than not, the work can fit within the sprints. You just have to try harder. Have you used any other techniques?

Enterprise Agile Adoption Requires a Phased Approach

Changing a major process in any enterprise is never easy. Software development certainly qualifies as a major process. If you’re thinking about having your enterprise software development teams change the process they follow, it won’t be easy.

Let’s assume the current process is more ad-hoc than not. You may call it waterfall, spiral, unified or something else but there’s a good chance it lacks discipline. Regardless, you decide to adopt an agile approach (because agile is all the rage these days) and Scrum seems like the best fit.

A phased approach is best.

Your first objective is to explore. Start with a small project. Consider it to be a pilot. Get the team trained and/or bring in a coach to help out (Note: I’m neither a trainer nor a coach so this is unbiased advice.)

Expect the team to struggle a bit and make a fair number of mistakes. Stories will be too long; estimates too optimistic; and expectations too high. It’s all good because the best way to learn is to fail.

Also, expect cultural obstacles. Getting sufficient business participation is likely to be a challenge. You’ll need to demonstrate some success to energize the business users and get their buy-in.

Assuming the pilot effort is successful, it’s time to execute on a broader scale. There will be two major areas to consider:

1) Multiple smaller-scale projects. The more independent these efforts are, the easier it will be for the development teams to focus and make headway. Keep in mind that pilot success does not assure success here. You’ll need to repeat what worked during the pilot and correct the mistakes. New Scrum teams will encounter many of the same challenges faced by the pilot team(s). Offer as much help and guidance as you can.

2) Large-scale projects. These will be more challenging. Hold off on tackling an enterprise-scale agile project until you have at least a few experienced teams with multiple successful outcomes.

During this execution phase, metrics and tools will become larger issues. Enterprise reporting will require standard metrics and standard presentation formats. Now is the time to begin addressing those issues.

Time for enterprise-wide rollout.

Getting the entire enterprise aligned with agile development and Scrum in particular raises many more issues: governance, titles, compensation, HR policies, funding, etc. None of these are unique to agile approaches or even to software development. They are artifacts of the corporate world.

This brief blog post only scratches the surface of adopting a new agile approach to enterprise software development. It’s harder than it looks and more rewarding than you think.

5 Early Warning Signs of Agile Project Trouble

Your agile project is losing momentum. Things aren’t going as well as they should, though it’s not clear why. Here are five early warning signs that an agile project isn’t so agile and may be headed for trouble.

1. Meetings keep getting longer and there are too many of them.

If your daily standups are getting longer and longer, watch out. Something is going wrong. The team may be losing focus or getting off track. If more and more meetings are being scheduled to resolve issues, the team is likely to be struggling.

These are early warning signs of trouble. Pay attention. Keep to the 15-minute time limit for standups and use informal discussions rather than formal meetings whenever possible.

2. The team follows a hierarchy and seeks approval before acting.

At times teams will establish one or more hierarchies. For example, there may be an acknowledged system architect, a keeper of the build, or a quality guardian. It’s fine to have designated subject matter experts (SME). Problems develop when the team stops and waits for SME approval. The SME quickly becomes a bottleneck, slowing the team down.

If there is a SME on the team, that person should guide the team not look over everyone’s shoulders. A SME that is protective of his territory will only hurt the team.

3. The team keeps getting bigger.

Sometimes agile teams will add people to try and do more in less time. The strategy can work if there is enough time left in the plan for the newcomer to have a meaningful impact.

Management may conclude that if adding one more person helped, why not add another? And another? Maybe the underlying problems that are causing the team to appear understaffed should be fixed. If the team gets too large, consider splitting it into two autonomous teams.

4. Deadlines are missed and sprints are lengthened.

One solution to the problem of missing sprint deadlines is to make the sprints longer, right? It sounds reasonable but if the team is having trouble planning and executing short sprints, it will only get tougher to manage longer ones.

Missed deadlines are a symptom. There are many possible causes of the underlying problem. The key point is that if deadlines are missed, you need to dig deep and understand why.

5. The team gets bogged down in documentation and becomes resistance to change

At times, some people new to agile development may be uncomfortable with a lightweight agile approach. They may attempt to make up for that by ‘writing things down’ and trying to control change. This may also result in each sprint decomposing into analysis, design, code and test segments.

This is a clear reversion to waterfall. Watch out for it.

Retrospectives can help.

Retrospectives are enormously valuable in dealing with these warning signs. Openly discuss the issues and agree on actionable steps to change behaviors.

You don’t have to wait until the end of a sprint to conduct a retrospective. You may want to look back and self-correct every week until the team feels comfortable with the process and with each other.

Don’t Wrap a Straitjacket Around Your Project!

Three constraints dominate discussions about controlling projects — time, cost and scope. I’d like to add people and quality to that list. Many managers try to control all five constraints and end up controlling none.

Common wisdom says you have to let at least one constraint float free if you want to tightly control the others. So which constraint is the best choice to let float? Let’s see.

Cost

I see many companies largely ignore project cost. They seem to feel that they have resources available so they may as well apply them to project X. Only when someone notices that the project has dragged on too long does cost become an issue. Too late!

Cost should always be calculated up front. It’s best to look at minimum, targeted and maximum (reasonable) cost. No company has unlimited funds to throw at any project. Cost is always an issue and should not be allowed to float (too much).

Time

Time is the constraint that seems to be fixed most often — and justifiably so. There are deadlines imposed by customers, vendors, trade shows, regulators, competitors, etc. Get used to it.

If you value your credibility, don’t let time float. Time is the easiest constraint to control; simply employ time-boxes and never break them.

Scope

Scope encompasses all the features, functions and attributes of the software. We spend much of our time analyzing and planning scope. Many projects try to rigidly control scope at the expense of time and cost — and quality.

Too many of us get emotionally attached to specific software behaviors and are willing to sacrifice all other constraints to get the behavior we want. Bad idea! Scope is the hardest constraint to control but it is not difficult to float predictably.

People

The biggest expense of almost every software project is people. But the topic of people as a constraint is about more than just cost. People aren’t merely ‘resources’ that you can move around from project to project without regard to skill sets, career goals or labor rates. Assign the wrong people to a project and you increase the failure risk, cost overrun risk, and quality risk.

In the short run, you’ve got little flexibility with people. In the long run, many possibilities exist. How much time do you have?

Quality

I hear many managers claim that quality is not a variable. Quality must be kept at a consistent level. Right. So the reason we have so much buggy, crappy, ugly software out there is that the quality standard is too low!

In reality, quality always suffers when teams are pressed for time, money, people, or scope. Quality should be fixed at a high level (assuming you know how to measure it). Quality is the easiest constraint to float and the hardest to measure. Don’t give in to the temptation to cut corners on quality. It will come back to haunt you.

Which constraint would you pick?

• Cost – Fiscal discipline is good. Blowing budgets is not.
• Time -We live by the clock and the calendar. Time is not flexible.
• People – Slow to change.
• Quality – Don’t even think about it!
• Scope – Define it; prioritize it; float it.

Make sense? If so, try an agile software development approach like Scrum or XP. They are optimized to manage cost, time, people and quality while letting scope float.

If you’re stuck in a waterfall world, select an iterative waterfall model and minimize the time devoted to requirements (scope).

Don’t wrap a straitjacket around your project!

The Benefits of Agile Development May Not Be What You Expect

Why do organizations decide to try agile software development? The simple response is that they want to improve their software development results. Sure, but what exactly are they trying to “improve“? Here are the most common improvement candidates:

• Time to market – they want to generate software releases in less time from start to finish.
• Better quality – they want to reduce the number of customer complaints (defects, usability problems, performance, etc.)
• Lower cost – they want to reduce the cost of developing and supporting their software

Can agile development deliver all of these benefits? Not likely. It can help but it’s not a panacea. If agile development doesn’t guarantee top and bottom line benefits, what business value does it offer?

Consistency and Predictability

Every business wants to be consistent in their delivery and predictable in their results.

One of the single biggest problems in the software industry is the haphazard nature of releases. Customers often don’t know when to expect a new release and as a result are unable to do any planning in advance. This leads to customers frequently ignoring new releases unless they contain critical security fixes or highly desirable features.

Prior to 2003, Microsoft was severely criticized for releasing software patches at random, often catching major customers by surprise. It adopted a consistent and predictable release schedule for Windows and Office security patches in 2003. The regular schedule makes it much easier for everyone to plan ahead and stay up to date. Out of sequence patches are still needed on occasion but they are relatively rare.

Companies like Salesforce.com and Canonical (Ubuntu Linux) issue major new releases on regular schedules. The contents of every release are not pre-determined. New features that are ready to go are included and those that miss the cutoff date are held until the next release. In essence, the feature set floats while time to market and quality level are fixed.

Can this approach work for everyone? There are exceptions to every rule, so the answer has to be no. However, my experience strongly suggests that the vast majority of customers prefer regular, predictable updates. They may not get all the features they want but they know they’ll get something of business value.

There are many ways to achieve consistency and predictability. The keys to success are setting priorities and establishing discipline.

Priorities and Discipline

Candidate features (preferably in the form of stories or use case scenarios) must be prioritized. Everyone on the team must be willing to accept that not every candidate feature will make it into the next release, even if it is marked as high priority. Why? The target date and the quality level will be maintained. If a high-priority feature proves to be overly complex or places the release at risk, it will be removed from (or disabled in) the build.

Establishing discipline is a matter of ensuring that everyone understands the goals and is committed to achieving them.

This approach may be awkward at first, especially for groups that are accustomed to making huge commitments and working hard for however long it takes to deliver. When releases are 6-, 12-, 18-months, or more in duration, having a feature miss a release is a big deal. When releases are 1-, 2- or 3-months apart, it’s not such a major concern.

Give it a try. I’ll bet you’re pleasantly surprised at the positive reaction from your customers. What do you think? Have you tried it?

Don’t Just Ask Questions, Explore Possibilities

Should the Product Owner have all the answers to business and functional questions that arise during a sprint? If she doesn’t, should the business stakeholders have all those answers? If not, what about the end users of the software?

Similar questions apply on the technical side. Should the software developers be able to answer any technical questions that arise? Should the testers know all the testing answers?

‘No’, to all of the above.

No one can be expected to have all the answers around a subject even if that person is the designated subject matter expert. That’s why agile teams emphasize interactions and collaboration. The best answers come from open discussion and exploration.

I see these situations play out in meetings all the time. Someone asks a complex question. Everyone turns toward the subject matter expert (SME) for the answer. The SME doesn’t have the answer and suggests that she’ll get back to the group. Interaction stops. Collaboration never begins.

The answer may be provided at the next meeting or via email to the group. Either way, the group misses out on an opportunity to understand how the answer was derived and to comprehend the meaning behind it.

These scenarios happen because people fall back into their comfort zones. In traditional software organizations, individuals and groups are expected to have expertise in an area. For example, someone may be a database expert, java expert, SQA expert, business process expert, etc. Everyone learns to defer to the designated expert when questions arise.

Want to be more agile? Stop doing that!

Ask more questions. Brainstorm. Explore possible answers. See if the team can find the answer or at least derive options by simply interacting and collaborating for a few minutes. Confirm the derived answer or the possible options with the experts. The team will learn more and retain it for future efforts.

Documents Should Encourage Team Collaboration Not Kill It

Much of our communication is in writing. We write plans, requirements, specifications, reports, code, tests, etc. For a major IT project, the volume of paper can be enormous, but is it valuable?

In today’s fast-paced, high-stress, businesses, no one has the time to read volumes of information no matter how nicely formatted and presented. Yet, many organizations continue to turn out the equivalent of a multi-volume encyclopedia on project after project.

Why do we continue this archaic practice?

It’s the thinking process that goes into creating documents that is really important, not the documents themselves. Unfortunately, documentation often serves as a vehicle for showing off what a good job we’re doing rather than simply conveying information.

Elaborate and fancy documents require a lot of effort but rarely provide equivalent value. They tend to contain large amounts of boilerplate and little original content.

Such lengthy documents tend to discourage readers. They are too lengthy to absorb electronically and too bulky to print out and thumb through. Readers tend to go for the executive summary, which while conveying a few major points, is short on specifics.

There’s a better way. Apply the principles of agile development and lean programming to produce documents your teams can really use.

1. Focus on content not form.

Before writing any document, there a few questions you need to answer. What is the purpose of the document? Who is the intended audience? How long will the information be needed?

If the document’s purpose is to impress someone such as a customer or business partner, appearance is important and the extra effort is justified. If the document is strictly for internal team use, appearance shouldn’t matter.

Reverse your thinking and follow an agile approach. Rather than beginning with a lengthy formal template and looking for ways to simplify it, begin with a short informal outline. Increase length and formality as needed. To save time, send out an informal document to get everyone moving and add formality later, if needed.

2. Keep it brief.

We like to create lengthy documents that cover all aspects of the problem and corresponding solution. Yet, in most companies, people are specialized. Most of the document contents don’t apply to particular individuals who ask themselves if it’s worth the time to wade through a tome to extract the few points of interest to them.

Think lean. Break up long rambling documents into a few short ones where each is tightly focused. Your goal is collaboration not just communication. Big documents communicate but they do little to enhance thought-provoking discussion.

3. Minimize the work effort.

Seek out ways to minimize the work to be done. Many problems are solved on a whiteboard. Often, someone takes the time to type up what’s on the whiteboard, re-draw the diagram, and send an email to the team. Yet often, the information is needed but the formality is not. Why not just take picture or two of the whiteboard using a cell phone camera and email it?

Lastly, avoid duplicating information. Rather than copy content from one document to another, simply refer to the original. Not only does this keep the new document shorter, it makes maintaining and updating document sets easier.

Recognize that the most efficient way to convey information is face to face. Documentation should encourage and facilitate discussion. Too often, it has the opposite effect. Agile documentation keeps teams focused on meaningful insights not fancy layouts. Do you have any other suggestions?

7 Tips for Creating an Agile Work Environment

Okay, full disclosure, what follows is common sense — basic managerial stuff. But, it’s doubly important for any organization that wants to be agile. Applying command-and-control management techniques to agile development slows everyone down and causes more problems that it solves.

Don’t just skim over this list. Pick an item from it and get to work creating an agile work environment — because agile software development involves a lot more than writing and testing code.

1. Engage in open and honest communication: Lack of communication is frequently cited as a failure point in project after project. Everyone should feel free to offer their thoughts and opinions without repercussion, as long as it’s done in a professional and respectful manner.
2. Empower your teams: Lack of empowerment is a leading workplace stress factor. Allow the team to make as many decisions as feasible regarding their workflow and the project deliverables.
3. Be flexible: Family and work demands are often at odds. How, where and when (daily) work gets done is less important than delivering high-quality results in a timely manner. You’ll create a tremendous amount of loyalty and goodwill.
4. Reward and praise great performances: Lack of recognition is a common complaint on many teams. Verbal praise, particularly in front of peers, carries a lot of weight. Rewards can be as simple as a free lunch, time off, or a tee shirt.
5. Maintain a positive outlook and tone: Negativity spreads faster than the fastest wild fire. Be positive and upbeat. Set the expectation that negative remarks need to be accompanied by positive recommendations.
6. Encourage social interactions: People like to learn more about other people. Picnics, office parties, outings, and other social events help build camaraderie.
7. Support career development: Lack of training or advancement opportunities is demotivating. Find out what each team member wants to learn and find ways to offer associated opportunities. Formal administrative promotions may be rare but there are many leadership possibilities.

Bonus Tip: Maintain a consistent pace of work: Frequent emergencies will wear everyone down. There will be inevitable crisis situations requiring an “all hands on deck” call. Communicate the reason for the call and request everyone’s understanding and assistance. And, don’t make a habit of it.

Scrum for One Can Be Done — Almost

I’ve touched upon some of the difficulties in expanding Scrum to very large projects. I don’t believe that the Scrum of Scrums approach really works. It helps solve the management problem but the system engineering problem is left untouched. Not good.

That issue aside, I’ve been thinking about the opposite problem. Can a lone software developer use Scrum? Assume there is no Scrum Master. A Product Owner (or at least a small group of people that ultimately own the software) exists. There is no dedicated QA support but the end users are glad to test the code and provide feedback.

Does Scrum add value or does it just add overhead?

Product Backlog

Any software project (except for extremely small initiatives) will have a list of features and functions to be implemented. That list may consist of ten or so epics or it may be much, much longer. Regardless, the list should be described, sized and prioritized. Having a prioritized to-do list is simply good practice in any non-trivial endeavor.

Sprint Backlog

For a single developer, sprint duration and keeping sprints to a fixed length are of less importance than for a larger team. A lone developer is likely to be interrupted often and is also likely to have production support issues to handle making sprint timing erratic.

What’s most important is to discuss the contents of the Product Backlog with the business folks. Jointly select the epics to be implemented in the next sprint. If an epic is very large (e.g. requires several calendar weeks to implement), it is wise to split it into two or more epics.

The chosen epic(s) must be split into stories that are estimated in greater detail and reviewed with the business. I strongly recommend being able to demonstrate weekly progress to the business users. This aspect of the development will be simpler than trying to do weekly demonstrations when the team is larger.

Daily Stand-Up

Okay, meeting in the team room in front of the Scrum board each day when you are a lone developer is a bit silly. However, you need some way of tracking progress and a Scrum board (or Scrum spreadsheet) may work for you. It’s also a good idea to go through the 3 famous Scrum questions. They will help organize your thoughts and plan your day (interruptions aside). If you see an impediment, escalate the issue to management or the business immediately.

Automated Builds and Testing

Anything you can do to automate repetitive tasks will only help you get more done and deliver better results. Whether or not to automate builds and regression tests is a judgement call. You are likely to find that some aspects of the development process benefit greatly from automation while other parts can just as easily be done manually.

Definition of Done

A checklist showing all the major steps that have to be completed before the software can be turned over to the users is always a good idea. It doesn’t have to be as rigorous as a formal definition of done but it will help organize your thoughts and workload.

Burndown Chart

A burndown chart is probably not needed by a lone developer. It may be interesting to look at but adds little value.

Is it really Scrum or is it another Scrum wannabe?

Honestly, it’s not Scrum simply because it does not meet a few of the essential characteristics — there is no Scrum Master and there is no Team. Regardless, following a structured approach to software development will improve your workflow and your deliverables no matter what you call it.

The discipline you instill in yourself will help your career as you advance to larger and more complex projects. What do you think?

Scrum’s Sprints Are Not Mini-Waterfalls

Teams that have extensive waterfall development experience often encounter a common problem when they adopt an agile approach like Scrum. The problem results from applying waterfall development to every sprint. The result is a project consisting of a series of mini-waterfalls.

For example, let’s say the team selects 4-week sprints. It feels quite natural for them to break up each sprint into three phases. They do analysis and design during the first week (phase 1); spend the next 2 weeks coding and unit testing (phase 2); followed by 1 week of system testing (phase 3).

This approach can work if the software features are prioritized and each phase is strictly time-boxed. There also needs to be a strong definition of done and a willingness to not deliver features that are incomplete or defective.

Those rules are usually violated with the obvious consequence that the testing effort gets time squeezed. There isn’t enough time to test the new code nor to fix any defects discovered. Project timing becomes erratic as the sprints get elongated. Incomplete and defective features are delivered to the business. Everyone gets frustrated.

“We tried Scrum and it didn’t work!”

Well, not exactly. They tried an iterative waterfall approach, blended it with some elements of Scrum, didn’t follow either approach rigorously, and ending up failing. Surprise!

One of the reasons teams fall into this trap is simply because their stories are too complex. Any story that requires significant analysis and design is too big. It needs to be split up into smaller chunks so that analysis, design, coding and testing can all be completed within a few days, or at most, about two weeks (assuming 4-week sprints).

In addition, the stories selected for a sprint should vary in size and difficulty. If the team selects a small number of big stories for a sprint, testing is more likely to get squeezed at the end.

It’s also helps to get completed code into test as soon as possible so that there is time to fix defects before the end of the sprint. A strong definition of done helps enforce that discipline.

There’s nothing inherently wrong with iterative waterfall — just don’t call it Scrum.