No Excuses: Be Ready to Deliver What You Promised

Being without electrical power for half the weekend, got me thinking about project teams that run into unexpected obstacles. For example:

• Third party software components that don’t function as expected
• Software tools that prove to be insufficient for the tasks.
• Code modules that cannot perform intended functions as currently implemented.
• Servers or networks that cannot handle the load expected of them.
• Office environments that are not conducive to knowledge work.
• Team members who cannot perform to the level needed by the team.

Regardless of the obstacle, the stakeholders and senior managers want their software — not excuses. Overcoming such obstacles requires two things (in no particular order):

A Recovery Plan

Every mission-critical decision should have an alternative. If the decision doesn’t play out as expected, the team will need to adopt an alternate approach. Software components and tools can be swapped out. Code can be re-written. Servers and networks can be upgraded. Office environments can be modified. It won’t be simple or quick but you’ll be ready.

The keys are understanding the risks and having an alternate approach in mind going in. There is no way to prevent or avoid every difficulty but being prepared makes them much easier to manage.

A Decision Process

The team can only execute the recovery plan if they have a process for identifying failure points and cutting losses. All of us tend to stick with a solution approach long after the approach proves to be inadequate — call it pride, tenacity or human nature. Anything that is not working as expected needs to be caught early and addressed quickly.

If you know where the risks lie, you can monitor those areas and be ready to act quickly. Simply calling attention to a problem often motivates the team, and anyone supporting the team, to fix it.

The unexpected happens. Be ready. (Maybe I should buy a portable generator — or some extra blankets.)

Don’t Just Respond to Change, Enable It

Being agile is not about responding to change. It’s about enabling change. I know; it sounds like a semantic debate. What’s the difference and why should you care?

There’s a world of difference between responding and enabling.

When you enable someone you give them the means or the power to do something. When you simply respond, you are reacting to something.

We think about software development teams responding to change. Change comes from the business stakeholders. As their needs change, as the competition changes, as the business evolves, changes are imposed on the developers. They must respond. But responding is no longer enough. The business needs to know that they can affect change at any time in the development process — and without warning.

Consider a scenario like this. A business stakeholder says, “… We needed feature X but now we don’t. We need feature Y instead and in the same timeframe we’ve discussed”. If the team is following a variant of waterfall development, this change will have to be vetted; a change request will be written, submitted and approved; an impact analysis will occur; finally, the change will get incorporated into the project plan.

Conversely, if the team is following an agile development variant like Scrum, Kanban, XP or Lean, the change will be added to the backlog and incorporated into the next cycle (sprint or iteration). If the team happens to be implementing “X” in the current sprint, the sprint could be cancelled immediately, but this is rarely required.

Don’t protect the process; protect the business.

In both cases, the teams are responding to change in a manner consistent with their development processes. However, in the waterfall case, the change process is cumbersome, lengthy and designed to protect the development process. In the agile case, the change process is integrated into the development process and operates seamlessly.

The agile team has enabled the business to change anything at any time and have the change incorporated into the next development cycle. If the cycles are short (I always recommend 2-week cycles at most and 1 week is better), the business may see some aspect of “Y” within a few weeks. (Waterfall teams often require a few weeks just to process the change request.)

Beware the downside

Be aware that there is a potential downside to enabling change. The development team may find itself handling many more change requests. If the business folks embrace agile development and actively participate, they will change their minds or even envision functionality that they never would have thought about prior to seeing the software in action.

Is that bad? I think not. Henry Ford once said “If I had asked people what they wanted, they would have said faster horses.” Of course they would. People can only envision what they know. Once they see something new and better, they want more of it. It’s our job as technologists to give it to them!

Scrum Works With Virtual Teams Too

One frequently stated “rule” of Scrum that I don’t accept is co-location — having the entire software development team in one place. The arguments are that co-location facilitates communication, speeds interaction, spurs impromptu discussion, and builds camaraderie. I suppose all of those things are true to some degree but it’s not as simple as having everyone together.

The environment matters

If you’re serious about co-location, the working environment becomes a big factor. Spreading the team around the building, even just a few cubical rows apart, dampens the advantages. In fact, six-foot cubicle walls counteract much of the claimed co-location advantage, even when the team sits in close proximity.

You could try having everyone sit on a big room, perhaps around a table or lined up along the walls. This eliminates the physical barriers but introduces the problem of noise — distractions. People will often wear headphones and listen to music to evade the noise — not exactly a useful approach to team building.

Major changes to any work environment will quickly translate into tens of thousands of dollars. That kind of expense will likely be difficult to justify as the productivity benefits are nebulous at best.

The virtual team concept can work

Given the physical, monetary and emotional obstacles to co-location, why not just leave people where they are? You’ll have the added advantage of being able to hire anyone in the world to be part of the team. (Beware of substantial time-zone differences, however, as these will introduce an additional layer of complexity.)

Making this work requires a focus on the concept of a virtual team. You’ll need the appropriate tools to be successful but chances are you already have some of these available:

• Conference call service (try a free service like freeconferencecall.com)
• Video conferencing (like Skype)
• Shared desktop software (like GoToMyPC or Webex)
• Wiki tools (like DokuWiki or SharePoint)
• Instant messaging (like Pidgin or Trillian)
• Shared documents (like Google Docs)
• eMail (but, of course, you have that already!)

The goal is to create an environment where anyone on the development team can engage any other team member at any time — just like being there. Also consider allowing phone calls, chats and conferences to be recorded as long as every participant agrees. This can be particularly useful when someone is diagramming at the whiteboard (or on their PC) and you want to capture the drawing and associated dialog.

Virtual software development teams really work and the cost savings and/or productivity improvements can be significant. Having the liberty to hire anyone, anywhere can speed the hiring process and accelerate projects. Giving the team some flexibility to work from home can be a great morale booster too.

Try to get the entire team together periodically; perhaps for release planning or major test cycles. Also, treat everyone equally. If you offer any perks to the team members stationed locally (e.g. free breakfast), be sure to offer something of comparable value to the remote members.

Software development teams don’t need to be co-located to be agile. It helps, but it’s not required.

Fix One Thing – Just One

Projects often suffer from the same old issues time after time.

Fix one thing. Easy to say. Hard to do.

I see project after project encounter the same old problems. Do you? Something on the following list perhaps?

• Lengthy requirements documents that few people read.
• Functional specifications that simply repeat the requirements.
• User stories that are too vague to validate and test.
• Use cases that only discuss a single “sunny day” scenario.
• Technical specifications that are mostly boilerplate.
• Source code that is poorly written and hard to follow.
• Test plans that are written like best practices documentation.
• Software systems that are full of defects seen before.
• Software applications that are just plain ugly.

I could go on but you get the idea. The same old problems, issues and complaints, over and over again.

Instead of putting up with such insanity, why not just fix one thing? Not everything. One thing. Pick one item on the list and fix it. Get a small group of coworkers together who share your concern. Formulate a plan to address the item.

Map out a plan to incrementally improve the situation. You don’t need to go from insanity to competence overnight.

Set goals that challenge the organization. What result do you want to achieve? Stay focused on the root problem. Don’t try to do too much. But, do stretch! Seek an optimal solution –  something grand.

Now define a few intermediate states that take the organization from the status quo to the goal. It should take a few software releases to reach the goal. That’s okay.

As long as the group you’ve assembled sticks together, you can make this work. You can drive change through the organization — and keep your sanity.

Overcoming the Top 5 Objections to Open-Source Adoption

There are significant objections to overcome before open-source software can be broadly adopted across an enterprise. These issues aren’t insurmountable, but they need to be adequately addressed before open-source can go head to head with proprietary software. Let’s explore the major objections:

• Support Availability
• Functional Limitations
• Software License Terms
• Rapid Software Release Cycles
• Package Roadmaps or Future Plans

These concerns have merit but are often overblown by the commercial vendors. Any major software rollout incurs risks. You need to understand the risks and plan for dealing with them. In the end, open-source involves no more risk than proprietary software.

Support Availability

Support availability, or lack of it, is the most often cited objection to adopting open-source, yet the number of support options is large. Hewlett-Packard, IBM, Novell, Oracle, Red Hat, and many others actively promote and support open-source software. Round-the-clock response is available from these vendors and others.

When you’re selecting an open-source package, look for a history of stable releases, multiple books published about the software, availability of foreign-language versions, active online forums and the existence of support options through a variety of vendors. These characteristics indicate that the software is well-known and widely used.

Functional Limitations

Functional limitations come into play whenever a company migrates from one software package to another. Many open-source packages don’t have the full suite of functionality offered by commercial equivalents. These deficiencies are being erased over time, but in the short run, they may present a challenge. Users are reluctant to give up features, even those they never use (but might need someday).

Software migrations are never easy. The key is to stay focused on core business objectives and not get sidetracked by useless bells and whistles. Such features result in needless complexity, generate support problems and introduce vulnerabilities.

Software License Terms

Software license terms have undergone close scrutiny since SCO sued IBM in a last-ditch attempt to avoid insignificance. Just as every software vendor has unique licensing terms, open-source packages do to. The most common licenses are the Apache License, the BSD License, the GNU General Public License and the Mozilla License. These are widely known and respected. Many open-source communities adopt one of these licenses.

There are two minefields to be aware of when it comes to licensing. First, when incorporating an open-source package into a commercial product, restrictions on how the product is distributed may apply. Second, when using or modifying portions of the open-source code, certain requirements may have to be met that could jeopardize your intellectual property. Read the fine print and seek legal advice in these cases.

Rapid Software Release Cycles

Rapid software release cycles concern many IT departments. It’s tough enough to deal with daily antivirus updates, weekly security updates and monthly Windows updates. Some open-source communities release feature updates on an irregular but frequent basis.

While it’s impossible to control software update frequency, not every update has to be installed. Review the changes in each release and decide what ‘s critical or valuable. Only perform an upgrade when enough value is present in a release to make it worth your while.

Package Roadmaps or Future Plans

Package roadmaps or future plans are important to most companies. Major vendors tend to heavily promote their roadmaps, even to the extent of publicizing future capabilities years in advance. Of course, there is no promise that any advertised feature will ever see the light of your computer display.

Some open-source groups publish roadmaps too. When any particular feature will appear is anyone’s guess. The best advice is to make decisions based on what you can see and touch. If a feature doesn’t exist, assume it never will, even if it shows up on a roadmap or vendor presentation.

With all these potential drawbacks and pitfalls, why would anyone consider using an open-source package versus buying a proprietary product? Ultimately, it’s not about cost, so forget all those total cost of ownership arguments. It’s about value and free-market choices. With any software acquisition, evaluate needs, explore options and select the best fit. Think of open-source as buying software from a small supplier. There may be additional risks, but the rewards can make it worthwhile.

Software Project Teams Always Face Budget Pressures

Software development teams are constantly under budget pressure regardless of their approach to building software systems. Saving money in areas that don’t impact project outcomes will free up funds that can be used on what really matters.

Despite all the attention to reducing costs in recent years, there are hidden budget busters in every technology department. If you can get these under control, you might free up enough money for that new server or software upgrade. Here are a few ideas.

Eliminate Shelfware

Start by looking for shelfware. You may be paying for software licenses that are unused. At times, a software package is replaced yet the license keeps being renewed year after year until someone notices.

Another type of shelfware occurs when a license is purchased for a large number of users but only a fraction of them actually use it. Perhaps some people tried the application but decided to revert to whatever they did previously.

Shelfware can also manifest itself in the form a license that is too broad. You may have purchased the right to use all the modules in a software package when in fact only a few are needed.

Re-examine software purchase decisions yearly. As the business evolves, so will your needs.

Another way to drive down software licensing costs is to go open source. While not a magic bullet, you can achieve significant savings. Start with infrastructure components like Linux, Apache and PostgreSQL. Save the desktop components until you get comfortable with open source models.

Constrain Service Level Agreements

In a similar vein, examine service-level agreements across software, hardware and services. The no-brainer option is to pay for 24/7/365 support with 2-4 hour response time. This type of agreement makes life easy – and expensive.

In a perfect world, all our systems would provide us with 5-nines (99.999%) uptime. The world isn’t perfect. Do all of your systems have to provide such a high level of service?

Find out what the team really needs and structure your SLAs appropriately. Review the recent history of incidents. Troublesome systems may require a higher level of service although the vendor should be pressured to stabilize them.

Perhaps some service issues could be managed better and faster internally. Think about creating a system continuity plan. If a non-critical system malfunctions, what could you do internally? Is there a backup system, a cloud-based solution, or some other way to temporarily work around the problem?

Obviously, the cost of any internal approach needs to be balanced against the cost savings of a reduced SLA. The key point is to evaluate alternatives and not simply select the most convenient option.

Winnow the Project Portfolio

Lastly, take a hard look at that project portfolio. Many software development groups are working on too many projects with too few people and too little money. Prioritize. Focus on what really matters and table the rest for another day.

When you undertake a new project, pay attention. Establish frequent milestones and checkpoints. If a project has spent 50% of its allocated time and budget but is only 25% done, it’s in serious trouble.

You may be told that things will get better and lost time will be made up. In reality, projects that start off running behind, almost always continue to run farther and farther behind.

Don’t let any project turn into a money pit. Kill it. This will give you a chance to re-group, re-focus and re-prioritize. Start again – reuse as much of the killed project as you can. You will anger some folks but you will send a strong message about commitment and accountability.

Money is always tight. Cut the waste and spend more project dollars on what really matters.

Agile Development Can Deliver Software Faster, If You Let it

Can you complete a software project faster using an agile approach?

That’s the wrong question to be asking. Let’s start by defining what it means to “complete” a software project?

1. Deliver all of the initially agreed upon functionality? – That’s not enough.
2. #1 and deliver a high-quality, robust solution? – Still not enough.
3. #1 and #2 including all of the high-priority changes requested during the project? – Nope, not yet.
4. #1, #2 and #3 while delighting the users with the functionality, ease of use and speed of the software. – Now it’s truly complete!

Such a rigorous definition of complete means that it can be difficult to know when the project is really done. Simply meeting time and budget goals is not good enough. The project isn’t done until the real users and business stakeholders say it’s done. Don’t even think about playing the user-acceptance-test game. Just because you delivered what the documents defined, doesn’t mean the real users will be delighted.

Agile deliver faster — over time.

Can an agile development approach like Scrum, Kanban, Lean or XP get your team to #4 faster? I believe the answer is a qualified yes, and here’s what I mean.

Traditional waterfall projects simply don’t engage the real users often enough or deeply enough to learn from them. We gather requirements, write documents (lots of documents), develop the software, conduct carefully choreographed demos, test the software, deploy it and declare victory. By the time real users get to do real work with the software, the development team has moved on. Now there will be an excruciatingly long wait until a new project is started to correct problems or shortcomings the real users will invariably find. That’s wrong!

We need to deliver less functionality with better quality and do it sooner. Get real user feedback early and often. Keep delivering a little more, a little faster. Get more feedback and repeat. Do this and you’ll get to #4 faster.

Don’t give up too soon.

However, be aware that it won’t happen during your first agile project. The development team and the organization will need time to master agile techniques. Don’t call agile development a failure after just one project. Give it time.

There will be administrative and cultural impediments to adopting any agile approach. Many business stakeholders are skeptical when it comes to the ability of software development teams to deliver anything of value in any reasonable time period. Such skepticism won’t evaporate overnight.

Keep delivering new functionality in small increments. Keep engaging the stakeholders. Keep improving quality.

Delivering faster will happen in time, if you let it.

To Deliver Quality Software, Do These Two Things

If your agile development team wants to deliver high-quality software applications, there are two things the team must do:

1. Automate the regression testing effort
2. Get real users involved in acceptance testing early

(Okay, there are lots of other things as well, but let’s focus on these two.)

Automate the regression testing effort

Face it — regression testing can be boring. How many times does a portion of the application have to be tested? Ideally, everything should be retested at the end of every sprint or iteration. If you have 10 sprints, you have 10 test cycles. Testing new features is fun. Retesting existing features over and over is not.

If you run with 1-2 week sprints, which I recommend, you’ll be doing a lot of retesting. After a while, any of us will begin to see what we expect to see. We may blow right by an error message as we race through menus and dialog boxes to complete the next test.

The problem many agile teams run into is that automating the regression tests can be complex and time-consuming. If your test software is too complex, you’ll spend large amounts of time testing and debugging the test code.

Get real users involved in acceptance testing early

This one should be obvious. No matter how good a job the QA engineers do, there is simply no substitute for real user feedback. They are familiar with the business processes. They know what the software needs to do. They understand the data characteristics.

Don’t fall into the trap of delivering what the users asked for but not what they really need. Engage them as early and often as possible.

Here are a few more things to keep in mind as you figure out how to test the software:

• Keep manual and automated tests simple, clear and repeatable, so they are easy to maintain and enhance.
• Make the test results concise and obvious; they should not be subject to human interpretation.
• Take advantage of automated test tools to minimize custom test-software development.
• Invest the most testing effort in areas of the software that are most complex.
• Strike a balance between manual and automated testing; don’t blindly automate everything.
• Aim for a minimum of 75-80% coverage in your automated tests; 100% is rarely needed.

Software testing may not be glamorous but it’s absolutely essential. Cut it short or skip it entirely and you’ll hear about it — from your soon-to-be-former customers.

It’s Time to Reboot the Relationship Between IT and the Business

What am I going to get and when am I going to get it? That’s what every business stakeholder of a software project wants to know. Once they know those answers, the project cost is a simple calculation.

You have three choices for answering the questions:

1. Conduct a detailed and thorough analysis to arrive at defendable conclusions;
2. Conduct a less than thorough analysis and add plenty of padding to cover yourself;
3. Tell the stakeholders what they want to hear and hope for the best.

It’s unlikely that you’ll have enough time for #1. It requires a substantial commitment by the stakeholders and they have a business to run. Estimating project size and building software is your job. (Note: #1 also requires that a crystal ball be available to anticipate future needs. Let me know how that goes!)

#2 may actually work if you have enough experience to know what you don’t know. The problem inherent in #2 is that it just doesn’t feel right. You’re providing a firm commitment around a set of vague definitions. Are you feeling lucky?

It may seem like #3 is there just for fun. It’s not. Stakeholders often demand that software features be developed by a fixed deadline. They are usually quite adamant and not open to negotiation. Just do whatever it takes … or else!

Stakeholders have been asking questions about features and time frames for years on project after project and have received acceptable answers (often after some give and take). However, they seem to forget that actual project results often differ substantially from what was committed. Deadlines are extended. Features are missing. Defects are rampant. Budgets are blown. Credibility is lost.

I’ve seen projects, estimated at 3 months, take 12 months. I’ve seen projects deliver on time yet be so full of defects as to be unusable. I’ve also seen projects deliver near on time and full featured though the end result was hardly optimal.

Time to reboot the relationship

Given such dismal results, why do stakeholders continue to ask the same old questions? Even worse, why do they continue to accept the answers they get?

It’s time to re-define the relationship between business stakeholders and software developers. Playing games built upon complex change control mechanisms (required for #1) is wasteful. Padding estimates (for #2) is unethical. Coercion (#3) is a dumb tactic.

Relationships succeed when they are built upon respect and trust. Those that aren’t fail and take down entire companies with them.

So the next time you’re asked to commit to a feature list and a time frame without enough information to do either, educate the requester as to what is needed to provide such estimates. Then suggest a collaborative, agile approach instead (e.g. Scrum, Kanban, Lean or XP), whereby software is delivered incrementally and stakeholders have ample opportunity to ask questions and offer feedback.

If that doesn’t work, consider finding another place to work. After all, who wants to work on a doomed software project?

10 Reasons to Not Define Requirements Up Front

Requirements — we spend more time on requirements than any other single area of application development. (Note: Some poorly-managed projects spend the most time in software quality assurance. I hope yours don’t.)

Why do we spend so much time on requirements? Simply because if we don’t get the requirements right, the project is doomed. The team will either deliver the wrong thing or not deliver something needed. Both are equally bad.

How do we define the requirements?

• Traditional Business Requirements Documents – often used in waterfall projects.
• Use Cases – often used in projects employing object modeling.
• User Stories – often used in agile projects following Scrum, Kanban or XP.
• Informal Discussion – often used in unstructured projects

Regardless of the final result, there are several techniques for capturing the requirements:

• Individual interviews
• Group discussions
• Existing application as an example
• Wireframes or other mockups
• Idea submission, review and modification

The ultimate goal of all of these approaches is gaining agreement on what the software needs to do. There has to be a common understanding of what the software receives as inputs and what it will produce as outputs. In many cases, this is tough to do at the outset as intimate knowledge of inputs and outputs often requires time for detailed design.

Just as important is how the user interacts with the software. What does the user see? What does he do? How can he find, display, edit and report the data? This can even tougher as the users often don’t know what they really want until they see something they can grasp and critique it.

Requirements Capture Is a High-Risk Endeavor

For these reasons, capturing, documenting and freezing detailed requirements at the outset of a software project is a high-risk endeavor. Yet, we continue to do it — over and over again. The next time your business users demand fully documented requirements before development begins, educate them on the following change risks.

1. Business needs change rapidly as strategic directions are fine-tuned.
2. Competitive announcements may force an unplanned response.
3. Regulatory changes may require software adjustments.
4. Personnel changes may affect expectations of the software.
5. Changes in technology may force software updates.
6. Adjustments to help desk and general maintenance procedures may impact the software.
7. Marketplace changes may affect customer demands of the software.
8. Business process changes may require a response from the software.
9. Budget cuts could impact the project.
10. Misunderstandings or miscommunications may need correction.

You get the idea. What are the chances that none of the above will happen before your project ends?

What’s the alternative? Keep the requirements high-level at the outset. Postpone the final details for as long as possible. Deliver the software incrementally and use the feedback received to guide future requirements. Be agile.