Managers Discover the Easy Button for Software Development

Don’t you wish that headline was true? It’s not. There is no easy button for enterprise software development.

Building software applications is a dichotomy (or worse) when it comes to the time required. Building a small standalone software application with a narrow scope is easy. Many of us have done it — often in a day or two. Add a few complex features and it may take a week or two. Not bad.

Add more complexity, import/export functions, integration with other software and you could be looking at several weeks of development. Now expand into enterprise-scale databases, large user populations, and military-grade security. Such projects may require multiple development teams and months of calendar time. Not easy.

The effort required grows exponentially as features are added, data volumes increase, and security concerns grow. That’s why context is so important when discussing various approaches to managing projects. Those who adamantly advocate specific techniques are always referring to a specific context. What works for them, may not work for you.

Just make it work.

Non-technical, business people don’t understand the complexities of software development, nor should they. Do you understand what it takes to design and build an automobile? What about a high-rise office building? Most of us don’t know or care. We just want our cars and workplaces to be functional and safe.

Yet, software systems are the nerve center of many businesses that simply could not function effectively without the software they rely upon. The pace of business operations seems to increase every year along with the competitive environment. Because software is so important to business people, they want changes and enhancements at an ever faster pace.

Unfortunately, as software becomes bigger and more complex, the pace of advancement slows. Just the opposite of what’s needed and expected. That’s exactly why start-up, software companies often flourish. They don’t have to manage enormous size and complexity enabling them to move faster and be more responsive.

There’s no easy button.

What can we do to simplify and accelerate enterprise software development? Here are a few ideas. Please contribute your own.

• Employ design patterns that offer reusable solutions to commonly occurring problems. Many of the problems encountered by software development teams have already been solved. Check out the Software design pattern entry in Wikipedia.
• Use a standard web application framework like Model-View-Controller. There are many environment-specific frameworks like Zend (PHP), Spring (Java), Ruby on Rails, and Django (Python). Don’t reinvent the wheel.
• Leverage existing applications as working examples. Chances are a software application already exists that does much of what you want to accomplish. Use it as a reference to jump-start your application design.
• Take advantage of cloud-based tools and services such as software-as-a-Service (SaaS) and Platform-as-a-Service (Paas). Options like Salesforce.com, Amazon Web Services and Google App Engine offer a wealth of tools to speed up web-application development.
• (Lastly) Keep it simple. Start small. Don’t boil the ocean. However you phrase it the key point is to break up the solution space into manageable chunks and add complexity as needed over time.

Now that you know what to do, let me add one thing that you should not do. Avoid reusing old code. It’s tempting to reuse, rework, or re-purpose existing source code for a new application. On the surface, it appears that time and money can be saved by taking advantage of what’s already been done.

In practice, this is notoriously hard to do well. Unless the old code was specifically designed and built for reuse, trying to untangle it and get it to do what you need is often more trouble than it’s worth. You may be able to extract some ideas and approaches from old code but it’s unlikely you’ll benefit from reusing old software components.

Stop Managing. Start Leading. Be Agile.

Managers: Want your company to be more agile? Learn to let go!

Managers are trained to maintain control. Their annual reviews reinforce the need to make good decisions and stay in control. And then along comes agile software development.

There’s an immediate and harsh conflict. Agile teams need to take control. They can’t be agile if they’re simply following orders. That’s much too restrictive.

When I refer to an “agile team”, I’m talking about all the people needed to make the product or software system successful. So it’s not just about software engineers. We need to include real end users, business stakeholders, marketers, UX designers, testers, data center operators, etc. Only multidisciplined teams can be truly agile.

Of course, with so many diverse groups represented on the team, we’ll have many managers involved. There will be one or more managers associated with each specialty group — for example, a group manager, department manager and director. Every group manager will want to control his or her team members. That spells c-o-n-f-l-i-c-t!

Those team members will be torn between allegiance to their team and allegiance to their managers. It’s not right to place them in that position. Let go!

It’s the difference between managing and leading.

Managers seek to control. They establish procedures and protocols. They define rules and systems to maintain order. They are primarily responsible for running the business.

Leaders seek to energize. They create goals and motivate people to reach them. They establish direction, communicate it clearly, and help clear obstacles. They are primarily responsible for aligning people with the needs of the business.

Of course, few, if any, companies refer to their managers as leaders. It’s probably just as well. Titles are rather meaningless, anyway. The bottom line is managers need to stop telling their people what to do and how to do it. Instead, they should ask questions. Listen to the answers. Guide. Motivate. Prioritize.

It’s fine to express concern and even to pitch an idea to the team. But, ultimately the team members need to make decisions guided most directly by the Product Owner and/or key business stakeholders. (Hopefully, those folks are being driven by the end users and strategic business goals!)

Undo Is a Software Feature That Agile Teams Should Embrace

Most software applications include an “undo” feature. It’s great, isn’t it? You can change something and if you don’t like the result, simply undo it. How cool is that!

Wouldn’t it be equally cool if software development teams could implement changes to software applications and if the users don’t like them, undo? This would be a little tougher than just clicking an undo menu item but it’s not that hard — if you plan ahead.

A good version control system like Git or Subversion can help but isn’t essential. If you plan ahead, the undo capability can be fairly simple.

As each sprint begins, examine the sprint backlog and ask the team a question.

“How will we undo this sprint if some or all of these changes flop or are rejected by the business?”

There’s no standard answer. How undo gets done depends on the software implementation. Here are some techniques I use within Salesforce.com. Hopefully, they will give you some ideas for your projects.

1. Create or refresh a sandbox (an isolated development environment). This gives me a snapshot of the production code and the database schema. There’s no ambiguity. I know exactly what I’m working with.
2. Make backup copies of anything you intend to change. Eclipse makes this easy for Salesforce.com whether you use a version control system or not.
3. Implement database changes as needed. I try to avoid changing existing database fields and prefer to insert new ones and copy over the existing data if needed. That way, I reduce the risk of breaking anything. I can always delete unused fields after deployment.
4. Define new VisualForce pages and Apex classes as needed. I usually implement skeletal pages and classes to start. This gives me software components to see and interact with. Then I build out the pages and classes iteratively to provide the features requested.
5. Modify existing VisualForce pages and Apex classes as needed. Making these changes undo-able is trickier. Isolating changes within code modules is always a good idea. (They are easier to remove.) I’ve also used parameters to control whether or not a feature is active (sometimes on a per profile or per user basis). This makes it easier to turn features on or off in production.
6. Finally, create or modify any validation rules or workflow rules associated with the new feature(s). I sometimes include profile or user information within the rules so I can control system behavior based on user privileges.

That’s it. I can make whatever changes I want and back them out quickly and easily, if needed. (Yes, I’ve had to do that on rare occasions.) The key ideas are: 1) isolate changes to the greatest extent possible, and 2) use run-time parameters to control feature activation. Give it a try.

User Stories and Security Stories Are Not the Same

The folks over at the non-profit group, Software Assurance Forum for Excellence in Code (SAFECode) have published a great document called “Practical Security Stories and Security Tasks for Agile Development Environments”. Here is the direct link to the PDF file.

It’s a 34-page document that undoubtedly required a lot of work by a number of people. I applaud them for the effort and thank them for making the document freely available to all of us.

That said, I’d also like to offer an opinion and guideline about the security-focused stories in the report. Here are a couple of story examples from the document:

Story #1: As a(n) architect/developer, I want to ensure AND as QA, I want to verify allocation of resources within limits or throttling

Story #17: As a(n) architect/developer, I want to ensure AND as QA, I want to verify that cross-site scripting attacks are prevented

These are not “User Stories” in the manner intended for use by agile development teams. A user story should focus on a user (or entity) activity and system functionality. It should provide value to the business. Even though these security stories are formatted as user stories, they are not focused on end users and business value. They are focused on the system being built. They are not so much focused on system functions as they are on preventing certain types of (malicious) behaviors.

As such, these security-focused stories would better serve us as acceptance criteria to the typical user stories we define to capture business requirements. The security stories fall into the category of non-functional requirements. They are important to the system under development and to the business but they don’t represent functionality.

Every system we build should be robust, reliable and secure. Oftentimes, end users come to expect such system characteristics without explicitly putting them in writing. It’s our job as subject matter experts to expand upon the user stories provided by the Product Owner and others to include acceptance criteria that can be verified as non-functional requirements.

I encourage you to download and read the SAFEcode document. Use it as a guide when defining, implementing and testing user stories. Security has become a major challenge for all of us and it’s not getting any easier.

Follow a Recipe to Succeed With Agile Software Development

When top consultants engage new clients, they offer ideas and suggestions to get started. A phrase that the consultants listen for from the client is something like “we tried that once before” or “we already did that”. It’s a comment that begs for probing questions. “What exactly did you try?” “What results did you observe?” “How long did you continue to use that approach?”

The reason for the probing questions is simply because people often say they tried an approach or tool but when we dig deeper we find that they really didn’t do what they claim. Or, they measured the wrong outcomes. Or, they didn’t give the effort enough time to show results.

Today, this is happening with agile software development approaches like Scrum, Kanban, Lean and XP.

Every company seems to be claiming to use agile software development — or wants to. Regrettably, the facts don’t support those claims. Many companies transform the recipe for agile development into their own custom “agile” recipe and wonder why they don’t get the results they expect.

Here’s what happens when you tamper with a proven recipe. I’ll use the base ingredients for an angle food cake as an example. (For those of you not familiar with angel food cake, it is a light and sweet cake best served with fruit or chocolate, and whipped cream.)

Base Ingredients for an Angel Food Cake

1 cup cake flour

1 3/4 cups sugar

12 large egg whites

Let’s say Company X decides to make an angel food cake but they want to use ingredients they have readily available. They’d also like to improve the recipe. They substitute whole wheat flour for cake flour because whole wheat flour is higher in fiber and better for you. They substitute a low-calorie sweetener for sugar because the sweetener will appeal to those on a diet. Finally, they substitute 4 whole eggs and 4 egg whites for the 12 egg whites because whole eggs are more nutritious.

The end result may taste okay but it will be nothing like angel food cake. It won’t have the same texture at all. It’ll be heavier and denser with a whole-grain flavor.

The same thing happens when you alter a recipe for agile software development and try to turn it into something you think is better.

• Adding a few agile techniques to waterfall development won’t make you agile.
• Simply holding daily meetings won’t make you agile.
• Pair programming alone won’t make you agile.
• Tracking project status on a whiteboard with sticky notes won’t make you agile.
• Breaking up the project into iterations won’t make you agile.

If you want to be agile you need to follow a standard recipe and give it time to mature. Initially, you may have to make a few short-term compromises. For example, engaging the business stakeholders as active participants may take some time and persistence. That’s okay. Stick with it and keep the final goal in view.

Here are a few agile recipe sites to get you started:

• Agile Alliance
• Extreme Programming
• Lean Software Development
• Limited WIP Society
• Scrum Guide
• ScrumAlliance

Fail to Fail and You’ll Fail to Improve

There are two critically important elements to agile software development that are often either overlooked or ignored.

1. Continuous Improvement
2. A Willingness to Fail

Any team committed to being agile will seek continuous improvement through regular retrospectives. There’s always room for improvement. Who among us believes that he is the best he can be — there’s nothing to improve? [If you find such a person, you likely don't want him on your team.]

What can you improve?

• Quality
• Team Dynamics
• Time to Market
• Total Development Cost
• Miscommunications
• Meeting Durations
• Number of Meetings
• User/Customer/Client Engagement
• Refactoring Efforts
• Technical Debt Load

I could go on but you get the idea. There’s never a shortage of improvements to be made. If you can’t find anything to improve, either you’re not looking hard enough or you’re delusional.

This leads to the next important element – a willingness to fail.

“Fail” might be too strong a word for some as it conjures up the image of a failed project. No one wants their project to fail but that’s not the point at all. The idea is to seek out small failures quickly. After all, big projects fail one small failure at a time — if those small failures go undetected and uncorrected!

Face it, your team is not perfect. Mistakes will be made, poor decisions will surface, and communication errors will occur. Obviously, the team will do it’s best. Obviously, they’ll do everything they can to prevent small failures. But, they won’t be — and can’t be — 100% successful. That’s life.

Expect the unexpected.

Learn to embrace failure as learning experience. The best way to learn is by doing. Take small chances and see what happens. At best, the risk will pay off. At worst, the team will learn and improve.

One aspect of this to be watchful of is repetitive failures. If you see the same type of failure happening over and over again, you have a serious problem. That’s an issue for the next retrospective.

So, being agile means admitting to imperfections and striving to fix them. Some managers have a hard time with this concept. No one wants to report failure to the boss. So we plan, review, adjust, plan, review, approve … and think about it some more. Those cycles could have been used by the team to learn and improve. What a waste.

Are You a Software Development Extremist?

Some of us tend toward extremism. It manifests itself in many ways in both our personal and professional lives. It could appear in the clothes we buy, the cars we drive, the way we work, or the way we look. In many situations, we’re not even aware that we’re extremists.

In software development, there are (at least) two extremes you should avoid.

One extreme is being rigid and inflexible regarding requirements, work plans, schedules, documentation, etc. Everything must be predefined and locked down. A change management (a.k.a. change prevention) process is used to see to it that nothing changes unless several managers agree to the change in writing.

The opposite extreme is letting every aspect of the project float. Nothing is locked down. Anything can change anytime. No single individual is ultimately in charge or accountable. The team wants to be left alone and succeeds or fails as a unit.

Do either of these extremes seem reasonable to you? If so, I’d love to hear from you so that I might better understand why you feel that way. I believe that the vast majority of us prefer to stay away from the extremes and seek out a balanced approach instead.

Because the focus of this blog is agile software development, I’ll direct my thoughts there. I see many definitions of agile development (in the form of Scrum, Kanban, Lean and XP) across the Internet.

Here are a few extreme misconceptions.

1. Agile development lacks controls and commitments — it’s chaotic. No it’s not. It’s fluid. Agile teams commit to deadlines, budgets and quality levels. Software features are allowed to float.
2. Being agile means eliminating project managers, business analysts, and system architects. No it doesn’t. Big projects still need senior people to guide the teams.
3. Agile teams don’t write specifications or documentation. Not true. They don’t waste time writing needless or redundant documents. They write documentation that matters.
4. You need an omnipotent Product Owner to be agile. No, Product Owners don’t need the ability to walk on water. They need to be skilled in solving a mix of technical and business problems.
5. The team has to be co-located to be agile. Just not true. With the right tools and the right management support, distributed teams can be just as agile as any other.
6. The project status board has to be a dedicated whiteboard with sticky notes. Actually, electronic status boards are fine. You simply need to invest in the right equipment and software.
7. Agile development does not work in regulated industries. Wrong. It takes discipline and structure to do agile development well. Those are also the qualities needed by regulated industries.
8. Agile developers are cowboys (or cowgirls). They just want to crank out code all day on their own terms. Nope. Anyone wanting to be a lone ranger should start their own company. Agile developers need to be multidisciplined and team oriented.
9. Agile development can’t be scaled. It’s for small teams. Agile development can be scaled though it’s hard to do well. The key is to start small and scale up using retrospectives as the continuous improvement tool.
10. Software architecture and user experience design are ignored by agile development teams. Ridiculous. Software architecture and UX can be improved through agile techniques. Start simple and add complexity as the software evolves.

Oh, and here’s a bonus extreme misconception for you to ponder.

• You can make your prescriptive (a.k.a. waterfall) approach to software development just like agile development by including a few agile best practices. NOT!

The Advantages of Distributed Software Development Are Enticing

Distributed software development teams are becoming more common. There are some good reasons for it. Here are a few:

• Access to potentially lower cost talent
• Ability to attract highly-specialized and hard-to-find talent
• Outreach to a larger and more diverse labor pool
• Advantages of an extended work day via time differences

The potential benefits are enticing though they come with unique challenges. Software development is hard enough when the entire team is together in one place. Adding the complexity of having team members scattered around the country or around the world only makes it harder.

The traditional approach to this added complexity is to document, approve and control every aspect of the development process. Fight complexity with complexity! This is an administrative and legal solution to a problem that’s founded in people and technical issues. It doesn’t work.

If you have an interest in distributed software development, I invite you to read a guest post I wrote for Colabpro. Agile development techniques reduce risk and simplify the management of distributed software teams. The post is called “How Agile Principles Make Distributed Software Development Better”.

Please take a look and let me know what you think by leaving a comment at Colabpro or here.

User Stories Need To Be Viewed From Multiple Perspectives

What the stakeholders and end users don’t want is just as important as what they do want. This is especially true of upgrades or enhancements to existing systems. It’s common to get caught in the flow of the existing design and to just keep doing more of the same.

Here’s an example.

Let’s say your company has a retail website. Customers register, place orders, track purchases, etc.

The site stores lots of information about customers and tries to make it easy for returning customers to place new orders. The site also displays ads both to entice people into buying products they haven’t tried before and also to show them what other products are available on the site. There are occasional pop-up ads in key situations.

Now the site wants to add a new feature — the ability for returning customers to reorder items that are used regularly such as paper products, toiletries, cleaners, etc. Multiple user stories will be needed to map out all of the features needed, of course.

Subtle differences really matter.

There will be a natural tendency to build the new re-order feature around all the existing operations of the site. Yet, there is a big difference between browsing for products and re-ordering products.

When browsing, shoppers like to see many items in a category and they like to compare and contrast them. Many will appreciate ads and even pop-ups as those elements help them learn more about the products they are interested in. Shopping becomes a learning and entertainment experience.

However, when re-ordering standard consumables like photo paper, toothpaste or hand soap, simplicity and speed are essential. If the shopper wants to browse for equivalent products, the site already lets her do so. If she wants to order something she has ordered before, the software needs to offer a different user experience.

This is where many things can go wrong. The software developers are likely to add the new re-order features to the existing site paradigm. This will include the pop-ups, ads and other design elements built into the site. While technically correct, this will result in a re-ordering process that is slow and cumbersome.

Consider all the angles.

The returning shopper likely wants to see a list of items purchased previously with the ability to select one or more and add them to a new order. As software system experts, we need to avoid the trap of simply doing more of the same. Deliver what the user wants and needs — not more, not less.

Explore new user stories from multiple angles to uncover what really matters. Here’s how:

• In addition to asking what the user wants to accomplish, ask what she doesn’t want.
• Probe for hidden assumptions built into every story. What is the user assuming? What are you assuming?
• Ask for examples of similar functionality. If they don’t exist, go the whiteboard and diagram the story.

Give it try. The key takeaway is to analyze each story from multiple perspectives. The better you understand what’s needed, the better the implementation will be.

Avoid the Extremes of Software Development; They’re Not Agile

Large enterprises have lots of rules — do this, don’t do that, ad nauseum. Thus, it’s no surprise that software development teams in large enterprises are forced to follow lots of rules — document this, approve that, wait for something else.

Then, along come the agilists promising a better way. You can build software systems without all those rules. The work will be done better, faster and cheaper. What’s not to like?

Of course, the defenders of the current organizational structures will resist. They’ll say you can’t build complex software systems in the midst of chaos! There have to be controls, rules, checkpoints, approvals, hand-offs, etc.

As usual, both sides have valid points. Too many controls and excessive oversight increase waste and lengthen time to market. However, eliminating all controls and oversight creates confusion and increases rework. True agile development is about finding the proper balance for your situation.

You need to flow with the rules.

When you review the Agile Manifesto or the Scrum Guide, you won’t find lots of rules. There are mostly principles and guidelines. That’s not a mistake. It’s intentional.

Enterprise software development is a highly complex undertaking. It’s a unique mix of computer and human engineering. Some aspects of it can be governed by rules but other parts need room to flow.

You have three choices:

Option 1: Keep all of the existing corporate rules and controls. Fit agile software development into the existing corporate structures.

Let me be clear. This is a terrible idea! Why bother changing anything? If your goal is to protect what already exists, forget agile development. I don’t want you claiming to be agile and giving the rest of us a bad reputation. Let it be.

Option 2: Toss all the existing rules and controls. Start writing code and see how it goes.

Good luck with that! While working software is the most important measure of progress, getting the software to work requires at least a few control mechanisms. We want agility not chaos. Furthermore, senior managers need some guidance on what will be delivered and when. After all, they are running a business that funds your paycheck!

Option 3: Start simple. Minimize complexity. Add rules and structure as the team matures.

That’s agile! Do a few things well. Add more in time. Assemble the right team, manage the backlogs, meet every day, review each sprint, conduct retrospectives, and agree on a solid “definition of done”. That’s enough to get you started.

Is agile development easier than waterfall development? No! Is it better, faster or cheaper? Better? Yes! Faster? Not initially but it can be if you get good at it. Cheaper? Software development and support costs as well as value to the business are notoriously complex. Your mileage may vary.