Agile Development Doesn’t Neglect System Architecture, But You Might

Agile approaches to software development are often criticized for failing to visualize the big picture from which a robust software architecture can emerge. When you are focused on small stories, short time frames, and quick deliveries, broad-based system and user design issues can easily be neglected.

But, is this a fundamental weakness of agile teams? Not necessarily. Here are a few situations where visualizing the software architecture should not be a problem:

• The software application is small in scope. The team should be able to grasp the entire system and lay it out on a whiteboard in a single design session.
• The software system is very much like something that already exists. The team can analyze or reverse engineer what exists to develop a complete understanding of the new solution space.
• The software is being built with standard components (usually based on a design pattern). Assuming the components have been tested and hardened, many architectural and deployment issues are muted.
• The team has many years of experience building solutions like the one under development. They are aware of the risks and know how to approach them.

None of the above situations will result in a project that’s easy. A project meeting such criteria will simply experience fewer and smaller risks.

Now it gets complicated.

What’s left? How about a large project, being developed mostly from custom modules, by a team with limited experience in this type of system?

Let’s agree that this type of software project will be difficult regardless of the methodology followed. Any approach can work but all approaches have an intrinsic failure risk that may not be acceptable to some teams.

For example, a waterfall approach may get bogged down in analysis. Once passed the analysis phase, the team may have to stop and regroup periodically as they learn more and realize what they missed.

An agile approach may result in rapid progress early on. However, progress could slow markedly as the system grows, defects emerge, and the backlog becomes difficult to manage.

Despite the pitfalls, I’d strongly recommend an agile approach such as Scrum, XP or Kanban especially when there are many unknowns. Agile teams learn by doing not by analyzing. Here are a few recommendations you should consider to manage the risks:

1. Know what you don’t know. That is, make a list of problems you foresee needing to be solved even though you don’t know how to solve them yet.
2. Tackle the most difficult problems as early as is practical. Don’t put them off until you have a small-scale solution built. You could be making decisions that will have to be reworked.
3. Think components. Split off software modules where feasible. Assemble separate development teams to build the modules if you can. There will be module integration issues but the overall development pace will be faster.
4. Keep your stories in epic form early on. Don’t get bogged down in defining a large number of small stories. Group the epics into themes so you can visualize how the system fits together functionally. (Build a hierarchy of ‘themes -> epics -> stories’ into your story backlog.)
5. Perform in-depth data analysis early on. Acquire some real-world data and compare it to your understanding of how the system will work. If real data is not available, simulate what you expect the data to look like. Your data model will be a critical element in building a robust solution.

Systems built using agile software development can be just as architecturally sound as those built using any other approach. Agile approaches to system development use both top down and bottom up design techniques. It’s a combination that’s proven to work.