Death by architecture

One of the less mentioned dangers of a waterfall or engineer type approache to software development is the focus on the initial activity of defining and implementation the architecture of the system.

Usually the critic of the 'old' plan-drive methologies runs along the lines of inflexibility and inability to cope with the unknowns of real world software development. This issue is often adressed in 'mature' organizations by assembling a group of experts in the discipline of IT system construction (IT architects), and tasking them with defining the high level technical aspect of the system.

One of first task addressed by the architects is usually to define the non-functional requirements, like performance, availability, scalability, maintainability and so on. These input are used to define the architecture of the system. this includes selecting the technologies to use for building the system. The architecture will often include such things as layer models, security models, persistence models, domain model and a lot of other models for crosscutting concerns.

If the company defining the architecture also provides the platforms for deploying the system the architect will also be tempted to include a lot of enabling technologies in the architecture. This will include a potent cocktail of enterprise, high-availability, cluster, SOA...... middleware, platforms and frameworks which would (in theory) solve most of the non-functional challenges.

This all sounds very clever, but this approach can in the real world be devastation to a project progress. The problem is that the initial focus of the development project will be centered on creating the system architecture, instead of producing visible progress.

If you are lucky you will have access to people understanding the technologies making up the architecture. But more often you will a team with a more generic competence profile, and all the middleware and enterprise platform will remain badly understood black boxes, continually impeding the development of the system, while the promise result gained by using ready-made enterprise software never materializes.

During this phase the trust between the customer and the software development team will deteriorate, because the development team is unable to demonstrate any progress and and understand of the problem domain will not increase significantly. The moral of the development team will also go down, because the members are also unable to see, where the project is going.

Many project have never really gotten through this phase, and have died struggling to define and implement the 'right' architect. If you are able to fight yourself out of the architecture desert, the scars on the customer relationship and the team moral will quite deep and make the work ahead more difficult.

So, how should one avoid this?

As always, the solution is to use a more agile approach. This include following the agile best-practices of:

• Keep it simple: Do not introduce complications needed because of non-functional requirement, until you can't postpone them any more.
  
• Make it work, make it right, make it perform: Avoid focusing on concerns, which wouldn't have an effect until later in the project. You should off course not totally ignore later stage challeges, but the development should always be driven by the shortterm gains.
• Evolutionary development: Grow the system from a simple seed, nursing and growing the application on a continuous basis.