Conceptual integrity

Most, if not all, of the software developers I know, when hearing the name Fred Brooks, immediately, and correctly, identify him as the author of 'The Mythical Man Month'. Of those only relatively few understand that the mythical man month was, albeit incredibly important, only one of several ideas presented in that legendary tome.

Another phenomenally important idea, described by Brooks as Paramount in System design, is Conceptual Integrity, concretely described as a system that reflects a single set of design ideas, as opposed to a collection of independent and uncoordinated ideas.

Conceptual integrity can only be maintained by separating architecture from implementation, that is by ensuring that architectural decisions are made by a small number of people, preferably a single architect, with the authority to approve or deny the inclusion of features if they do not fit seamlessly with the system architecture.

I think anyone, with an IQ higher than their European shoe size, recognizes this as common sense; which begs the question, why do so few systems exhibit this quality?

It all comes down to a lack of authority and the behaviour of inexperienced or perhaps arrogant developers in such an environment.

Having been asked to design a system, one expects to evolve the design as new features are added and deployment environments change, but, it is imperative that the architect have the authority to deny or defer a feature until the system can accommodate it, it is the visibility of these features and environmental conditions that allow the system to evolve in a controlled and consistent manner.

If the authority of the architect is circumvented, the conceptual integrity of the system will inevitably fail and ultimately the system will become intractable and a candidate for yet another rewrite.

Without explicit authority, it is common for the initial design to be seen as the foundation upon which the house is to be built, a static, solid base, never to be seen again. The application is seen as incidental to the system, not part of it.

How does this happen? As you would expect, it comes down to human weakness, lets take a look at a couple of scenarios which I have experienced.

The technical director
In some cases there exists, in the management chain, a beast, the beast will appear generous and the beast will cajole with soft words and mighty bonus checks, but the beast has a secret! The beast used to be you, (s)he used to be a developer, probably a damn good one, but it is this very background that persuades the beast that (s)he understands the system and what it can or can not tolerate, combined with the fact that the beasts authority supersedes the architects and the decent into hell begins. If the beast is amongst you there is very little you can do but look forward to the next rewrite. Note, not all technical managers are the beast, but the beast dwells within them.

Look before you leap
In some cases, the conceptual integrity of a system can be undone by a developer who refuses to become familiar with the system. For instance, if a particular subsystem does not exactly match the needs of the developer it will be abandoned in favour of new code, specific to task. The developer delivers the feature on time, management is happy but the system is now less than it was.

Look what I can do
We have all said or thought, at one time or another, wouldn't it be awesome if the system did THIS.

THIS being some function that is not part of the design but, to you at least, would make the product better.

The right thing to do with THIS is submit it as a feature request, and hope it makes it into the system.

I am not saying that you can not engage in skunkworks projects but unless you have enough experience to ensure that the feature is implemented in a manner that is consistent with the system design, you can guarantee that your must have feature will be the source of significant bugs and instability.

Whether these activities are motivated by a genuine desire to make the system all it can be or they are the arrogant actions of a DS watching* scum-bag, the potential for drama is the same. If you are going to engage on little side projects, it might be worth having a quick chat with the architect about your plans.

* DS Watcher -- Military phrase, implying that a soldier only puts in effort when he knows that his superiors can see him. Commonly found bleeding in the shower block.

The next time you look at a code base and wring your hands, furrow your brow and vilify the original development team, try and remember that the system probably started life with conceptual integrity and was undone by ignorance of that quality; then modify your own behaviour accordingly.