Nightmare of component based development (1)

At one of the companies where I work, they have introduced component-based development. The total system is divided into subsystems and each subsystem is divided into components: (1) the subsystem's external interface and (2) the subsystem's internal "body". The project is divided in a number of sub-projects, each responsible for a number of subsystems.

After several months of experience with this approach, we do not seem to get the integration process right. The primary reason is that each subsystem is developed against a "stable" system baseline, where the system baseline is the integration (or composition) of individual subsystem baselines. But each of the subsystem is evolving in parallel with the other subsystems, so when trying to integrate newer baselines of all subsystems, the configuration is a composition of subsystem versions that have never worked together before.

Some consequences:

• Build failure
• Smoke test failure (verifying only basic functionality)
• Regression test failure (verifying former functionality, but not new functionality)
• Integration test failure (verifying new functionality in an integrated environment)

As a result, subsystems must be rebuilt against different configurations and possibly need some extra modifications. Even more, those small modifications may (and do) result in additional inconsistencies, some formerly obscured by other problems, some introduced by the new modifications. This causes delays in the system integration. Consequently, the next "stable" system baseline is delayed and consequently the subsystems can not develop against it (unless they develop against an "instable" configuration).

Our struggle is how to get this right, so if you have ideas then do not hesitate to give me some suggestions. I will follow up on this subjects in my blog.