Since the establishment of software engineering, any non-trivial software system had to be divided in parts that could be assigned to separate teams. During the development of the system, the teams interact in order to manage the dependencies between the parts. Over time, the system tends to age and develop more and more implicit dependencies between the components. This, in turn, leads to reduced predictability of development as the dependencies between the components easily cause problems during the final, integration phase of the development process.
Over time, the integration phase has grown consistently in most development organizations, to the point that it can take up to 80% of the total development effort. During the last years, the broad adoption of software product lines, the emergence of global software engineering and, more recently, software ecosystems is causing yet another order of magnitude of complexity in terms of the dependencies between the components.
The consequences are that obvious in many software development organizations. Coordination overhead in terms of time spent in meetings for requirements management and roadmapping, alignment between teams during development, integrating the various components at the end of the cycle and the management of post-deployment issues. Other symptoms include the large and growing teams assigned to key components, the inverse resource allocation where the largest amounts of resources are assigned to components that have little or no relevance for customers, the constantly increasing cycle length of releases and the resistance against replacing non-differentiating internal components with open-source or commercial components.
Over the last five years in industry, I have become increasingly convinced that the integration-centric approach to software development has reached the end of the road and that we need a fundamentally new approach to large scale software development, i.e. composition-oriented software engineering.