How well can we predict changes at architecture design time

Two years ago, we analyzed the architecture of Sagitta 2000/SD, a large business information system being developed on behalf of Dutch Customs. We were in particular interested in assessing the capabilities of the system to accommodate future complex changes. We asked stakeholders to bring forward possible changes to the system, and next investigated how these changes would affect the software architecture. Since then, the system has been implemented and used, and actual modifications have been proposed and realized. We studied all 117 change requests submitted since our initial analysis. The present paper addresses how well we have been able to predict complex changes during our initial analysis, and how and to what extent the process to elicit and assess the impact of such changes might be improved. This study suggests that architecture analysis can be improved if we explicitly challenge the initial requirements. The study also hints at some fundamental limitations of this type of analysis: (1) fundamental modifiabilityrelated decisions need not be visible in the documentation available, (2) the actual evolution of a system remains, to a large extent, unpredictable and (3) some changes concern complex components, and this complexity might not be known at the architecture level, and/or be unavoidable.