An Architecture for Dynamic Data Source Integration

Integrating multiple heterogeneous data sources in to applications is a time-consuming, costly and error-prone engineering task. Relatively mature technologies exist that make integration tractable from an engineering perspective. These technologies however have many limitations, and hence present opportunities for breakthrough research. This paper briefly describes some of these limitations. It then provides an overview of the Data Concierge research project and prototype that is attempting to provide solutions to some of these limitations. The paper focuses on the core architecture and mechanisms in the Data Concierge for dynamically attaching to a previously unidentified source of information. The generic API supported by the Data Concierge is described, along with the architecture and prototype tools for describing the meta-data necessary to facilitate dynamic integration. In addition, we describe the outstanding challenges that remain to be solved before the Data Concierge concept can be realized.