Integrating Data Mining with Relational DBMS: A Tightly-Coupled Approach

Data mining is rapidly finding its way into mainstream computing. The development of generic methods such as itemset counting has opened the area to academic inquiry and has resulted in a large harvest of research results. While the mined datasets are often in relational format, most mining systems do not use relational DBMS. Thus, they miss the opportunity to leverage the database technology developed in the last couple of decades. In this paper, we propose a data mining architecture, based on the query flock framework, that is tightly-coupled with RDBMS. To achieve optimal performance we transform a complex data mining query into a sequence of simpler queries that can be executed efficiently at the DBMS. We present a class of levelwise algorithms that generate such transformations for a large class of data mining queries. We also present some experimental results that validate the viability of our approach.