Gigabit Concept Mining: A Sensitivity Analysis, Masters Thesis, December 2006

GIGABIT CONCEPT MINING: A SENSITIVITY ANALYSIS by Andrew A. Levine ADVISOR: Prof. R. P. Loui and Prof. John W. Lockwood December 2006 Saint Louis, Missouri Massive amounts of data are passed over public networks. There is a need for network administrators to analyze this traffic, but it was not previously possible to analyze live network data at high speed. It has been shown that streaming computation and deep packet analysis are possible at very high rates through the use of hardware acceleration. This work provides analysis for a larger project that involves digesting large amounts of network traffic. In this system, we process the traffic using hardware that has constraints. The workings of the system are first discussed. Tradeoffs in the design of hardware and software components are also discussed. Next, an experiment to classify topics of newsgroups is described that utilizes the system. The contribution of this thesis is to show that it is possible to change the parameters of the system to minimize the representation of concepts. To My Mother Pat Levine Dedication to my mother Pat Levine and my family. I dedicate this document to my mother Pat Levine and my family. I want to thank them for all the free meals and support. Without them, I would not have been able to complete my degree.