Adaptive Learning Models Evaluation in Twitter’s Timelines

Current challenges in machine learning include dealing with temporal data streams, drift and non-stationary scenarios, often with text data, whether in social networks or in business systems. This dynamic nature tends to limit the performance of traditional static learning models and dynamic learning strategies must be put forward. However, acquiring the performance of those strategies is not a straightforward issue, as sample’s dependency undermines the use of validation techniques, like crossvalidation. In this paper we propose to use the McNemar’s test to compare two distinct approaches that tackle adaptive learning in dynamic environments, namely DARK (Drift Adaptive Retain Knowledge) and Learn++. NSE (Learn++ for Non-Stationary Environments). The validation is based on a Twitter case study benchmark constructed using the DOTS (Drift Oriented Tool System) dataset generator. The results obtained demonstrate the usefulness and adequacy of using McNemar’s statistical test in dynamic environments where time is crucial for the learning algorithm.