LMS Log Data Analysis from Fully-Online Flipped Classrooms: An Exploratory Case Study via Regularization

The COVID-19 pandemic has changed the education system worldwide. Online learning is no longer an option, and an increasing number of online classes incorporate components of flipped classrooms in an effort to improve the quality of learning and instruction. Although students' active involvement in pre-class activities is greatly emphasized as a necessary condition to enhance in-class learning and instruction, it has been a foggy area whether students completed the assigned pre-class activities and whether pre-class activities lead to desired outcomes. This study illustrated an exploratory study of LMS log data from undergraduate fully-online flipped classrooms. A total of 237 students' instructional video watching behaviors were extracted from LMS, and were analyzed with background variables to predict students’ final performance. Regularization was proposed a suitable ML technique, as it produces interpretable prediction models. Specifically, Enet (elastic net) and Mnet were employed and compared to random forest, a method well-known for its prediction capabilities. As results, the prediction models of Enet and Mnet were comparable to that of random forest in terms of prediction measures, and identified 19 and 21 important predictors of final performance out of 157, respectively. In particular, both regularization models were able to screen lower-performing students as early as the first week of the course. Mere attempts to watch the most difficult videos after class increased the final scores. Despite the importance in FCs, however, pre-class assignments turned out to be not effective. Compared to 11 after-class variables selected important, only two before-class variables were selected, and the students on average completed at most 1/5 of the videos before class. Stronger links need to be established between pre-class assignments and in-class team projects.