ProLog2vec: Detecting novices’ difficulty in programming using deep learning

Encountering difficulties that emerge in programming learning are ubiquitous for novices. Those difficulties may reduce novices’ motivation for learning, even lead them to drop out. Automatically detecting the difficulties has been proved as an effective way to solve this problem. A potential method for the detection is to identify programmers’ behavioural patterns based on the sequences of programming events (e.g., editing, deleting, or debugging) when they experience programming difficulties. However, due to the diversity of these events, the plentiful difficulties generated from novices, and the naturally complicated behavioural patterns, accurately detecting the difficulties remains challenging. In this study, we propose a deep representation based on the programming events as the pre-treatment of a recurrent neural network with an attention mechanism to detect programming difficulty. This representation can be conducive to encode the complicated behavioural features (e.g., co-occurrence, sequences, and time intervals of the events) of the difficulty patterns for deep learning models. The results show that the accuracy of our method reaches 94%, which increases by 16% compared with the traditional machine learning model.