From Orthography to Semantics: a Study of Morphological Processing through Deep Learning Neural Networks

This paper presents results of using deep learning neural network techniques to map words, represented at the levels of letters, into semantic, distributed vector representations. We study and compare algorithms that have been proposed as models for human orthographic and morphological processing, such as the two layers symbolic network (AKA Naive Discriminatory Learning network) proposed by Baayen et al [1], and the dual-route approach presented by Grainger and Ziegler [2]. In addition, we study the effect of representing letters as one-hot vectors or via distributed vector representations, much like the natural language processing field has done for words. Experiment results show a better performance for the dual-route algorithm using distributed letter representations, both in terms of accuracy and speed. Our results are obtained with training sets in the tens of thousands of words, as opposed to the hundreds of thousands of words used elsewhere in the literature. These results point to the feasibility of taking advantage of morpho-orthographic patterns in order to assist with word processing in natural language processing tasks.