Intrinsic Evaluation of Word Embeddings for Turkish

Word embeddings are evaluated through intrinsic and extrinsic tests. Similarity and analogy test are mainly preferred for intrinsic evaluation and natural language processing tasks such as named entity recognition and question answering are prefferred for extrinsic evaluation. Although there are various intrinsic evaluation datasets for English, the datasets for Turkish are very limited and measuring the degree of similarity and relatedness between words without specifying the type of semantic relation. In this paper, we propose an intrinsic evaluation dataset for evaluating different semantic relations other than a synonym, antonym, hypernym, and meronym as well as morphological relations of individual Turkish words. Moreover, we benchmark three publicly available word-embedding models on the proposed dataset and discuss agglutinative characteristics of the Turkish language for language modeling.