Automatic Bilingual Phrase Extraction from Comparable Corpora

In this work we present an approach for extracting parallel phrases from comparable news articles to improve statistical machine translation. This is particularly useful for under-resourced languages where parallel corpora are not readily available. Our approach consists of a phrase pair generator that automatically generates candidate parallel phrases and a binary SVM classifier that classifies the candidate phrase pairs as parallel or non-parallel. The phrase pair generator is also used to automatically create training and testing data for the SVM classifier from parallel corpora. We evaluate our approach using English-German, English-Greek and English-Latvian language pairs. The performance of our classifier on the test sets is above 80% precision and 97% accuracy for all language pairs. We also perform an SMT evaluation by measuring the impact of phrases extracted from comparable corpora on SMT quality using BLEU. For all language pairs we obtain significantly better results compared to the baselines.