Towards fairer evaluations of commercial MT systems on basic travel expressions corpora.

We compare the performance of several SYSTRAN systems on the BTEC corpus. Two language pairs: Chinese to English and Japanese to English are used. Whenever it is possible the system will be used “off the shelf” and then tuned. The first system we use is freely available on the web. The second system, SYSTRAN Premium, is commercial. It is used in two ways: (1) choosing and ordering available original dictionaries and setting parameters, (2) same + user dictionaries. As far as the evaluation is concerned, we competed in the unlimited data track.