Assessment methods of speech synthesis systems for Chinese

A national assesment of the performance of speech synthesis systems for Chinese has been carried out regularly since 1994. The synthetic speech quality of five defferent systems were evaluated and diagnosed by using speech intelligibility tests. 16 college students(8 male, 8 female) with no experience with synthetic speech were the listeners, they were asked to do open response task by pencil-paper. In addition, speech naturalness was measured by Mean Opinion Score (MOS) on a ten point scale. The perceptual confusion matrices of consonants were analyzed in order to give some diagnostic information of the synthesis systems at segmental level. And the statistical relations between speech intelligibilities at different levels of synthetic speech were compared with that of natural speech to explore some deficiencies in prosody. It is shown that stable and rational quality index and diagnostic information can be obtained in this method.The eveluation methods of prosody have not been become available.