Ming Zhou

eBay (Ireland)

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

Improving Grammatical Error Correction with Machine Translation Pairs

Wangchunshu Zhou Tao Ge Chang Mu Ke Xu Furu Wei

We propose a novel data synthesis method to generate diverse error-corrected sentence pairs for improving grammatical error correction, which is based on a pair of machine translation models (e.g., Chinese to English) of different qualities (i.e., poor and good). The poor translation model can resemble the ESL (English as a second language) learner and tends to generate translations of low quality in terms of fluency and grammaticality, while the good translation model generally generates fluent and grammatically correct translations. With the pair of translation models, we can generate unlimited numbers of poor to good English sentence pairs from text in the source language (e.g., Chinese) of the translators. Our approach can generate various error-corrected patterns and nicely complement the other data synthesis approaches for GEC. Experimental results demonstrate the data generated by our approach can effectively help a GEC model to improve the performance and achieve the state-of-the-art single-model performance in BEA-19 and CoNLL-14 benchmark datasets.

Grammaticality

Complement

Benchmark (surveying)

10.18653/v1/2020.findings-emnlp.30

Cite

Citations (32)

Topic Aware Neural Response Generation

arXiv (Cornell University) (2016)

Xing Chen Wei Wu Yu Wu Jie Liu Yalou Huang

We consider incorporating topic information into the sequence-to-sequence framework to generate informative and interesting responses for chatbots. To this end, we propose a topic aware sequence-to-sequence (TA-Seq2Seq) model. The model utilizes topics to simulate prior knowledge of human that guides them to form informative and interesting responses in conversation, and leverages the topic information in generation by a joint attention mechanism and a biased generation probability. The joint attention mechanism summarizes the hidden vectors of an input message as context vectors by message attention, synthesizes topic vectors by topic attention from the topic words of the message obtained from a pre-trained LDA model, and let these vectors jointly affect the generation of words in decoding. To increase the possibility of topic words appearing in responses, the model modifies the generation probability of topic words by adding an extra probability item to bias the overall distribution. Empirical study on both automatic evaluation metrics and human annotations shows that TA-Seq2Seq can generate more informative and interesting responses, and significantly outperform the-state-of-the-art response generation models.

Sequence (biology)

10.48550/arxiv.1606.08340

Cite

Citations (331)

Finding Target Language Correspondence for Lexicalized EBMT System.

NLPRS (2001)

Wei Wang Jin‐Xia Huang Ming Zhou Changning Huang

This paper presents a three-phase approach to find the correspondence in Target Language (TL) sentence for a fragment of Source Language (SL) sentence in a lexicalized EBMT system. To be practical, it exploits surface information as much as possible instead of using parsers. Experiments show that, although not so perfect, it is very robust and effective. The three phases are: First, align the sentence pair at word level in order to provide anchors for phrase alignment. Second, based on the aligned anchors, find all the TL fragments which may possibly correspond to a SL fragment. And finally, using a score function, select the best TL fragment as the correspondence of a SL fragment.

Fragment (logic)

Phrase

Word order

Source

Cite

Citations (4)

Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training

arXiv (Cornell University) (2019)

Gen Li Nan Duan Yuejian Fang Ming Gong Daxin Jiang

We propose Unicoder-VL, a universal encoder that aims to learn joint representations of vision and language in a pre-training manner. Borrow ideas from cross-lingual pre-trained models, such as XLM and Unicoder, both visual and linguistic contents are fed into a multi-layer Transformer for the cross-modal pre-training, where three pre-trained tasks are employed, including Masked Language Modeling (MLM), Masked Object Classification (MOC) and Visual-linguistic Matching (VLM). The first two tasks learn context-aware representations for input tokens based on linguistic and visual contents jointly. The last task tries to predict whether an image and a text describe each other. After pretraining on large-scale image-caption pairs, we transfer Unicoder-VL to caption-based image-text retrieval and visual commonsense reasoning, with just one additional output layer. We achieve state-of-the-art or comparable results on both two tasks and show the powerful ability of the cross-modal pre-training.

Commonsense reasoning

10.48550/arxiv.1908.06066

Cite

Citations (96)

Pseudo-Masked Language Models for Unified Language Model Pre-Training

International Conference on Machine Learning (2020)

Hangbo Bao Li Dong Furu Wei Wenhui Wang Nan Yang

Source

Cite

Citations (0)

Jointly Modeling Topics and Intents with Global Order Structure

Proceedings of the AAAI Conference on Artificial Intelligence (2016)

Bei Chen Jun Zhu Nan Yang Tian Tian Ming Zhou

Modeling document structure is of great importance for discourse analysis and related applications. The goal of this research is to capture the document intent structure by modeling documents as a mixture of topic words and rhetorical words. While the topics are relatively unchanged through one document, the rhetorical functions of sentences usually change following certain orders in discourse. We propose GMM-LDA, a topic modeling based Bayesian unsupervised model, to analyze the document intent structure cooperated with order information. Our model is flexible that has the ability to combine the annotations and do supervised learning. Additionally, entropic regularization can be introduced to model the significant divergence between topics and intents. We perform experiments in both unsupervised and supervised settings, results show the superiority of our model over several state-of-the-art baselines.

Regularization

Divergence (linguistics)

10.1609/aaai.v30i1.10346

Cite

Citations (1)

Dense Procedure Captioning in Narrated Instructional Videos

Botian Shi Lei Ji Yaobo Liang Nan Duan Peng Chen

Understanding narrated instructional videos is important for both research and real-world web applications. Motivated by video dense captioning, we propose a model to generate procedure captions from narrated instructional videos which are a sequence of step-wise clips with description. Previous works on video dense captioning learn video segments and generate captions without considering transcripts. We argue that transcripts in narrated instructional videos can enhance video representation by providing fine-grained complimentary and semantic textual information. In this paper, we introduce a framework to (1) extract procedures by a cross-modality module, which fuses video content with the entire transcript; and (2) generate captions by encoding video frames as well as a snippet of transcripts within each extracted procedure. Experiments show that our model can achieve state-of-the-art performance in procedure extraction and captioning, and the ablation studies demonstrate that both the video frames and the transcripts are important for the task.

Closed captioning

Snippet

10.18653/v1/p19-1641

Cite

Citations (59)

Chunk-based Decoder for Neural Machine Translation

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2017)

Shonosuke Ishiwatari JingTao Yao Shujie Liu Mu Li Ming Zhou

Shonosuke Ishiwatari, Jingtao Yao, Shujie Liu, Mu Li, Ming Zhou, Naoki Yoshinaga, Masaru Kitsuregawa, Weijia Jia. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017.

Association (psychology)

10.18653/v1/p17-1174

Cite

Citations (19)

A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation

Shuo Ren Yu Wu Shujie Liu Ming Zhou Shuai Ma

The commonly used framework for unsupervised machine translation builds initial translation models of both translation directions, and then performs iterative back-translation to jointly boost their translation performance. The initialization stage is very important since bad initialization may wrongly squeeze the search space, and too much noise introduced in this stage may hurt the final performance. In this paper, we propose a novel retrieval and rewriting based method to better initialize unsupervised translation models. We first retrieve semantically comparable sentences from monolingual corpora of two languages and then rewrite the target side to minimize the semantic gap between the source and retrieved targets with a designed rewriting model. The rewritten sentence pairs are used to initialize SMT models which are used to generate pseudo data for two NMT models, followed by the iterative back-translation. Experiments show that our method can build better initial unsupervised translation models and improve the final translation performance by over 4 BLEU scores. Our code is released at https://github.com/Imagist-Shuo/RRforUNMT.git.

Initialization

Transfer-based machine translation

Code (set theory)

10.18653/v1/2020.acl-main.320

Cite

Citations (19)

Named entity translation with web mining and transliteration

International Joint Conference on Artificial Intelligence (2007)

Long Jiang Ming Zhou Lee‐Feng Chien Cheng Niu

This paper presents a novel approach to improve the named entity translation by combining a transliteration approach with web mining, using web information as a source to complement transliteration, and using transliteration information to guide and enhance web mining. A Maximum Entropy model is employed to rank translation candidates by combining pronunciation similarity and bilingual contextual co-occurrence. Experimental results show that our approach effectively improves the precision and recall of the named entity translation by a large margin.

Transliteration

Named entity

Complement

Margin (machine learning)

Source

Cite

Citations (74)