Multi-Task Semantic Matching model for Small Noisy Data Set

The dominant semantic matching models are based on deep neural networks, but the performance of these models can be limited by the quality of data. In this paper, we propose a Multi-Task Semantic Matching model(MTSM) for small noisy data collected from a specific domain. The MTSM uses the pre-trained language model RoBERTa as the shared layer of multitask learning and sets different networks at the task-specific layer for sentence pairs with different literal matching degrees in the data set to fit the matching patterns of data precisely. The MTSM also introduces semantic matching tasks from different but relative domains as auxiliary tasks so that the main task can benefit from large amounts of data and a regularization effect of other tasks. Experiments show that the MTSM is superior in dealing with small noisy data compared to the RoBERTa model and the traditional machine learning model XGBoost. We also demonstrate that the MTSM generalizes well when using limited training data to overcome the impact of label errors.