Learning Unsupervised SVM Classifier for Answer Selection in Web Question Answering
11
Citation
17
Reference
20
Related Paper
Citation Trend
Abstract:
Previous machine learning techniques for answer selection in question answering (QA) have required question-answer training pairs. It has been too expensive and labor-intensive, however, to collect these training pairs. This paper presents a novel unsupervised support vector machine (USVM) classifier for answer selection, which is independent of language and does not require hand-tagged training pairs. The key ideas are the following: 1. unsupervised learning of training data for the classifier by clustering web search results; and 2. selecting the correct answer from the candidates by classifying the question. The comparative experiments demonstrate that the proposed approach significantly outperforms the retrieval-based model (Retrieval-M), the supervised SVM classifier (S-SVM), and the pattern-based model (Pattern-M) for answer selection. Moreover, the cross-model comparison showed that the performance ranking of these models was: U-SVM > PatternM > S-SVM > Retrieval-M.Keywords:
Ranking SVM
Cite
In order to respond correctly to a free form factual question given a large collection of texts, one needs to understand the question to a level that allows determining some of the constraints the question imposes on a possible answer. These constraints may include a semantic classification of the sought after answer and may even suggest using different strategies when looking for and verifying a candidate answer.This paper presents a machine learning approach to question classification. We learn a hierarchical classifier that is guided by a layered semantic hierarchy of answer types, and eventually classifies questions into fine-grained classes. We show accurate results on a large collection of free-form questions used in TREC 10.
Questions and answers
Cite
Citations (1,275)
The Open-domain Question Answering system (QA) has been attached great attention for its capacity of providing compact and precise results for sers. The question classification is an essential part in the system, affecting the accuracy of it. The paper studies question classification through machine learning approaches, namely, different classifiers and multiple classifier combination method. By using compositive statistic and rule classifiers, and by introducing dependency structure from Minipar and linguistic knowledge from Wordnet into question representation, the research shows high accuracy in question classification.
Statistic
Cite
Citations (9)
Text categorization is one of the key functions for utilizing vast amount of documents. It can be seen as a classification problem, which has been studied in pattern recognition and machine learning fields for a long time and several classification methods have been developed such as statistical classification, decision tree, support vector machines and so on. Many researchers applied those classification methods to text categorization and reported their performance (e.g., decision tree[3], Bayes classifier[2], support vector machine[l]). Yang conducted comprehensive study of comparison or text categorization and reported that k nearest neighbor and support vector machines works well for text categorization[4].In the previous studies, classification methods were usually compared using single pair of training and test data However, classification method with more complex family of classifiers requires more training data and small training data may result in deriving unreliable classifier, that is, the performance of the derived classifier varies much depending on training data. Therefore, we need to take the size of training data into account when comparing and selecting a classification method. In this paper, we discuss how to select a classifier from those derived by various classification methods and how the size of training data affects the performance of the derived classifier.In order to evaluate the reliability of classification method, we consider the variance of accuracy of derived classifier. We first construct a statistical model. In the text categorization, each document is usually represented with a feature vector that consists of weighted frequencies of terms. In the vector space model, document is a point in high dimensional feature space and a classifier separates the feature space into subspaces each of which is labeled with a category.
Quadratic classifier
Bayes error rate
Linear classifier
Cite
Citations (1)
Cite
Citations (6)
This paper regards Question Answering (QA) as Question-Biased Term Extraction (QBTE). This new QBTE approach liberates QA systems from the heavy burden imposed by question types (or answer types). In conventional approaches, a QA system analyzes a given question and determines the question type, and then it selects answers from among answer candidates that match the question type. Consequently, the output of a QA system is restricted by the design of the question types. The QBTE directly extracts answers as terms biased by the question. To confirm the feasibility of our QBTE approach, we conducted experiments on the CRL QA Data based on 10-fold cross validation, using Maximum Entropy Models (MEMs) as an ML technique. Experimental results showed that the trained system achieved 0.36 in MRR and 0.47 in Top5 accuracy.
Questions and answers
Cite
Citations (35)
Question classification is of crucial importance for question answering.In question classification, the accuracy of ML algorithms was found to significantly outperform other approaches.The two key issues in classification with a ML-based approach are classifier design and feature selection.Support Vector Machines is known to work well for sparse, high dimensional problems.However, the frequently used Bag-of-Words approach does not take full advantage of information contained in a question.To exploit this information we introduce three new feature types: Subordinate Word Category, Question Focus and Syntactic-Semantic Structure.As the results demonstrate, the inclusion of the new features provides higher accuracy of question classification compared to the standard Bag-of-Words approach and other ML based methods such as SVM with the Tree Kernel, SVM with Error Correcting Codes and SNoW.A classification accuracy of 85.6 % obtained using the three introduced feature types is, as of yet the highest reported in the literature, bringing error reduction of 27% compared to the Bag-of-Words approach.
Cite
Citations (13)
Stacking model Fusion is a combination classification method for natural language processing and text categorization. Compared to a single weak classifier, model Fusion has the advantage of combining the classification strengths of multiple classifiers, so the combination classifier is often more accurate than a single classifier, and the research of this field has been developed rapidly in recent years, and the combination classifier has been applied in various natural language processing tasks. But only by using the prediction results of the first layer weak classifier to train the second layer classifier, it has a strong limitation, only considers the training of the classification result and ignores the semantic information. We think that the method of training the weak classifier by TFIDF to the document, the expression of the document is not enough, only the information about the frequency of the document and the document is lack of the semantic information of the word2vector. In this paper, a new combination classification method is proposed, which combines the various weak classifiers trained by TFIDF and Word2vector to express the documents in many aspects, and the feature expression can fully utilize the information provided by the document. It has better classification effect than individual word2vector expression and classification and simple weak classifier combination classification.
Word2vec
tf–idf
Quadratic classifier
Document classification
Cite
Citations (9)
In question answering system, the process of classifying a question to appropriate class and identification of the focus word play key role in determining accurate answer. In this paper, we propose an integrated pattern matching and machine learning approach for higher education domain that focuses on factoid question answering. We have developed a question taxonomy for higher education domain and defined 9 coarse classes and 63 fine classes. We adopted pattern matching for the primary stage of classification and focus word identification and used machine learning approach i.e., Support Vector Machine (SVM) for the secondary classification approach only to those questions whose pattern are not present in question pattern corpus. Our experimental result shows that the accuracy of question classification using integrated approach outperforms the accuracy shown by individual approaches. SVM enhances the classification accuracy while focus word identification is achieved by virtue of pattern matching. The integrated approach shows the accuracy of 92.5% and 87.8% for coarse and fine class respectively and achieved focus word identification up to 83.4%.
Identification
Cite
Citations (4)
Cite
Citations (14)
Question classification is of crucial importance for question answering. In question classification, the accuracy of machine learning algorithms was found to significantly outperform other approaches. The two key issues in classification with a ML-based approach are classifier design and feature selection. Support vector machines is known to work well for sparse, high dimensional problems. However, the frequently used bag-of-words approach does not take full advantage of information contained in a question. To exploit this information we introduce three new feature types: subordinate word category, question focus and syntactic-semantic structure. As the results demonstrate, the inclusion of the new features provides higher accuracy of question classification compared to the standard bag-of-words approach and other ML based methods such as SVM with the tree kernel, SVM with error correcting codes and SNoW. A classification accuracy of 84.6% obtained using the three introduced feature types is as of yet the highest reported in the literature.
Feature (linguistics)
Relevance vector machine
Tree kernel
Kernel (algebra)
Feature vector
Cite
Citations (2)