Label Quality Improvement in Crowdsourcing with Ensemble TSK Fuzzy Classifier.

At present, crowdsourcing, as a distributed solution, provides an effective and cheap solution for solving large tasks. However, due to the difference of workers’ knowledge and skill, and the existence of fraudsters, the labels quality of crowdsourcing can’t be effectively controlled and guaranteed. This paper proposes a novel label quality improvement method based on ensemble TSK fuzzy classifier with high interpretability, i.e., EW-TSK-CS. Each subclassifier TSKnoise-FC is an improved zero-order TSK fuzzy classifier which is trained by noisy label training data and is more robust. The objective function of each fuzzy sub-classifier has considered the existence of label noise, and the fuzzy subclassifier has the ability to deal with uncertain data. All the subclassifier integrated together by augmenting the original noisy-free validation data space with the output of each subclassifier in an incremental way. The augmented validation data is conducted by running the classical FCM clustering methods on the augmented validation data and using KNN to obtain the dictionary data. The label noise correction mechanism is based on the dictionary data. The experimental results on datasets Adult, chess and waveform3 show that this method can effectively improve the label quality of crowdsourcing compared with tradition label noise robustness methods, ensemble methods, and classical TSK fuzzy classifiers.