FoPro: Few-Shot Guided Robust Webly-Supervised Prototypical Learning
3
Citation
0
Reference
10
Related Paper
Citation Trend
Abstract:
Recently, webly supervised learning (WSL) has been studied to leverage numerous and accessible data from the Internet. Most existing methods focus on learning noise-robust models from web images while neglecting the performance drop caused by the differences between web domain and real-world domain. However, only by tackling the performance gap above can we fully exploit the practical value of web datasets. To this end, we propose a Few-shot guided Prototypical (FoPro) representation learning method, which only needs a few labeled examples from reality and can significantly improve the performance in the real-world domain. Specifically, we initialize each class center with few-shot real-world data as the ``realistic" prototype. Then, the intra-class distance between web instances and ``realistic" prototypes is narrowed by contrastive learning. Finally, we measure image-prototype distance with a learnable metric. Prototypes are polished by adjacent high-quality web images and involved in removing distant out-of-distribution samples. In experiments, FoPro is trained on web datasets with a few real-world examples guided and evaluated on real-world datasets. Our method achieves the state-of-the-art performance on three fine-grained datasets and two large-scale datasets. Compared with existing WSL methods under the same few-shot settings, FoPro still excels in real-world generalization. Code is available at https://github.com/yuleiqin/fopro.Keywords:
Leverage (statistics)
Code (set theory)
Adversarial attacks on machine learning-based classifiers, along with defense mechanisms, have been widely studied in the context of single-label classification problems. In this paper, we shift the attention to multi-label classification, where the availability of domain knowledge on the relationships among the considered classes may offer a natural way to spot incoherent predictions, i.e., predictions associated to adversarial examples lying outside of the training data distribution. We explore this intuition in a framework in which first-order logic knowledge is converted into constraints and injected into a semi-supervised learning problem. Within this setting, the constrained classifier learns to fulfill the domain knowledge over the marginal distribution, and can naturally reject samples with incoherent predictions. Even though our method does not exploit any knowledge of attacks during training, our experimental analysis surprisingly unveils that domain-knowledge constraints can help detect adversarial examples effectively, especially if such constraints are not known to the attacker. While we also show that an adaptive attack exploiting knowledge of the constraints may still deceive our classifier, it remains an open issue to understand how hard for an attacker would be to infer such constraints in practical cases. For this reason, we believe that our approach may provide a significant step towards designing robust multi-label classifiers.
Intuition
Lying
Training set
Cite
Citations (10)
Deep learning models have been used in creating various effective image classification applications. However, they are vulnerable to adversarial attacks that seek to misguide the models into predicting incorrect classes. Our study of major adversarial attack models shows that they all specifically target and exploit the neural networking structures in their designs. This understanding makes us develop a hypothesis that most classical machine learning models, such as Random Forest (RF), are immune to adversarial attack models because they do not rely on neural network design at all. Our experimental study of classical machine learning models against popular adversarial attacks supports this hypothesis. Based on this hypothesis, we propose a new adversarial-aware deep learning system by using a classical machine learning model as the secondary verification system to complement the primary deep learning model in image classification. Although the secondary classical machine learning model has less accurate output, it is only used for verification purposes, which does not impact the output accuracy of the primary deep learning model, and at the same time, can effectively detect an adversarial attack when a clear mismatch occurs. Our experiments based on CIFAR-100 dataset show that our proposed approach outperforms current state-of-the-art adversarial defense systems.
Adversarial machine learning
Deep Neural Networks
Complement
Cite
Citations (0)
Features play a crucial role in several computational tasks. Feature values are input to machine learning algorithms for the prediction. The prediction accuracy depends on various factors such as selection of dataset, features and machine learning classifiers. Various feature selection and reduction approaches are experimented with to obtain better accuracies and reduce the computational overheads. Feature engineering is designing new features suitable for a specific task with the help of domain knowledge. The challenges in feature engineering are presented for the computational music domain as a case study. The experiments are performed with different combinations of feature sets and machine learning classifiers to test the accuracy of the proposed model. Music emotion recognition is used as a case study for the experimentation. Experimental results for the task of music emotion recognition provide insights into the role of features and classifiers in prediction accuracy. Different machine learning classifiers provided varied results, and the choice of a classifier is also an important decision to be made in the proposed model. The engineered features designed with the help of domain experts improved the results. It emphasizes the need for feature engineering for different domains for prediction accuracy improvement. Approaches to design an optimized model with the appropriate feature set and classifier for machine learning tasks are presented.
Feature Engineering
Feature (linguistics)
Cite
Citations (0)
Abstract The availability of source code for both exploits and malicious code is higher than it has ever been in the history of computing. More important, the source codes for highly successful, high- profile malicious tools are now available. Several years ago it was almost impossible to obtain the source for these worms and exploits, forcing actors to rely on minor changes to binaries to generate new variants of a family. With the source code, attackers can easily edit code, compile new “undetected” variants, and copy and paste code from multiple creations to create new codes.
Code (set theory)
Cite
Citations (0)
In general, a large amount of labels are needed for supervised learning algorithms to achieve satisfactory performance. It's typically very time-consuming and money-consuming to get such kind of labeled data. Recently, crowdsourcing services provide an effective way to collect labeled data with much lower cost. Hence, crowdsourced learning (CL), which performs learning with labeled data collected from crowdsourcing services, has become a very hot and interesting research topic in recent years. Most existing CL methods exploit only the labels from different workers (annotators) for learning while ignoring the attributes of the instances. In many real applications, the attributes of the instances are actually the most discriminative information for learning. Hence, CL methods with attributes have attracted more and more attention from CL researchers. One representative model of such kind is the personal classifier (PC) model, which has achieved the state-of-the-art performance. However, the PC model makes an unreasonable assumption that all the workers contribute equally to the final classification. This contradicts the fact that different workers have different quality (ability) for data labeling. In this paper, we propose a novel model, called robust personal classifier (RPC), for robust crowdsourced learning. Our model can automatically learn an expertise score for each worker. This expertise score reflects the inherent quality of each worker. The final classifier of our RPC model gives high weights for good workers and low weights for poor workers or spammers, which is more reasonable than PC model with equal weights for all workers. Furthermore, the learned expertise score can be used to eliminate spammers or low-quality workers. Experiments on simulated datasets and UCI datasets show that the proposed model can dramatically outperform the baseline models such as PC model in terms of classification accuracy and ability to detect spammers.
Crowdsourcing
Discriminative model
Labeled data
Cite
Citations (6)
Activity recognition in humans is one of the active challenges that finds its application in numerous fields such as, medical health care, military, manufacturing, assistive techniques and gaming. Due to the advancements in technologies the usage of smartphones in human lives become inevitable. The sensors in the smartphones help us to measure the essential vital parameters. These measured parameters enable us to monitor the activities of humans, which we call as human activity recognition. In this paper, we have proposed an automatic human activity recognition system that independently recognizes the actions of the humans. Four deep learning approaches and thirteen different machine learning classifiers such as Multilayer Perceptron, Random Forest, Support Vector Machine, Decision Tree Classifier, AdaBoost Classifier, Gradient Boosting Classifier and others are applied to identify the efficient classifier for human activity recognition. Our proposed system is able to recognize the activities such as Laying, Sitting, Standing, Walking, Walking downstairs and Walking upstairs. Benchmark dataset has been used to evaluate all the classifiers implemented. We have investigated all these classifiers to identify a best suitable classifier for this dataset. The results obtained show that, the Multilayer Perceptron has obtained 98.46% of overall accuracy in detecting the activities. The second-best performance was observed when the classifiers are combined together.
AdaBoost
Activity Recognition
Multilayer perceptron
Gradient boosting
Boosting
Perceptron
Cite
Citations (12)
Adversarial attacks on machine learning-based classifiers, along with defense mechanisms, have been widely studied in the context of single-label classification problems. In this paper, we shift the attention to multi-label classification, where the availability of domain knowledge on the relationships among the considered classes may offer a natural way to spot incoherent predictions, i.e., predictions associated to adversarial examples lying outside of the training data distribution. We explore this intuition in a framework in which first-order logic knowledge is converted into constraints and injected into a semi-supervised learning problem. Within this setting, the constrained classifier learns to fulfill the domain knowledge over the marginal distribution, and can naturally reject samples with incoherent predictions. Even though our method does not exploit any knowledge of attacks during training, our experimental analysis surprisingly unveils that domain-knowledge constraints can help detect adversarial examples effectively, especially if such constraints are not known to the attacker. We show how to implement an adaptive attack exploiting knowledge of the constraints and, in a specifically-designed setting, we provide experimental comparisons with popular state-of-the-art attacks. We believe that our approach may provide a significant step towards designing more robust multi-label classifiers.
Intuition
Boosting
Cite
Citations (13)
Machine learning is often perceived as a sophisticated technology accessible only by highly trained experts. This prevents many physicians and biologists from using this tool in their research. The goal of this paper is to eliminate this out-dated perception. We argue that the recent development of auto machine learning techniques enables biomedical researchers to quickly build competitive machine learning classifiers without requiring in-depth knowledge about the underlying algorithms. We study the case of predicting the risk of cardiovascular diseases. To support our claim, we compare auto machine learning techniques against a graduate student using several important metrics, including the total amounts of time required for building machine learning models and the final classification accuracies on unseen test datasets. In particular, the graduate student manually builds multiple machine learning classifiers and tunes their parameters for one month using scikit-learn library, which is a popular machine learning library to obtain ones that perform best on two given, publicly available datasets. We run an auto machine learning library called auto-sklearn on the same datasets. Our experiments find that automatic machine learning takes 1 h to produce classifiers that perform better than the ones built by the graduate student in one month. More importantly, building this classifier only requires a few lines of standard code. Our findings are expected to change the way physicians see machine learning and encourage wide adoption of Artificial Intelligence (AI) techniques in clinical domains.
Learning classifier system
Cite
Citations (65)
Construction of multi-class classifiers using homogeneous combination of Extreme Learning Machine (ELM) based one-class classifiers have been proposed in this paper. Each class has been trained using individual one-class classifier and any new sample will belong to that class, which will yield maximum value. Proposed methods can be used to detect unknown outliers using multi-class classifiers. Two recently proposed one-class classifiers viz., kernel and random feature mapping based one-class ELM, is extended for multi-class construction in this paper. Further, we construct one-class classifier based multi-class classifier in two ways: with rejection and without rejection of few samples during training. We also perform consistency based model selection for optimal parameters selection in one-class classifier. We have tested the generalization capability of the proposed classifiers on 6 synthetic datasets and two benchmark datasets.
Extreme Learning Machine
One-class classification
Margin classifier
Cite
Citations (3)
This paper presents an exploratory machine learning attack based on deep learning to infer the functionality of an arbitrary classifier by polling it as a black box, and using returned labels to build a functionally equivalent machine. Typically, it is costly and time consuming to build a classifier, because this requires collecting training data (e.g., through crowdsourcing), selecting a suitable machine learning algorithm (through extensive tests and using domain-specific knowledge), and optimizing the underlying hyperparameters (applying a good understanding of the classifier's structure). In addition, all this information is typically proprietary and should be protected. With the proposed black-box attack approach, an adversary can use deep learning to reliably infer the necessary information by using labels previously obtained from the classifier under attack, and build a functionally equivalent machine learning classifier without knowing the type, structure or underlying parameters of the original classifier. Results for a text classification application demonstrate that deep learning can infer Naive Bayes and SVM classifiers with high accuracy and steal their functionalities. This new attack paradigm with deep learning introduces additional security challenges for online machine learning algorithms and raises the need for novel mitigation strategies to counteract the high fidelity inference capability of deep learning.
Learning classifier system
Adversarial machine learning
Cite
Citations (85)