Reinforcement Learning Based Multi-modal Feature Fusion Network for Novel Class Discovery
0
Citation
0
Reference
10
Related Paper
Abstract:
With the development of deep learning techniques, supervised learning has achieved performances surpassing those of humans. Researchers have designed numerous corresponding models for different data modalities, achieving excellent results in supervised tasks. However, with the exponential increase of data in multiple fields, the recognition and classification of unlabeled data have gradually become a hot topic. In this paper, we employed a Reinforcement Learning framework to simulate the cognitive processes of humans for effectively addressing novel class discovery in the Open-set domain. We deployed a Member-to-Leader Multi-Agent framework to extract and fuse features from multi-modal information, aiming to acquire a more comprehensive understanding of the feature space. Furthermore, this approach facilitated the incorporation of self-supervised learning to enhance model training. We employed a clustering method with varying constraint conditions, ranging from strict to loose, allowing for the generation of dependable labels for a subset of unlabeled data during the training phase. This iterative process is similar to human exploratory learning of unknown data. These mechanisms collectively update the network parameters based on rewards received from environmental feedback. This process enables effective control over the extent of exploration learning, ensuring the accuracy of learning in unknown data categories. We demonstrate the performance of our approach in both the 3D and 2D domains by employing the OS-MN40, OS-MN40-Miss, and Cifar10 datasets. Our approach achieves competitive competitive results.Keywords:
Feature (linguistics)
Feature Learning
Learning representations of data, and in particular learning features for a subsequent prediction task, has been a fruitful area of research delivering impressive empirical results in recent years. However, relatively little is understood about what makes a representation `good'. We propose the idea of a risk gap induced by representation learning for a given prediction context, which measures the difference in the risk of some learner using the learned features as compared to the original inputs. We describe a set of sufficient conditions for unsupervised representation learning to provide a benefit, as measured by this risk gap. These conditions decompose the problem of when representation learning works into its constituent parts, which can be separately evaluated using an unlabeled sample, suitable domain-specific assumptions about the joint distribution, and analysis of the feature learner and subsequent supervised learner. We provide two examples of such conditions in the context of specific properties of the unlabeled distribution, namely when the data lies close to a low-dimensional manifold and when it forms clusters. We compare our approach to a recently proposed analysis of semi-supervised learning.
Feature Learning
Representation
Feature (linguistics)
Empirical risk minimization
Supervised Learning
Cite
Citations (0)
Reinforcement Learning continues to show promise in solving problems in new ways. Recent publications have demonstrated how utilizing a reinforcement learning approach can lead to a superior policy for optimization. While previous works have demonstrated the ability to train without gradients, most recent works has focused on the simpler regression problems. This work will show how a Multi-Agent Reinforcement Learning approach can be used to optimize models in training without the need for the gradient of the loss function, and how this approach can benefit defense applications.
Presentation (obstetrics)
Cite
Citations (0)
Data representation has received much attention in the fields of machine learning and pattern recognition. It is becoming an indispensable tool for many learning tasks. It can be useful for all learning paradigms: unsupervised, semi-supervised, and supervised. In this paper, we present a graph-based, deep and flexible data representation method using feature propagation as an internal filtering step. The presented framework ensures several desired features such as a graph-based regularization, a flexible projection model, a graph-based feature aggregation, and a deep learning architecture. The model can be learned layer by layer. In each layer, the nonlinear data representation and the unknown linear model are jointly estimated with a closed form solution. We evaluate the proposed method on semi-supervised classification tasks using six public image datasets. These experiments demonstrate the effectiveness of the presented scheme, which compares favorably to many competing semi-supervised approaches.
Feature Learning
Representation
Regularization
Supervised Learning
Feature (linguistics)
External Data Representation
Labeled data
Cite
Citations (0)
The excellent performance of transfer learning has emerged in the past few years. How to find feature representations which minimizes the distance between source and target domain is the crucial problem in transfer learning. Recently, deep learning methods have been proposed to learn higher level and robust representation. However, in traditional methods, label information in source domain is not designed to optimize both feature representations and parameters of the learning model. Additionally, data redundance may incur performance degradation on transfer learning. To address these problems, we propose a novel semi-supervised representation learning framework for transfer learning. To obtain this framework, manifold regularization is integrated for the parameters optimization, and the label information is encoded using a softmax regression model in auto-encoders. Meanwhile, whitening layer is introduced to reduce data redundance before auto-encoders. Extensive experiments demonstrate the effectiveness of our proposed framework compared to other competing state-of-the-art baseline methods.
Feature Learning
Transfer of learning
Softmax function
Autoencoder
Representation
Regularization
Feature (linguistics)
Cite
Citations (1)
In multi-agent reinforcement learning, the state space grows exponentially in terms of the number of agents, which makes the training episode longer than before. It will take more time to make learning convergent. In order to improve the efficiency of the convergence, we propose an algorithm to find shortcuts from episode in multi-agent reinforcement learning to speed up convergence. The loops that indicate the ineffective paths in the episode are removed, but all the shortest state paths from each other state to the goal state within the original episode are kept, that means no loss of state space knowledge when remove these loops. So the length of episode is shortened to speed up the convergence. Since a large mount of episodes are included in learning process, the overall improvement accumulated from every episode's improvement will be considerable. The episode of multi-agent pursuit problem is used to illustrate the effectiveness of our algorithm. We believe this algorithm can be introduced into most other reinforcement learning approaches for speeding up convergence, because its improvement is made on episode, which is the most foundational learning unit of reinforcement learning.
Cite
Citations (1)
Learning a high-performance trade execution model via reinforcement learning (RL) requires interaction with the real dynamic market. However, the massive interactions required by direct RL would result in a significant training overhead. In this paper, we propose a cost-efficient reinforcement learning (RL) approach called Deep Dyna-Double Q-learning (D3Q), which integrates deep reinforcement learning and planning to reduce the training overhead while improving the trading performance. Specifically, D3Q includes a learnable market environment model, which approximates the market impact using real market experience, to enhance policy learning via the learned environment. Meanwhile, we propose a novel state-balanced exploration scheme to solve the exploration bias caused by the non-increasing residual inventory during the trade execution to accelerate model learning. As demonstrated by our extensive experiments, the proposed D3Q framework significantly increases sample efficiency and outperforms state-of-the-art methods on average trading cost as well.
Q-learning
Cite
Citations (1)
Q-learning
Spectrum management
Cite
Citations (13)
Unmanned Aerial Vehicle (UAV) is increasingly becoming an important tool used for a variety of tasks. In addition, Reinforcement Learning (RL) is a popular research topic. In this paper, these two fields are combined together and we apply the reinforcement learning into the UAV field, promote the application of reinforcement learning in our real life. We design a reinforcement learning framework named ROS-RL, this framework is based on the physical simulation platform Gazebo and it can address the problem of UAV motion in continuous action space. We can connect our algorithms into this framework through ROS and train the agent to control the drone to complete some tasks. We realize the autonomous landing task of UAV using three different reinforcement learning algorithms in this framework. The experiment results show the effectiveness of algorithm in controlling UAV which flights in a simulation environment close to the real world.
Drone
Cite
Citations (4)
Recent years have witnessed the significant success of representation learning and deep learning in various prediction and recognition applications. Most of these previous studies adopt the two-phase procedures, namely the first step of representation learning and then the second step of supervised learning. In this process, to fit the training data the initial model weights, which inherits the good properties from the representation learning in the first step, will be changed in the second step. In other words, the second step leans better classification models at the cost of the possible deterioration of the effectiveness of representation learning. Motivated by this observation we propose a joint framework of representation and supervised learning. It aims to learn a model, which not only guarantees the "semantics" of the original data from representation learning but also fit the training data well via supervised learning. Along this line we develop the model of semi-supervised Auto encoder under the spirit of the joint learning framework. The experiments on various data sets for classification show the significant effectiveness of the proposed model.
Autoencoder
Representation
Feature Learning
Supervised Learning
External Data Representation
Cite
Citations (12)
Feature Learning
Feature (linguistics)
Supervised Learning
Representation
Graph Embedding
External Data Representation
Regularization
Cite
Citations (3)