Trung Le

Australian Regenerative Medicine Institute

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Dinh Phung

Monash University

132

He Zhao

York University

Tu Dinh Nguyen

University of Technology

Dat Tran

University of Canberra

Khanh Nguyen

San Jose State University

Van Nguyen

National Agriculture and Food Research Organization

Wanli Ma

Guangzhou Institutes of Biomedicine and Health

Dharmendra Sharma

Indian Institute of Technology Mandi

Paul Montague

Defence Science and Technology Group

Olivier De Vel

Defence Science and Technology Group

Cooperative Institutions

Fermi National Accelerator Laboratory

233

European Organization for Nuclear Research

147

Istituto Nazionale di Fisica Nucleare

113

Centre National de la Recherche Scientifique

111

Brookhaven National Laboratory

Colorado State University

University of Chicago

Institut National de Physique Nucléaire et de Physique des Particules

University of Cincinnati

University of California, Irvine

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

A Generic Framework for Soft Subspace Pattern Recognition

Dat Tran Wanli Ma Dharmendra Sharma Len Bui Trung Le

In statistical pattern recognition, hidden Markov model (HMM) is the most important technique for modelling patterns that include temporal information such as speech and handwriting. If the temporal information is not taken into account, Gaussian mixture model (GMM) is used. This GMM technique uses a mixture of Gaussian densities to model the distribution of feature vectors extracted from training data. When little training data are available, vector quantisation (VQ) technique is also effective (Tran & Wagner 2002). In fuzzy set theory-based pattern recognition, fuzzy clustering techniques such as fuzzy cmeans and fuzzy entropy are used to design re-estimation techniques for fuzzy HMM, fuzzy GMM, and fuzzy VQ (Tran & Wagner 2000). The first stage in pattern recognition is data feature selection. A number of features that best characterises the considering pattern is extracted and the selection of features is dependent on the pattern to be recognised and has direct impact on the recognition results. The abovementioned pattern recognition methods cannot select features automatically because they treat all features equally. We propose that the contribution of a feature to pattern recognition should be measured by a weight that is assigned to the feature in the modelling process. This method is called soft subspace pattern recognition. There have been some algorithms proposed to calculate weights for soft subspace clustering (Huang et al. 2005, Jing et al. 2007). However a generic framework for the above-mentioned modelling methods has not been built. A generic framework for soft subspace pattern recognition will be proposed in this chapter. A generic objective function will be designed for HMM and maximizing this function will provide an algorithm for calculating weights. Other weight calculation algorithms for GMM and VQ will also be determined from the algorithm for HMM. The proposed soft subspace pattern recognition methods will be evaluated in network intrusion detection. Some preliminary experiments have been done and experimental results showed that the proposed algorithms could improve the recognition rates.

Feature vector

Feature (linguistics)

10.5772/6683

Cite

Citations (1)

A Theoretical Framework for Multi-sphere Support Vector Data Description

Lecture notes in computer science (2010)

Trung Le Dat Tran Wanli Ma Dharmendra Sharma

Data set

Decision boundary

10.1007/978-3-642-17534-3_17

Cite

Citations (16)

One-Pass Logistic Regression for Label-Drift and Large-Scale Classification on Distributed Systems

Vu Nguyen Tu Dinh Nguyen Trung Le Svetha Venkatesh Dinh Phung

Logistic regression (LR) for classification is the workhorse in industry, where a set of predefined classes is required. The model, however, fails to work in the case where the class labels are not known in advance, a problem we term label-drift classification. Label-drift classification problem naturally occurs in many applications, especially in the context of streaming settings where the incoming data may contain samples categorized with new classes that have not been previously seen. Additionally, in the wave of big data, traditional LR methods may fail due to their expense of running time. In this paper, we introduce a novel variant of LR, namely one-pass logistic regression (OLR) to offer a principled treatment for label-drift and large-scale classifications. To handle largescale classification for big data, we further extend our OLR to a distributed setting for parallelization, termed sparkling OLR (Spark-OLR). We demonstrate the scalability of our proposed methods on large-scale datasets with more than one hundred million data points. The experimental results show that the predictive performances of our methods are comparable orbetter than those of state-of-the-art baselines whilst the executiontime is much faster at an order of magnitude. In addition, the OLR and Spark-OLR are invariant to data shuffling and have no hyperparameter to tune that significantly benefits data practitioners and overcomes the curse of big data cross-validationto select optimal hyperparameters.

10.1109/icdm.2016.0145

Cite

Citations (9)

Code Action Network for Binary Function Scope Identification

Lecture notes in computer science (2020)

Van Nguyen Trung Le T. Kim Le Khanh Nguyen Olivier De Vel

Scope (computer science)

Identification

Code (set theory)

Binary code

10.1007/978-3-030-47426-3_55

Cite

Citations (3)

Approximation Vector Machines for Large-scale Online Learning

arXiv (Cornell University) (2016)

Trung Le Tu Dinh Nguyen Vu Nguyen Dinh Phung

One of the most challenging problems in kernel online learning is to bound the model size and to promote the model sparsity. Sparse models not only improve computation and memory usage, but also enhance the generalization capacity, a principle that concurs with the law of parsimony. However, inappropriate sparsity modeling may also significantly degrade the performance. In this paper, we propose Approximation Vector Machine (AVM), a model that can simultaneously encourage the sparsity and safeguard its risk in compromising the performance. When an incoming instance arrives, we approximate this instance by one of its neighbors whose distance to it is less than a predefined threshold. Our key intuition is that since the newly seen instance is expressed by its nearby neighbor the optimal performance can be analytically formulated and maintained. We develop theoretical foundations to support this intuition and further establish an analysis to characterize the gap between the approximation and optimal solutions. This gap crucially depends on the frequency of approximation and the predefined threshold. We perform the convergence analysis for a wide spectrum of loss functions including Hinge, smooth Hinge, and Logistic for classification task, and $l_1$, $l_2$, and $ε$-insensitive for regression task. We conducted extensive experiments for classification task in batch and online modes, and regression task in online mode over several benchmark datasets. The results show that our proposed AVM achieved a comparable predictive performance with current state-of-the-art methods while simultaneously achieving significant computational speed-up due to the ability of the proposed AVM in maintaining the model size.

10.48550/arxiv.1604.06518

Cite

Citations (12)

Neural Topic Model via Optimal Transport.

arXiv (Cornell University) (2020)

He Zhao Dinh Phung Viet Huynh Trung Le Wray Buntine

Recently, Neural Topic Models (NTMs) inspired by variational autoencoders have obtained increasingly research interest due to their promising results on text analysis. However, it is usually hard for existing NTMs to achieve good document representation and coherent/diverse topics at the same time. Moreover, they often degrade their performance severely on short documents. The requirement of reparameterisation could also comprise their training quality and model flexibility. To address these shortcomings, we present a new neural topic model via the theory of optimal transport (OT). Specifically, we propose to learn the topic distribution of a document by directly minimising its OT distance to the document's word distributions. Importantly, the cost matrix of the OT distance models the weights between topics and words, which is constructed by the distances between topics and words in an embedding space. Our proposed model can be trained efficiently with a differentiable loss. Extensive experiments show that our framework significantly outperforms the state-of-the-art NTMs on discovering more coherent and diverse topics and deriving better document representations for both regular and short texts.

Representation

Source

Cite

Citations (1)

Frequency Attention for Knowledge Distillation

2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2024)

Cuong The Pham Van Anh Nguyen Trung Le Dinh Phung Gustavo Carneiro

Knowledge distillation is an attractive approach for learning compact deep neural networks, which learns a lightweight student model by distilling knowledge from a complex teacher model. Attention-based knowledge distillation is a specific form of intermediate feature-based knowledge distillation that uses attention mechanisms to encourage the student to better mimic the teacher. However, most of the previous attention-based distillation approaches perform attention in the spatial domain, which primarily affects local regions in the input image. This may not be sufficient when we need to capture the broader context or global information necessary for effective knowledge transfer. In frequency domain, since each frequency is determined from all pixels of the image in spatial domain, it can contain global information about the image. Inspired by the benefits of the frequency domain, we propose a novel module that functions as an attention mechanism in the frequency domain. The module consists of a learnable global filter that can adjust the frequencies of student's features under the guidance of the teacher's features, which encourages the student's features to have patterns similar to the teacher's features. We then propose an enhanced knowledge review-based distillation model by leveraging the proposed frequency attention module. The extensive experiments with various teacher and student architectures on image classification and object detection benchmark datasets show that the proposed approach outperforms other knowledge distillation methods.

10.1109/wacv57701.2024.00227

Cite

Citations (7)

GoGP: scalable geometric-based Gaussian process for online regression

Knowledge and Information Systems (2018)

Trung Le Khanh Nguyen Vu Nguyen Tu Dinh Nguyen Dinh Phung

Kernelization

Overfitting

Kernel (algebra)

10.1007/s10115-018-1239-1

Cite

Citations (3)

Bayesian Multi-Hyperplane Machine for Pattern Recognition

2022 26th International Conference on Pattern Recognition (ICPR) (2018)

Khanh Nguyen Trung Le Tu Nguyen Dinh Dinh Phung

Current existing multi-hyperplane machine approach deals with high-dimensional and complex datasets by approximating the input data region using a parametric mixture of hyperplanes. Consequently, this approach requires an excessively time-consuming parameter search to find the set of optimal hyper-parameters. Another serious drawback of this approach is that it is often suboptimal since the optimal choice for the hyper-parameter is likely to lie outside the searching space due to the space discretization step required in grid search. To address these challenges, we propose in this paper BAyesian Multi-hyperplane Machine (BAMM). Our approach departs from a Bayesian perspective, and aims to construct an alternative probabilistic view in such a way that its maximum-a-posteriori (MAP) estimation reduces exactly to the original optimization problem of a multi-hyperplane machine. This view allows us to endow prior distributions over hyper-parameters and augment auxiliary variables to efficiently infer model parameters and hyper-parameters via Markov chain Monte Carlo (MCMC) method. We then employ a Stochastic Gradient Descent (SGD) framework to scale our model up with ever-growing large datasets. Extensive experiments demonstrate the capability of our proposed method in learning the optimal model without using any parameter tuning, and in achieving comparable accuracies compared with the state-of-art baselines; in the meantime our model can seamlessly handle with large-scale datasets.

Hyperplane

Hyperparameter Optimization

Stochastic Gradient Descent

10.1109/icpr.2018.8545139

Cite

Citations (1)

Fast support vector clustering

Vietnam Journal of Computer Science (2016)

Tung Pham Hang T. Dang Trung Le Thái Hoàng Lê

Support-based clustering has recently absorbed plenty of attention because of its applications in solving the difficult and diverse clustering or outlier detection problem. Support-based clustering method perambulates two phases: finding the domain of novelty and performing the clustering assignment. To find the domain of novelty, the training time given by the current solvers is typically over-quadratic in the training size. This fact impedes the application of support-based clustering method to the large-scale datasets. In this paper, we propose applying stochastic gradient descent framework to the first phase of support-based clustering for finding the domain of novelty in the form of a half-space and a new strategy to perform the clustering assignment. We validate our proposed method on several well-known datasets for clustering task to show that the proposed method renders a comparable clustering quality to the baselines while being faster than them.

Constrained clustering

Data stream clustering

Clustering high-dimensional data

10.1007/s40595-016-0068-y

Cite

Citations (3)