Qing Tao

University of Oklahoma

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Trends

Author Order

Document Type

Co-Authors

Gaowei Wu

Chinese Academy of Sciences

Wei Tao

China Earthquake Administration

Zhisong Pan

PLA Army Engineering University

Dejun Chu

Hefei University of Technology

Dun Liu

Hubei University of Technology

Jue Wang

Tsinghua University

Ningning Zhou

Nanjing University of Posts and Telecommunications

Jian Cheng

Hubei University of Technology

Linmao Qian

Southwest Jiaotong University

Deyuan Lou

Hubei University of Technology

Cooperative Institutions

Chinese Academy of Sciences

Tsinghua University

Hubei University of Technology

University of Chinese Academy of Sciences

University of Science and Technology of China

China University of Mining and Technology

Peking University

Huazhong University of Science and Technology

Southwest Jiaotong University

University of Oklahoma

Author Statistics

Papers

Citation

H-Index

i-10 index

Research Field

The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods

arXiv (Cornell University) (2021)

Wei Tao Sheng Long Gaowei Wu Qing Tao

The adaptive stochastic gradient descent (SGD) with momentum has been widely adopted in deep learning as well as convex optimization. In practice, the last iterate is commonly used as the final solution to make decisions. However, the available regret analysis and the setting of constant momentum parameters only guarantee the optimal convergence of the averaged solution. In this paper, we fill this theory-practice gap by investigating the convergence of the last iterate (referred to as individual convergence), which is a more difficult task than convergence analysis of the averaged solution. Specifically, in the constrained convex cases, we prove that the adaptive Polyak's Heavy-ball (HB) method, in which only the step size is updated using the exponential moving average strategy, attains an optimal individual convergence rate of $O(\frac{1}{\sqrt{t}})$, as opposed to the optimality of $O(\frac{\log t}{\sqrt {t}})$ of SGD, where $t$ is the number of iterations. Our new analysis not only shows how the HB momentum and its time-varying weight help us to achieve the acceleration in convex optimization but also gives valuable hints how the momentum parameters should be scheduled in deep learning. Empirical results on optimizing convex functions and training deep networks validate the correctness of our convergence analysis and demonstrate the improved performance of the adaptive HB methods.

Momentum (technical analysis)

Ball (mathematics)

10.48550/arxiv.2102.07314

Cite

Citations (6)

Intellectual property rights protection and quality upgrading: Evidence from China

Economic Modelling (2021)

Xueyin Song Xianhai Huang Qing Tao

10.1016/j.econmod.2021.105602

Cite

Citations (14)

A new maximum margin algorithm for one-class problems and its boosting implementation

Pattern Recognition (2005)

Qing Tao Gaowei Wu Jue Wang

Boosting

Margin (machine learning)

Complement

10.1016/j.patcog.2004.10.010

Cite

Citations (19)

The Recognition of Radar Trajectory Based on the Boosting Learning Algorithms

Qing Tao

Trajectory prediction plays a crucial role in reconnaissance and adjustment of radar,and the performance of radar for reconnaissance and adjustment is directly determined by its accuracy.In our paper[1],it was proposed that the stage of trajectory prediction can be divided into the recognition phase and prediction phase of specific trajectories,and the application of SVM in trajectory recognition was systematically investigated.In this paper,the Boosting classification technique was introduced to recognize the trajectories.Several experiments indicate that the efficient decision-tree-based Boosting algorithms reach higher precision than kernel-based SVM.

Boosting

Gradient boosting

Cite

Citations (0)

Kernel covering algorithm and a design principle for feed-forward neural networks

Gaowei Wu Qing Tao Jue Wang

Kernel technique supplies a systematic and principled approach to training learning machines and the good generalization performance achieved can be readily justified using statistical learning theory. In this paper, we convert classification problem into a set cover one, present a kernel covering algorithm which combines kernel technique with covering approach. This algorithm is constructive, and bypasses the problems of convergence and convergence speed. Analyzing the statistical properties of the covering classifier, we offer a bound of the actual risk. In virtue of the variety of kernels, a general design principle for feed-forward neural networks is drawn.

Constructive

Kernel (algebra)

Statistical learning theory

10.1109/iconip.2002.1198223

Cite

Citations (0)

The Optimal Individual Convergence Rate for the Projected Subgradient Method with Linear Interpolation Operation

Journal of Computer Research and Development (2017)

Wei Tao Zhisong Pan Zhu Xiao-hui Qing Tao

Subgradient method

Interpolation

10.7544/issn1000-1239.2017.20160155

Cite

Citations (1)

The theoretical analysis of kernel technique and its applications

Qing Tao Jiaqi Wang Gaowei Wu Jue Wang

Investigates linear separability in feature space from Tietze extension theorem and function approximation theory, and kernel technique is motivated theoretically. Two kernel-based algorithms are presented. One of them is a general learning algorithm for feedforward neural networks, and they can solve large-scale classification problems.

Kernel (algebra)

Feedforward neural network

Graph kernel

Feature vector

10.1109/ijcnn.2002.1005535

Cite

Citations (2)

Adapting Step-size: A Unified Perspective to Analyze and Improve Gradient-based Methods for Adversarial Attacks

arXiv (Cornell University) (2023)

Tao Wei Lei Bao Long Sheng Gaowei Wu Qing Tao

Learning adversarial examples can be formulated as an optimization problem of maximizing the loss function with some box-constraints. However, for solving this induced optimization problem, the state-of-the-art gradient-based methods such as FGSM, I-FGSM and MI-FGSM look different from their original methods especially in updating the direction, which makes it difficult to understand them and then leaves some theoretical issues to be addressed in viewpoint of optimization. In this paper, from the perspective of adapting step-size, we provide a unified theoretical interpretation of these gradient-based adversarial learning methods. We show that each of these algorithms is in fact a specific reformulation of their original gradient methods but using the step-size rules with only current gradient information. Motivated by such analysis, we present a broad class of adaptive gradient-based algorithms based on the regular gradient methods, in which the step-size strategy utilizing information of the accumulated gradients is integrated. Such adaptive step-size strategies directly normalize the scale of the gradients rather than use some empirical operations. The important benefit is that convergence for the iterative algorithms is guaranteed and then the whole optimization process can be stabilized. The experiments demonstrate that our AdaI-FGM consistently outperforms I-FGSM and AdaMI-FGM remains competitive with MI-FGSM for black-box attacks.

10.48550/arxiv.2301.11546

Cite

Citations (1)

A dynamic genetic algorithm based on continuous neural networks for a kind of non-convex optimization problems

Applied Mathematics and Computation (2003)

Qing Tao Xin Liu Meisheng Xue

10.1016/s0096-3003(03)00309-6

Cite

Citations (22)

Robust and Adaptive Online Time Series Prediction with Long Short-Term Memory

Computational Intelligence and Neuroscience (2017)

Haimin Yang Zhisong Pan Qing Tao

Online time series prediction is the mainstream method in a wide range of fields, ranging from speech analysis and noise cancelation to stock market analysis. However, the data often contains many outliers with the increasing length of time series in real world. These outliers can mislead the learned model if treated as normal points in the process of prediction. To address this issue, in this paper, we propose a robust and adaptive online gradient learning method, RoAdam (Robust Adam), for long short-term memory (LSTM) to predict time series with outliers. This method tunes the learning rate of the stochastic gradient algorithm adaptively in the process of prediction, which reduces the adverse effect of outliers. It tracks the relative prediction error of the loss function with a weighted average through modifying Adam, a popular stochastic gradient method algorithm for training deep neural networks. In our algorithm, the large value of the relative prediction error corresponds to a small learning rate, and vice versa. The experiments on both synthetic data and real time series show that our method achieves better performance compared to the existing methods based on LSTM.

10.1155/2017/9478952

Cite

Citations (46)