Robust Higher Order Statistics.
33
Citation
9
Reference
20
Related Paper
Citation Trend
Abstract:
Sample estimates of moments and cumulants are known to be unstable in the presence of outliers. This problem is especially severe for higher order statistics, like kurtosis, which are used in algorithms for independent components analysis and projection pursuit. In this paper we propose robust generalizations of moments and cumulants that are more insensitive to outliers but at the same time retain many of their desirable properties. We show how they can be combined into series expansions to provide estimates of probability density functions. This in turn is directly relevant for the design of new robust algorithms for ICA. We study the improved statistical properties such as B-robustness, bias and variance while in experiments we demonstrate their improved behavior.Keywords:
Cumulant
Robustness
Higher-order statistics
Kurtosis
Robust Statistics
Projection pursuit
Cite
The paper considers an application of blind identification to beamforming. The key point is to use estimates of directional vectors rather than resort to their hypothesised value. By using estimates of the directional vectors obtained via blind identification, i.e. without knowing the array manifold, beamforming is made robust with respect to array deformations, distortion of the wave front, pointing errors etc., so that neither array calibration nor physical modelling is necessary. Rather suprisingly, 'blind beamformers' may outperform 'informed beamformers' in a plausible range of parameters, even when the array is perfectly known to the informed beamformer. The key assumption on which blind identification relies is the statistical independence of the sources, which is exploited using fourth-order cumulants. A computationally efficient technique is presented for the blind estimation of directional vectors, based on joint diagonalisation of fourth-order cumulant matrices; its implementation is described, and its performance is investigated by numerical experiments.
Identification
Distortion (music)
Cumulant
Cite
Citations (2,729)
Median-of-means (MOM) based procedures have been recently introduced in learning theory (Lugosi and Mendelson (2019); Lecué and Lerasle (2017)). These estimators outperform classical least-squares estimators when data are heavy-tailed and/or are corrupted. None of these procedures can be implemented, which is the major issue of current MOM procedures (Ann. Statist. 47 (2019) 783–794). In this paper, we introduce minmax MOM estimators and show that they achieve the same sub-Gaussian deviation bounds as the alternatives (Lugosi and Mendelson (2019); Lecué and Lerasle (2017)), both in small and high-dimensional statistics. In particular, these estimators are efficient under moments assumptions on data that may have been corrupted by a few outliers. Besides these theoretical guarantees, the definition of minmax MOM estimators suggests simple and systematic modifications of standard algorithms used to approximate least-squares estimators and their regularized versions. As a proof of concept, we perform an extensive simulation study of these algorithms for robust versions of the LASSO.
Robust Statistics
Cite
Citations (97)
Robustness
Cite
Citations (0)
Whole robustness is an appealing attribute to look for in statistical models. It implies that the impact of outliers, defined here as the observations that are not in line with the general trend, gradually vanishes as they move further and further away from this trend. Consequently, the conclusions obtained are consistent with the bulk of the data. Nonrobust models may lead to inferences that are not in line with either the outliers or the nonoutliers. In this paper, we make the following contribution: we generalise existing whole robustness results for simple linear regression through the origin to the usual linear regression model. The strategy to attain whole robustness is simple: replace the traditional normal assumption on the error term by a super heavy-tailed distribution assumption. Analyses are then conducted as usual, and typical methods of inference as statistical hypothesis testing are available.
Robustness
Statistical Inference
Robust regression
Cite
Citations (1)
Part 1 Probability and Random Variables 1 The Meaning of Probability 2 The Axioms of Probability 3 Repeated Trials 4 The Concept of a Random Variable 5 Functions of One Random Variable 6 Two Random Variables 7 Sequences of Random Variables 8 Statistics Part 2 Stochastic Processes 9 General Concepts 10 Random Walk and Other Applications 11 Spectral Representation 12 Spectral Estimation 13 Mean Square Estimation 14 Entropy 15 Markov Chains 16 Markov Processes and Queueing Theory
Cite
Citations (16,610)
The Classical Tukey-Huber Contamination Model (CCM) is a usual framework to describe the mechanism of outliers generation in robust statistics. In a data set with $n$ observations and $p$ variables, under the CCM, an outlier is a unit, even if only one or few values are corrupted. Classical robust procedures were designed to cope with this setting and the impact of observations were limited whenever necessary. Recently, a different mechanism of outliers generation, namely Independent Contamination Model (ICM), was introduced. In this new setting each cell of the data matrix might be corrupted or not with a probability independent on the status of the other cells. ICM poses new challenge to robust statistics since the percentage of contaminated rows dramatically increase with $p$, often reaching more than $50\%$. When this situation appears, classical affine equivariant robust procedures do not work since their breakdown point is $50\%$. For this contamination model we propose a new type of robust methods namely composite robust procedures which are inspired on the idea of composite likelihood, where low dimension likelihood, very often the likelihood of pairs, are aggregate together in order to obtain an approximation of the full likelihood which is more tractable. Our composite robust procedures are build over pairs of observations in order to gain robustness in the independent contamination model. We propose composite S and $\tau$-estimators for linear mixed models. Composite $\tau$-estimators are proved to have an high breakdown point both in the CCM and ICM. A Monte Carlo study shows that our estimators compare favorably with respect to classical S-estimators under the CCM and outperform them under the ICM. One example based on a real data set illustrates the new robust procedure.
Robustness
Robust Statistics
Cite
Citations (0)
An important challenge in big data is identification of important variables. In this paper, we propose methods of discovering variables with non-standard univariate marginal distributions. The conventional moments-based summary statistics can be well-adopted for that purpose, but their sensitivity to outliers can lead to selection based on a few outliers rather than distributional shape such as bimodality. To address this type of non-robustness, we consider the L-moments. Using these in practice, however, has a limitation because they do not take zero values at the Gaussian distributions to which the shape of a marginal distribution is most naturally compared. As a remedy, we propose Gaussian Centered L-moments which share advantages of the L-moments but have zeros at the Gaussian distributions. The strength of Gaussian Centered L-moments over other conventional moments is shown in theoretical and practical aspects such as their performances in screening important genes in cancer genetics data.
Univariate
Robustness
Bimodality
Cite
Citations (0)
The ratio between two probability density functions is an important component of various tasks, including selection bias correction, novelty detection and classification. Recently, several estimators of this ratio have been proposed. Most of these methods fail if the sample space is high-dimensional, and hence require a dimension reduction step, the result of which can be a significant loss of information. Here we propose a simple-to-implement, fully nonparametric density ratio estimator that expands the ratio in terms of the eigenfunctions of a kernel-based operator; these functions reflect the underlying geometry of the data (e.g., submanifold structure), often leading to better estimates without an explicit dimension reduction step. We show how our general framework can be extended to address another important problem, the estimation of a likelihood function in situations where that function cannot be well-approximated by an analytical form. One is often faced with this situation when performing statistical inference with data from the sciences, due the complexity of the data and of the processes that generated those data. We emphasize applications where using existing likelihood-free methods of inference would be challenging due to the high dimensionality of the sample space, but where our spectral series method yields a reasonable estimate of the likelihood function. We provide theoretical guarantees and illustrate the effectiveness of our proposed method with numerical experiments.
Intrinsic dimension
Density estimation
Empirical likelihood
Kernel (algebra)
Statistical Inference
Cite
Citations (8)
In the context of robust covariance matrix estimation, this paper proposes a new approach to understanding the behavior of scatter matrix M-estimators introduced during the 70's in the statistics community. During the last decade, a renewed interest for robust estimators appeared in the signal processing community, mainly due to their flexibility to the statistical model and their robustness to outliers and/or missing data. However, the behavior of these estimators still remains unclear and not understood well enough. A major disadvantage is that they are described by fixed-point equations that make their statistical analysis very difficult. To fill this gap, the main contribution of this work is to compare these estimators to the well-known sample covariance matrix (SCM) in order to infer more accurately their statistical behaviors. Indeed, under the Gaussian assumption, the SCM follows a Wishart distribution. However, when observations turn to be non-Gaussian, the SCM performance can be seriously degraded. In this article, we propose to use robust estimators, more adapted to the case of non-Gaussian models and presence of outliers, and to rely on the SCM properties in a Gaussian framework for establishing their theoretical analysis. To confirm our claims we also present results for a widely used function of $M$-estimators, the Mahalanobis distance. Finally, Monte Carlo simulations for various robust scatter matrix estimators are presented to validate theoretical results.
Extremum estimator
Mahalanobis distance
Scatter matrix
Robustness
Cite
Citations (0)
Linear mixed models with large imbalanced crossed random effects structures pose severe computational problems for maximum likelihood estimation and for Bayesian analysis. The costs can grow as fast as $N^{3/2}$ when there are N observations. Such problems arise in any setting where the underlying factors satisfy a many to many relationship (instead of a nested one) and in electronic commerce applications, the N can be quite large. Methods that do not account for the correlation structure can greatly underestimate uncertainty. We propose a method of moments approach that takes account of the correlation structure and that can be computed at O(N) cost. The method of moments is very amenable to parallel computation and it does not require parametric distributional assumptions, tuning parameters or convergence diagnostics. For the regression coefficients, we give conditions for consistency and asymptotic normality as well as a consistent variance estimate. For the variance components, we give conditions for consistency and we use consistent estimates of a mildly conservative variance estimate. All of these computations can be done in O(N) work. We illustrate the algorithm with some data from Stitch Fix where the crossed random effects correspond to clients and items.
Mixed model
Cite
Citations (7)