Estimation and optimal structure selection of high-dimensional Toeplitz covariance matrix
7
Citation
52
Reference
10
Related Paper
Citation Trend
Keywords:
Matérn covariance function
CMA-ES
Covariance mapping
Abstract Data analysis in cosmology requires reliable covariance matrices. Covariance matrices derived from numerical simulations often require a very large number of realizations to be accurate. When a theoretical model for the covariance matrix exists, the parameters of the model can often be fit with many fewer simulations. We write a likelihood-based method for performing such a fit. We demonstrate how a model covariance matrix can be tested by examining the appropriate χ 2 distributions from simulations. We show that if model covariance has amplitude freedom, the expectation value of second moment of χ 2 distribution with a wrong covariance matrix will always be larger than one using the true covariance matrix. By combining these steps together, we provide a way of producing reliable covariances without ever requiring running a large number of simulations. We demonstrate our method on two examples. First, we measure the two-point correlation function of halos from a large set of 10000 mock halo catalogs. We build a model covariance with 2 free parameters, which we fit using our procedure. The resulting best-fit model covariance obtained from just 100 simulation realizations proves to be as reliable as the numerical covariance matrix built from the full 10000 set. We also test our method on a setup where the covariance matrix is large by measuring the halo bispectrum for thousands of triangles for the same set of mocks. We build a block diagonal model covariance with 2 free parameters as an improvement over the diagonal Gaussian covariance. Our model covariance passes the χ 2 test only partially in this case, signaling that the model is insufficient even using free parameters, but significantly improves over the Gaussian one.
Matérn covariance function
CMA-ES
Scatter matrix
Covariance mapping
Cite
Citations (7)
Matérn covariance function
Covariance mapping
Interpolation
Cite
Citations (29)
Data analysis in cosmology requires reliable covariance matrices. Covariance matrices derived from numerical simulations often require a very large number of realizations to be accurate. When a theoretical model for the covariance matrix exists, the parameters of the model can often be fit with many fewer simulations. We write a likelihood-based method for performing such a fit. We demonstrate how a model covariance matrix can be tested by examining the appropriate $χ^2$ distributions from simulations. We show that if model covariance has amplitude freedom, the expectation value of second moment of $χ^2$ distribution with a wrong covariance matrix will always be larger than one using the true covariance matrix. By combining these steps together, we provide a way of producing reliable covariances without ever requiring running a large number of simulations. We demonstrate our method on two examples. First, we measure the two-point correlation function of halos from a large set of $10000$ mock halo catalogs. We build a model covariance with $2$ free parameters, which we fit using our procedure. The resulting best-fit model covariance obtained from just $100$ simulation realizations proves to be as reliable as the numerical covariance matrix built from the full $10000$ set. We also test our method on a setup where the covariance matrix is large by measuring the halo bispectrum for thousands of triangles for the same set of mocks. We build a block diagonal model covariance with $2$ free parameters as an improvement over the diagonal Gaussian covariance. Our model covariance passes the $χ^2$ test only partially in this case, signaling that the model is insufficient even using free parameters, but significantly improves over the Gaussian one.
Matrix (chemical analysis)
Cite
Citations (1)
Variograms and covariance functions are key tools in geostatistics. However, various properties, characterizations, and decomposition theorems have been established for covariance functions only. We present analogous results for variograms and explore the connections with covariance functions. Our findings include criteria for covariance functions on intervals, and we apply them to exponential models, fractional Brownian motion, and locally polynomial covariances. In particular, we characterize isotropic locally polynomial covariance functions of degree 3.
Matérn covariance function
Covariance mapping
Geostatistics
Cite
Citations (64)
It is proved in this paper that covariance hypotheses which are linear in both the covariance and the inverse covariance are products of models each of which consists of either (i) independent identically distributed random vectors which have a covariance with a real, complex or quaternion structure or (ii) independent identically distributed random vectors with a parametrization of the covariance which is given by means of the Clifford algebra. The models (i) are well known. For models (ii) we have found, under the assumption that the distribution is normal, the exact distributions of the maximum likelihood estimates and the likelihood ratio test statistics.
Matérn covariance function
Covariance mapping
Cite
Citations (71)
Matérn covariance function
Covariance mapping
Collocation (remote sensing)
Cite
Citations (40)
We present a method for fast evaluation of the covariance matrix for a two-point galaxy correlation function (2PCF) measured with the Landy-Szalay estimator. The standard way of evaluating the covariance matrix consists in running the estimator on a large number of mock catalogs, and evaluating their sample covariance. With large random catalog sizes (data-to-random objects ratio M>>1) the computational cost of the standard method is dominated by that of counting the data-random and random-random pairs, while the uncertainty of the estimate is dominated by that of data-data pairs. We present a method called Linear Construction (LC), where the covariance is estimated for small random catalogs of size M = 1 and M = 2, and the covariance for arbitrary M is constructed as a linear combination of these. We validate the method with PINOCCHIO simulations in range r = 20-200 Mpc/h, and show that the covariance estimate is unbiased. With M = 50 and with 2 Mpc/h bins, the theoretical speed-up of the method is a factor of 14. We discuss the impact on the precision matrix and parameter estimation, and derive a formula for the covariance of covariance.
Matérn covariance function
Covariance mapping
Covariance and correlation
Cite
Citations (2)
This chapter contains sections titled: Specified Covariance Matrix Sphericity Intraclass Covariance Structure Test for Independence Tests for Equality of Covariance Matrices
Sphericity
Matérn covariance function
Independence
CMA-ES
Covariance mapping
Matrix (chemical analysis)
Cite
Citations (0)
The authors derived a best linear unbiased estimate (BLUE) for the estimation of the noise covariance matrix, where the covariance matrix of the residuals of the linear regression equation is required. In the previous paper, the sample covariance of the residuals was given, but the consistency of the sample covariance was not analyzed. In this paper, theoretical analysis shows that the proposed sample covariance matrix is a consistent estimate of the covariance matrix in a certain sense.
Matérn covariance function
Covariance mapping
Cite
Citations (0)
Variograms and covariance functions are key tools in geostatistics. However, various properties, characterizations, and decomposition theorems have been established for covariance functions only. We present analogous results for variograms and explore the connections with covariance functions. Our findings include criteria for covariance functions on intervals, and we apply them to exponential models, fractional Brownian motion, and locally polynomial covariances. In particular, we characterize isotropic locally polynomial covariance functions of degree 3.
Matérn covariance function
Covariance mapping
Geostatistics
Cite
Citations (14)