Denoising and retrieval algorithm based on a dual ensemble Kalman filter for elastic lidar data

Abstract The intensity of a lidar signal decreases with transmittance and the square of detection range. Consequently, the effective measure range and retrieval accuracy are severely affected. A method for denoising and retrieval of lidar data is proposed in this study by combining dual ensemble Kalman filter (DEnKF) and Fernald methods to avoid the abovementioned issue. Compared with ensemble Kalman filter (EnKF) method, the DEnKF method provides a feedback function in the iteration; thus, the DEnKF method provides a generally improved accuracy of denoising and retrieval. We select an ensemble size of 60 and determine the covariance δ on the basis of the defined performance function. The DEnKF, EnKF and standard Fernald methods are tested using complex simulated and real signals. Results show that aerosol backscatter coefficient retrieved through the DEnKF method demonstrates lower uncertainty in the far range (above 4 km) than the coefficients obtained through the two other methods and fits the results retrieved through the two other methods in the near range (below 4 km). In addition, the results indicate that the retrieval results are better through the DEnKF method than through the 64 min averaged signals, which can divide the standard error thrice (i.e. averaging 64 replications). Overall, the results demonstrate that the DEnKF method is effective and useful for retrieving signals with low signal-to-noise ratios, such as the far-range signals of a ground lidar and full-range signals of a space lidar.