Predicting observed and hidden extreme events in complex nonlinear dynamical systems with partial observations and short training time series

Extreme events appear in many complex nonlinear dynamical systems. Predicting extreme events has important scientific significance and large societal impacts. In this paper, a new mathematical framework of building suitable nonlinear approximate models is developed, which aims at predicting both the observed and hidden extreme events in complex nonlinear dynamical systems for short-, medium-, and long-range forecasting using only short and partially observed training time series. Different from many ad hoc data-driven regression models, these new nonlinear models take into account physically motivated processes and physics constraints. They also allow efficient and accurate algorithms for parameter estimation, data assimilation, and prediction. Cheap stochastic parameterizations, judicious linear feedback control, and suitable noise inflation strategies are incorporated into the new nonlinear modeling framework, which provide accurate predictions of both the observed and hidden extreme events as well as the strongly non-Gaussian statistics in various highly intermittent nonlinear dyad and triad models, including the Lorenz 63 model. Then, a stochastic mode reduction strategy is applied to a 21-dimensional nonlinear paradigm model for topographic mean flow interaction. The resulting five-dimensional physics-constrained nonlinear approximate model is able to accurately predict extreme events and the regime switching between zonally blocked and unblocked flow patterns. Finally, incorporating judicious linear stochastic processes into a simple nonlinear approximate model succeeds in learning certain complicated nonlinear effects of a six-dimensional low-order Charney–DeVore model with strong chaotic and regime switching behavior. The simple nonlinear approximate model then allows accurate online state estimation and the short- and medium-range forecasting of extreme events.Extreme events appear in many complex nonlinear dynamical systems. Predicting extreme events has important scientific significance and large societal impacts. In this paper, a new mathematical framework of building suitable nonlinear approximate models is developed, which aims at predicting both the observed and hidden extreme events in complex nonlinear dynamical systems for short-, medium-, and long-range forecasting using only short and partially observed training time series. Different from many ad hoc data-driven regression models, these new nonlinear models take into account physically motivated processes and physics constraints. They also allow efficient and accurate algorithms for parameter estimation, data assimilation, and prediction. Cheap stochastic parameterizations, judicious linear feedback control, and suitable noise inflation strategies are incorporated into the new nonlinear modeling framework, which provide accurate predictions of both the observed and hidden extreme events as well as t...