Operator-induced structural variable selection with applications to materials genomes

We propose a new method for variable selection with operator-induced structure (OIS), in which the predictors are engineered from a limited number of primary variables and a set of elementary algebraic operators through compositions. Standard practice directly analyzes the high-dimensional candidate predictor space in a linear model; statistical analyses are then substantially hampered by the daunting challenge posed by millions of correlated predictors with limited sample size. The proposed method iterates nonparametric variable selection to achieve effective dimension reduction in linear models by utilizing the geometry embedded in OIS. This enables variable selection based on \textit{ab initio} primary variables, leading to a method that is orders of magnitude faster than existing methods, with improved accuracy. The proposed method is well suited for areas that adopt feature engineering and emphasize interpretability in addition to prediction, such as the emerging field of materials informatics. We demonstrate the superior performance of the proposed method in simulation studies and a real data application to single-atom catalyst analysis. An OIS screening property for variable selection methods in the presence of feature engineering is introduced; interestingly, finite sample assessment indicates that the employed Bayesian Additive Regression Trees (BART)-based variable selection method enjoys this property. Our numerical experiments show that the proposed method exhibits robust performance when the dimension of engineered features is out of reach of existing methods.