Gaussian Process Structure Learning via Probabilistic Inverse Compilation

There is a widespread need for techniques that can learn interpretable models from data. Recent work by Duvenaud et al. (2013) and Lloyd et al. (2014) showed that it is possible to use Gaussian Processes (GPs) to discover symbolic structure in univariate time series. This abstract shows how to reimplement the approach from Duvenaud et al. (2013) using under 100 lines of probabilistic code in Venture (Mansinghka et al., 2014; Lu, 2016), improving on a previous implementation from Schaechtle et al. (2015). The key idea is to formulate structure learning as a kind of probabilistic inverse compilation, where the kernel structure is represented as source code and the resulting GP model is represented as an executable probabilistic program produced by compiling that source code. Figures 1 and 2 give an overview of the inverse compilation framework. Figure 3 shows example kernel structures, including program source, an English summary, and typical data corresponding to the given structure. Figure 4 shows the complete Venture source code for reimplementing the approach from Duvenaud et al. (2013), and Figure 5 shows an application to real-world time series data describing air travel volume.