Easy-but-effective Domain Sub-similarity Learning for Transfer Regression

Transfer covariance functions, which can model domain similarities and adaptively control the knowledge transfer across domains, are widely used in transfer learning. In this paper, we concentrate on Gaussian process ( ${{GP}}$ ) models using a transfer covariance function for regression problems in a black-box learning scenario. Precisely, we investigate a family of rather general transfer covariance functions, ${{T}_{*}}$ , that can model the heterogeneous sub-similarities of domains through multiple kernel learning. A necessary and sufficient condition to obtain valid ${{GP}}$ s using ${{T}_{*}}$ ( ${GP_{T_{*}}}$ ) for any data is given. This condition becomes specially handy for practical applications as (i) it enables semantic interpretations of the sub-similarities and (ii) it can readily be used for model learning. In particular, we propose a computationally inexpensive model learning rule that can explicitly capture different sub-similarities of domains. We propose two instantiations of ${GP_{T_{*}}}$ , one with a set of predefined constant base kernels and one with a set of learnable parametric base kernels. Extensive experiments on 36 synthetic transfer tasks and 12 real-world transfer tasks demonstrate the effectiveness of ${GP_{T_{*}}}$ on the sub-similarity capture and the transfer performance.