Search for rare decays of and Higgs bosons to and a photon in proton-proton collisions at = 13

A search is presented for decays of and Higgs bosons to a meson and a photon, with the subsequent decay of the to . The analysis uses data from proton-proton collisions with an integrated luminosity of 35.9 at collected with the CMS detector at the LHC. The observed limit on the decay branching fraction, assuming that the meson is produced unpolarized, is at 95% confidence level, which corresponds to a rate higher than expected in the standard model by a factor of 15. For extreme-polarization scenarios, the observed limit changes from to with respect to the unpolarized scenario. The observed upper limit on the branching fraction for where the meson is assumed to be transversely polarized is , a factor of 260 larger than the standard model prediction. The results for the Higgs boson are combined with previous data from proton-proton collisions at to produce an observed upper limit on the branching fraction for that is a factor of 220 larger than the standard model value.