Observation of nonlinear dynamics in an optical levitation system

Optical levitation of mechanical oscillators has been suggested as a promising way to decouple the environmental noise and increase the mechanical quality factor. Here, we investigate the dynamics of a free-standing mirror acting as the top reflector of a vertical optical cavity, designed as a testbed for a tripod cavity optical levitation setup. To reach the regime of levitation for a milligram-scale mirror, the optical intensity of the intracavity optical field approaches 3 MW cm$^{-2}$. We identify three distinct optomechanical effects: excitation of acoustic vibrations, expansion due to photothermal absorption, and partial lift-off of the mirror due to radiation pressure force. These effects are intercoupled via the intracavity optical field and induce complex system dynamics inclusive of high-order sideband generation, optical bistability, parametric amplification, and the optical spring effect. We modify the response of the mirror with active feedback control to improve the overall stability of the system.