Fine-Grained Video Deblurring with Event Camera

Despite CNN-based deblurring models have shown their superiority on solving motion blurs, how to restore photorealistic images from severe motion blurs remains an ill-posed problem due to the loss of temporal information and textures. Video deblurring methods try to extract meaningful temporal information in multiple consecutive blurry frames during a long period of time. However, the information obtained in such a coarse period is overcomplicated, and all frames may suffer from severe motion blurs. Event cameras such as Dynamic and Active Pixel Vision Sensor (DAVIS) can simultaneously produce gray-scale Active Pixel Sensor (APS) frames and events, which capture motions as the events at very high temporal resolution, i.e., 1\\, \\upmu \\mathrm{s} , and provide useful information for blurry APS frames. In this paper, we propose a deep fine-grained video deblurring pipeline consisting of a deblurring module and a recurrent module to address severe motion blurs. Concatenating the blurry image with event representations at a fine-grained temporal period, our proposed model achieves state-of-the-art performance on both popular GoPro and real blurry datasets captured by DAVIS, and is capable of generating high frame-rate video by applying a tiny shift to event representations in the recurrent module.