Super-resolution of spatiotemporal event-stream image

Abstract Super-resolution (SR) is a useful technology to generate a high-resolution (HR) visual output from the low-resolution (LR) visual inputs overcoming the physical limitations of the cameras. However, SR has not been applied to enhance the spatial resolution of event-stream images captured by the frame-free dynamic vision sensor (DVS). SR of event-stream image aims to recover the same statistics of events which is fundamentally different from the existing frame-based schemes. In this work, a two-stage scheme is proposed to solve the spatial SR problem of the spatiotemporal event-stream image. We use a nonhomogeneous Poisson point process to model the event sequence, and sample the events of each pixel by simulating a nonhomogeneous Poisson process according to the specified event number and rate function. Firstly, the event number of each pixel of the HR DVS image is generated by obtaining the HR event-count map (ECM) from the LR DVS recording with a sparse representation based method. The rate function over time line of the point process of each HR pixel is computed using a spatiotemporal filter on the corresponding LR neighbor pixels. Secondly, the event sequence of each new pixel is obtained with a thinning based event sampling algorithm. A metric is proposed to assess the event-stream SR quality. The effectiveness of the proposed method is demonstrated through obtaining HR event-stream images from a series of DVS recordings. This work enables many potential.