Processing Near Sensor Architecture in Mixed-Signal Domain With CMOS Image Sensor of Convolutional-Kernel-Readout Method

In the era of Artificial Intelligence (AI), bio-inspired perceptual computing system design brings favorable opportunities, while still facing considerable challenges in the meantime. Especially for tasks of image recognition in power-limited vision-based Internet of Things (IoT) devices, energy constraints due to the end of Dennard scaling limit the performance of Neural Network (NN) algorithms on popular digital platforms, which would not reach the energy efficiency requirement for embedded AI applications. In this paper, a processing near sensor architecture in mixed-signal domain with CMOS Image Sensor (CIS) of convolutional-kernel-readout method is proposed. Visual data is collected from a smart CIS, which can realize maximum $5 x5$ kernel-readout with minimum one slide step for convolutional operations. The outputs of CIS are directly processed by analog processing units locating near CIS without the constraint of digital clock and bottleneck of Analog-to-Digital Converter (ADC). By analyzing the effects of analog noise on classification accuracy, we further evaluate the fault-tolerance of the system to circuit noise and the device imperfection, such as mismatch and process variation. A mixed-signal visual perception chip is fabricated with a 32 x 32 image sensor and a Binarized Neural Network (BNN) processing array integrated with SMIC 180nm standard CMOS mixed-signal process. Measurement results show up to 545.4 GOPS/W energy efficiency with 1.8mW power consumption taking the advantages of ADC-free processing architecture. This work provides a promising alternative for low-power vision-based IoT intelligent applications.