A 0.8 V Intelligent Vision Sensor With Tiny Convolutional Neural Network and Programmable Weights Using Mixed-Mode Processing-in-Sensor Technique for Image Classification

This article presents an intelligent vision sensor (IVS) with embedded tiny convolutional neural network (CNN) model and programmable processing-in-sensor (PIS) circuit for real-time inference applications of low-power edge devices. The proposed imager realizes the full computing functions of a customized three-layers tiny network, which includes a $3 imes 3$ convolution layer (stride $=$ 3) with activation function of rectified linear unit (ReLU), a $2 imes 2$ maximum pooling (MP) layer (stride $=$ 2), and a $1 imes 1$ fully connected (FC) layer for inference. A 0.8 V $128 imes 128$ IVS prototype was fabricated and verified in TSMC 0.18 $mu$ m standard CMOS technology. In normal image mode, it consumed 76.4 $mu$ W with full-resolution ( $126 imes 126$ active resolution) image output at 125 f/s. In CNN mode, it consumed 134.5 $mu$ W at 250 f/s and an achieved iFoMs of 33.8 pJ/pixel $cdot$ frame. Using the proposed mixed-mode PIS circuits, the prototype is configured to demonstrate a &null face or not detection&null task with an achieved accuracy of 93.6%.