Abstract
Advanced artificial-intelligence (Al) edge devices require high energy-efficiency (etaE) and high inference-accuracy [2,4-6]. An SRAM-based compute-in-memory (CIM) based on MAC operations is well-suited for improving the etaE of Al edge devices. However, without support for floating-point (FP) computation, Al chips using integer-based SRAM-CIMs (INT-CIM) [2,4-5] are prone to precision loss when applied to complex datasets or neural network models. Product (textPD=textINtimes W). alignment-based FP-MACs align the product's mantissa (textPDM) prior to accumulation, based on the product's exponent (textPDE). This approach is commonly used for digital circuits [3] and for near-memory compute [1], but is not practical for in-memory-compute (IMC) macros: each textPDE within a physical row/column is different and thus cannot be accumulated. An INT-IMC with off-macro digital circuits and off-chip software pre-alignment was used in [6] to process the exponents of inputs (textINE) and weights (WE) externally for the FP-MAC. An INT-CIM with extra FP-to-INT converters can emulate an FP-MAC, but incurs additional area, power consumption, and latency (PPA). Researchers have yet to develop a true FP-IMC macro capable of exponent and mantissa computation. Analog CIMs suffer from a low readout accuracy due to intrinsic transistor variation. Digital CIMs are insensitive to variation, but are limited in terms of compute parallelism due to routing congestion, as Fig. 7.1.1 shows. This paper presents a true FP-IMC macro featuring (1) a hybrid-domain macro structure that enables computation of both the exponent and mantissa in an FP-MAC within the same IMC macro. A high etaE and accuracy are achieved by exploiting advantages of computing in the time, digital, and analog-voltage domain by identifying the proper functional blocks for the FP-MAC [2,4-5]. (2) Time-domain based textPDE generation, a textmaximum-textPDE(textPDE-textMAX) finder (TD-MPEF), and a textPDE-textPDE-textMAX generator (textTD-textPDE-textDG) to achieve a high etaE for all exponent computation. (3) textPDE -based input-mantissa alignment (PEB-IMA) scheme to enable accumulation for textPDM in the same column. (4) A place-value dependent digital/analog-hybrid computing scheme for mantissa computation with a high inference accuracy and etaE. A 22-nm 832-kb FP SRAM-IMC macro is fabricated using foundry-provided compact-6T SRAM cells. The FP SRAM-IMC support FP-MACs with 128-accumulators (ACCU) for BF16 inputs (IN) and weights (W) with FP32 outputs (OUT) and achieves the highest reported FP-MAC etaE, 70.2TFLOPS/W.