Abstract
Computing-in-memory (CIM) improves energy efficiency by enabling parallel multiply-and-accumulate (MAC) operations and reducing memory accesses [1 -4]. However, today's typical neural networks (NNs) usually exceed on-chip memory capacity. Thus, a CIM-based processor may encounter a memory bottleneck [5]. Tensor-train (TT) is a tensor decomposition method, which decomposes a d-dimensional tensor to d 4D tensor-cores left(operatorname{TCs}: G_{k}left[r_{k-1}, n_{k}, m_{k}, r_{k}ight], k=1, ldots, dight) [6]. G_{k} can be viewed as a 2D n_{k} imes m_{k} array, where each element is an r_{k-1} imes r_{k} matrix. The TCs require Sigma_{k in[1, d]} r_{k-1} n_{k} m_{k} r_{k} parameters to represent the original tensor, which has Pi_{mathrm{k} in[1, mathrm{d}]} mathrm{n}_{mathrm{k}} mathrm{m}_{mathrm{k}} parameters. Since rk is typically small, kernels and weight matrices of convolutional, fully-connected and recurrent layers can be compressed significantly by using TT decomposition, thereby enabling storage of an entire NN in a CIM-based processor.