Logo image
Training and Optimization of Spiking Transformer Models Under ReRAM Non-Ideal Effects
會議論文

Training and Optimization of Spiking Transformer Models Under ReRAM Non-Ideal Effects

Chih-Yu Hsu, Tsu-Hsiang Chen, Chih-Tsun HuangJing-Jia Liou
2025 International VLSI Symposium on Technology, Systems and Applications (VLSI TSA), 頁碼.1-4
IEEE
2025 International VLSI Symposium on Technology, Systems and Applications (VLSI TSA) (Hsinchu, Taiwan, 21/04/2025–24/04/2025)
21/04/2025

摘要

Accuracy Computational modeling Hardware Memristor Array Neuromorphic engineering ReRAM SNN Spiking Neural Network Spiking neural networks Training Transformer Models Transformers Tuning Variability Very large scale integration Computer Architecture
Memristor-based spiking neural networks (SNNs) offer energy efficiency and area compactness, yet most existing studies focus on CNN-based models with idealized hardware characteristics. This paper introduces a five-stage training framework for transformer-based models on ReRAM architectures, addressing critical non-ideal effects, including nonlinearity and variability. The proposed approach incorporates quantization, resistance modeling, and post-training variability tuning to enhance model robustness and accuracy. Experimental results demonstrate accuracy improvements of 1. 4 5 %, 9. 1 8 % , and 19.22 % on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets, respectively, under non-ideal ReRAM effects. These results validate the effectiveness of the proposed method in advancing transformer-based SNNs for efficient neuromorphic computing.

相關連結

指標

1 檢視次數

詳細資料

Logo image