Logo image
返回
Comparative Study of Edge Deployment of LLM on Arm and RISC-V SBC Clusters
會議論文

Comparative Study of Edge Deployment of LLM on Arm and RISC-V SBC Clusters

Tay-Jyi Lin, Bo-Chen Liao, Chih-Yen Liao, Kai-Siang Shih, Chingwei Yeh, Peng-Sheng Chen, Shih-Chieh Chang, Tien-Fu Chen 和 Jinn-Shyan Wang
International SoC Design Conference, 頁碼.1-2
IEEE
International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers
2025
DOI: https://doi.org/10.1109/ISOCC66390.2025.11329552

摘要

Computers computing clusters distributed Llama (dLlama) Hardware large language model (LLM) Large language models Neon Quantization (signal) Single instruction multiple data single-board computers (SBC) Software design Visualization Computer Architecture Software
Connecting multiple low-cost single-board computers (SBCs) through network to form a computing cluster is a feasible solution for deploying large language models (LLM) at the edge. This work compares the deployment of Llama3 8B on computing clusters composed of leading-edge Arm and RISC-V SBCs. The performance numbers (as tokens/sec) are given and analyzed in correspondence to the hardware architecture & software design implementations.

相關連結

指標

1 檢視次數

詳細資料

Logo image