Logo image
G-Storm: A GPU-Aware Storm Scheduler
Conference paper

G-Storm: A GPU-Aware Storm Scheduler

Yi-Ren Chen and Che-Rung Lee
Proceedings - 2016 IEEE 14th International Conference on Dependable, Autonomic and Secure Computing, DASC 2016, 2016 IEEE 14th International Conference on Pervasive Intelligence and Computing, PICom 2016, 2016 IEEE 2nd International Conference on Big Data Intelligence and Computing, DataCom 2016 and 2016 IEEE Cyber Science and Technology Congress, CyberSciTech 2016, DASC-PICom-DataCom-CyberSciTech 2016, pp.738-745
10/2016

Abstract

Apache Storm;GPU;heterogeneous clusters;scheduling;stream processing;G-Storm;GPU-aware;Storm Scheduler Computer Vision and Pattern Recognition,Information Systems,Computer Science (miscellaneous),Artificial Intelligence,Computer Networks and Communications
Many systems for big data processing have been developed to analyze and process huge amount of data. One of them is Storm, whose target is stream data processing. The default scheduler in Storm uses the round robin method to assign tasks, which is not optimal for heterogeneous computing environments. In this paper, we proposed and implemented a new Storm scheduling algorithm, named G-Storm, which takes the GPU capacity into consideration and can make better use of GPU to speed up the overall performance. The experimental results show that G-Storm can achieve 1.65x to 2.04x performance improvement on lightly weight and heavily loading of topology, comparing to the results of the original Storm scheduler.

Metrics

1 Record Views

Details

Logo image