Abstract
MapReduce that attends data locality and fault tolerance is one of the most successful programming paradigms for data intensive and high-throughput applications. While many researches focus on the system performance of MapReduce, the application level performance optimization techniques are seldom discussed. In this paper, we present strategies to optimize the performance of Stochastic Singular Value Decomposition Collaborative Filtering (SSVD-CF) algorithm on MapReduce architectures. We applied the performance optimization techniques from scientific computing, including BLAS 3 matrix operations, Tall and Skinny QR (TSQR) decomposition, tiled algorithm, and distributed cache, to the SSVD-CF computation. Experiments showed although the MapReduce architecture is different from traditional high performance computing environments, those techniques can be still very effective. Near eight times speedup can be achieved for the most time consuming job, and more than five times speedup can be obtained for the entire program for large datasets.