Abstract
Convolutional neural networks are one of the most widely used deep architectures in machine learning. While they achieve superior performance of recognition especially for images, the training remains a computational challenge which prevents them from practical uses. Even for GPUs possessing great computational power might take days to produce results. In this thesis, we propose a method based on separable filters to reduce the train- ing time. First, by using SVDs, the 2D filters in the convolutional neural networks are approximated by the product of two 1D filters. Second, two 1D convolutions are per- formed with the previous 1D filters. In our GPU implementation, a batched SVDs that can compute multiple small matrices simultaneously, and 3 methods which use different memory spaces according to the filter size are presented. Our experiment results shown that 1.38x ∼ 2.66x speedup was achieved in the for- ward and the backward pass. The overall training time could be reduced by 13% with 1% drop in the recognition accuracy.