A Review of Reduced Kernel Trick in Machine Learning

We give a comprehensive introduction to the reduced support vector machine, its extensions and applications. We describe original RSVM algorithm and the statistical theory behind it. Three schemes for selecting the representative reduced set are introduced. These schemes lead to a smaller reduced set than the random sampling scheme without sacrificing prediction accuracy. Although smaller reduced set will have faster support vector machine training, one has to pay extra CPU time in learning the reduced set selection. In addition to classification, applications of reduced kernel trick to regression and dimension reduction are also included in this survey paper. We finally embed the RSVMs in the MapReduce framework for extremely large scale datasets. Some preliminary numerical studies show that RSVMs in MapReduce framework has a good potential for solving large scale nonlinear support vector machines. We believe that the reduced kernel trick will be an important technique in the Big Data era.