Abstract
This research focuses on improving the process of feature selection before we use data mining techniques to analyze database. The studies in the past always used the kind of algorithms to execute feature selection in data mining process, but the kind of algorithms sometimes may delete unnecessary features or attributes incompletely. In this situation, these unnecessary features or attributes may reduce the speed of the algorithm and affect data mining result in incorrect prediction or decision rule. To improve this problem, we propose a new process for feature selection with a statistical method. The goal of our method is to completely reduce the unnecessary features or attributes totally before processing data mining and to kept independence and representation of the original data. In processing feature selection or classification, this research will take the principal component analysis to reduce the features and attributes in benchmark databases. The most important target of this analysis is to set the reduced data to keep the independence、representation and simplicity. After using principal component analysis to reduce data, we will use the two-step cluster method, hierarchical logistic regression and dummy data to process and improve data mining expect for increasing the accuracy for the result and reducing the experiment time. This study uses the UCI databases are the experiment examples and benchmark questions. By combining statistical methods, we can set up the new process for data mining and data classification. We hope that this study can offer new ideas in data mining combining and feature selection with statistical methods. Key words: feature selection, classification, principal component analysis, two-step cluster method, hierarchical logistic regression, dummy data