Abstract
A Study on Cluster Analysis: Improving Performance on Stability Measurement Several techniques and algorithms have been designed for clustering. Given that the results of these techniques come from describing the hidden structures of unlabeled data, the validation of the output and method thus becomes a hard task. This raises several problems, some of which have still not been solved. As examples, consider the following: Which clustering method should we use? Which distance or similarity method should we choose to compare data? How should we assess the significance of the cluster? How can we determine the number of clusters? Several proposals to address these problems have been proposed. In this research, we focus on the stability score, which is measured by calculating the expected distance of a perturbed version of the original data. This is commonly used to know the number of clusters in a dataset, however this method becomes difficult to run when the data size increases. Calculating on large data also affects the performance of the measurements. Depending on the nature of the data can take several weeks to get a precise result. To address this issue we present a variation of the stability algorithm named “Critical Stability”, that focuses on the main perturbations that can destroy patterns, replacing the randomly generated ones thus improving the performance of the measurement. To validate this new algorithm, we tested real and artificial datasets that have known patterns and compared time and results for both the stability algorithm and the critical stability.