VISIT: Virtual-Targeted Sequential Training with Hierarchical Federated Learning on Non-IID Data

Recently, Federated Learning (FL) has realized Artificial Intelligence of Things (AIoT) applications to train a shared model while preserving user privacy collectively. However, the legacy FL framework performance is fundamentally threatened by scale limitation, non-independent and identically distributed (non-IID) data, and communication costs. Therefore, we propose VIrtual-targeted SequentIal Training with Hierarchical Federated Learning (VISIT), a novel framework to systematically distribute clients to suitable clusters for balancing data distributions among all FL subgroups. To the best of our knowledge, this work is the first attempt to introduce a Virtual Target concept along with a key metric, Virtual Target Similarity (VTS), to quantify the data harmonization in the whole HFL system. Based on our insightful Client Set arranging strategy, VISIT can wisely select each FL subgroup member to optimize diversity within each Client Set and similarity across different clusters while preserving user privacy. Numerical results demonstrate that VISIT improves accuracy by 41% and reduces total communication rounds by 82% compared to other state-of-the-art baselines with non-IID data on EMNIST and CIFAR-10 datasets.