E-means: 10x faster k-means at equal loss

Clustering operations on large datasets will now complete significantly faster and with better results.

Researchers at Meta FAIR have developed and open-sourced a new k-means algorithm called e-means. This algorithm reportedly achieves approximately a 10x speedup in both training epochs and wall-clock time compared to traditional k-means (Lloyd's algorithm) while also reducing the mean squared error (MSE) loss. The performance gains are attributed to using mini-batch updates with exponential moving averages, specialized GPU kernels for parallelization, automatic hyperparameter tuning based on scaling laws, improved initialization, and a dead-cluster revival strategy. e-means reaches a lower test loss than Lloyd's algorithm within 1.2 to 3 epochs. Benchmarks show wall-clock reductions of 10-45x on GPUs and 8-15x on CPUs against state-of-the-art implementations. The algorithm is available in the FAISS library.

e-means represents a substantial evolution in clustering algorithms, moving beyond the widely adopted but computationally intensive full-batch Lloyd's algorithm. By leveraging techniques common in modern deep learning, such as mini-batch updates and exponential moving averages, it offers a novel approach to the static k-means problem. The automatic hyperparameter tuning derived from scaling laws eliminates a key barrier to effective k-means implementation, making advanced performance more accessible without extensive tuning.

This advancement is particularly impactful for machine learning pipelines that rely heavily on k-means for tasks like dimensionality reduction, feature engineering, and data segmentation. The combination of speed and accuracy suggests that e-means could become the new default for these applications, potentially influencing the design of downstream models and the efficiency of training large-scale AI systems. Organizations with significant clustering needs may see direct improvements in processing times and model quality.