Adversarial examples for fast hash functions

The speed of hashing data has just become demonstrably less secure against determined adversaries.

A new analysis by Thomas Dybdahl Ahle reveals that many popular fast hash functions, despite passing statistical tests, exhibit significant weaknesses against adversarial inputs. These weaknesses, which allow attackers to force collisions much more frequently than expected, were uncovered using AI-powered analysis on a broad selection of hashes tested within the SMhasher project. Ahle's work specifically targets the trade-off between speed and collision resistance, showing that high-speed hashes often sacrifice robustness. While many hashes claim to be b-bit universal (meaning collisions occur with probability at most L * 2^-b for inputs of length L), Ahle found specific inputs that cause collisions to happen at rates far exceeding these theoretical bounds, sometimes by over 20 bits. The analysis examined numerous hashes including CityHash64, FarmHash64, xxHash, MurmurHash3, and others, identifying vulnerabilities in many that were previously unproven or assumed secure.

The analysis highlights a critical shift in the threat landscape for software security, where previously obscure vulnerabilities in hashing algorithms are now easily discoverable by AI. This implies that systems relying on these hashes for security, data integrity, or performance optimizations like hash tables may be unknowingly vulnerable to DoS attacks or data manipulation. The findings suggest that the common practice of trading collision resistance for speed in many popular hash functions is no longer a safe assumption when security is a concern.

This research introduces a new standard for evaluating hash functions, emphasizing the need for provable correctness beyond basic statistical tests. It suggests that developers and security professionals should prioritize hashes with verified proofs of collision resistance, particularly in contexts where adversarial input is a possibility. The study also points to the potential for developing faster, yet provably secure, hashing algorithms, as demonstrated by the author's own ChainHash, which balances high throughput with strong collision guarantees.