A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost

Identity resolution evaluations become more accurate and affordable, enabling faster iteration and easier maintenance of IDR systems.

Hightouch, a customer data platform, has tested a new method for evaluating identity resolution (IDR) accuracy. They found that a hybrid model, combining the faster, cheaper Jev model with the more powerful Claude Opus, can achieve near-Opus level accuracy at over four times the cost savings. The benchmark involved creating 1,000 simulated identities with various data errors and over-merges. While Jev and GPT-6 Luna, both fast decision models, performed adequately with 88.8-97.3% accuracy, their error analysis revealed inconsistencies. Jev's mistakes were mostly low-confidence, whereas Luna's were high-confidence. This calibration allowed Jev, when paired with Opus, to effectively leverage Opus's capabilities for more accurate IDR evaluation at a significantly reduced price point compared to using Opus alone.

The findings suggest a paradigm shift in how identity resolution systems can be validated. Instead of relying solely on expensive frontier LLMs like Claude Opus or time-consuming manual sampling, companies can now opt for a hybrid approach. This combination leverages the cost-efficiency and speed of decision models like Jev, while still achieving high accuracy by intelligently incorporating the strengths of more powerful LLMs.

This development directly impacts businesses that depend on accurate customer data for personalization, marketing, and analytics. By reducing the cost and increasing the speed of IDR evaluation, companies can iterate more quickly on their IDR strategies, leading to more precise customer profiles and improved campaign effectiveness without incurring prohibitive expenses.