Amazon saves millions by replacing Sonnet 4.6 with Strands Decider
First reported by Strandsagents ·
Using a decision model like Strands Decider instead of a large language model like Sonnet 4.6 can cut inference costs by over 98%.
Amazon engineers replaced Claude Sonnet 4.6 with Strands Decider, a new decision model, for labeling tickets in a large internal system. This system categorizes various ticket types, including customer escalations, capacity requests, and security findings, into 20 predefined categories. Sonnet 4.6 was previously used for this task, balancing intelligence with cost. The Strands Decider model, specifically the E4B Gemma 4 build, demonstrated comparable performance to Sonnet 4.6 but at a significantly lower cost, estimated at 98% less. This transition is projected to save millions of dollars annually in token costs. The implementation involved an A/B testing approach using AWS AppConfig to gradually switch from Sonnet 4.6 to Strands Decider, ensuring performance was validated before full adoption.
This shift indicates a growing trend in leveraging specialized decision models for specific tasks within large organizations, moving away from general-purpose LLMs where not strictly necessary. Strands Decider, by focusing on probabilistic output for predefined categories rather than generative text, offers a more efficient and cost-effective solution for classification and labeling at scale. This approach not only slashes operational expenses but also streamlines the process, as seen with the 78% reduction in average call time.
The success of this implementation within Amazon's vast internal ticketing system suggests that similar cost-saving opportunities exist for other companies dealing with high-volume, structured data classification tasks. Organizations can explore employing decision models or similar specialized AI components to handle specific functions, reserving powerful but expensive LLMs for more complex, generative, or nuanced applications. The validation process, involving parallel runs and confidence threshold tuning, highlights a best practice for migrating critical AI workloads.
AI-written summary. May contain errors.