Start with costs by feature
Break down AI spending by feature and check whether it leads to results. ABTO records the model, token usage, cost, and response time for each call.
Choose a cheaper model when outcomes hold up
Give some users a cheaper model and check that outcomes such as purchases and conversions hold up before switching. Once it meets your criteria, gradually increase its share and reduce the existing variant to 0%.
If the existing model performs better, you have numbers to judge whether the difference is worth its cost.
Nonsuri reduced reasoning effort and shortened explanations, cutting model costs by 41% while increasing the retake rate by 18%.
Prompt tokens cost money too
Removing unnecessary parts of a prompt can lower cost per call without changing the model. After shortening it, check that your success metrics remain steady.
Find the cause of a sudden cost increase
When costs spike, narrow the investigation from a feature to a variant to an individual call.