Choose an AI Model by Risk, Not Hype
A decision method for matching AI models to tasks based on what a wrong answer costs you — not on leaderboards or launch-day noise.
Every model release arrives with a chorus telling you everything changed. Occasionally the chorus is right. But your business does not run on benchmarks — it runs on tasks, and each task has its own answer to the only question that matters here: what does a wrong answer cost, and who catches it?
The method
Step one — write the failure, not the feature (20 minutes). For each candidate task, describe the worst plausible output and what happens next. “The draft email is clumsy and I rewrite it” is a different universe from “the quote is wrong and a customer accepts it.”
Step two — place each task on two axes (15 minutes). Cost of error (minutes of rework → lost customer → legal exposure) against visibility of error (I will obviously notice → someone might notice → nobody would notice until it compounds). The dangerous quadrant is never the expensive-and-obvious one; it is the cheap-looking-and-invisible one.
Step three — let the quadrant pick the class of model (20 minutes). Low-cost, visible errors tolerate fast, cheap models. High-cost or invisible errors demand the most capable model you can access — and usually a human checkpoint regardless of model. The specific vendor matters less than refusing to let one default model quietly serve every quadrant.
Step four — schedule the re-check (10 minutes). Model capabilities move. Your risk map mostly doesn’t. Put a quarterly thirty-minute review on the calendar and ignore launch-day noise between reviews.
The claim underneath this guide
Businesses don’t get hurt by choosing the second-best model. They get hurt by making one undifferentiated choice, letting it spread across tasks with wildly different failure costs, and never writing down what a wrong answer costs. Fix the writing-down part and the model choice becomes almost boring.