A cheap model can be the expensive route through a workflow. A usage charge or infrastructure bill is only the visible line. The P&L carries everything required to turn an attempt into accepted work: tools, retries, review, rework, and recovery when the output fails.
That changes the unit of comparison. A benchmark can inform a technical assessment, but it cannot tell a leader what a completed, accepted task costs inside the operating model. Model mix should be decided task by task, against the full path from intake to accepted outcome.
Price the accepted task, not the attempt
Consider a task class that prepares a case and recommends the next action. One model and operating route may have a lower direct price yet require more retries, a heavier checking layer, or more reviewer attention. Another may cost more per attempt but meet the task’s reliability threshold more consistently under the intended controls. Either route can be economical. The answer has to be observed on the task, not inferred from a price sheet or leaderboard.
The full cost can include model usage or computing infrastructure, supporting tools, orchestration, fallback routes, monitoring, human review, exception handling, rework, and recovery. Model and deployment choices may move several of those lines. The relevant denominator is not a token or model call. It is an accepted unit of work.
Control choices shape the same economics. Model rights and deployment route are separate decisions. An open-weight model may be self-hosted or accessed through a managed route; self-hosting moves more infrastructure, monitoring, and lifecycle work inside the enterprise. A managed frontier route carries a different responsibility and dependence profile. A hybrid portfolio can send different tasks down different paths, but its control layer can erase the flexibility if it becomes too costly or fragile to operate.
As the consequence of failure rises, recovery deserves more weight. A reversible internal draft and a customer-facing decision do not need the same route, review, or fallback. Greater task reliability may reduce retries or review. It may also bring commercial or operating dependence that leaders should price rather than discover during a failure or migration.
Write the policy at task level
Approve a task-level model policy, not one enterprise winner. For each task class, record:
- the stakes of a wrong, delayed, or unusable outcome;
- the accepted-outcome threshold and the data and deployment constraints;
- the permitted model and operating routes, including required review;
- the fallback and recovery path, with named owners;
- the full cost components and practical switching requirements; and
- the date for routine review, with earlier retesting after a relevant capability, commercial, or control change.
A portfolio does not require constant switching. It preserves deliberate choice. Leaders should know where changing routes is practical, where integration creates dependence, and what a transition would require. A policy without a review date eventually becomes an untested assumption.
Keep the value record honest as the portfolio changes. Cash is a documented and attributable change in money paid or received. Released reviewer or engineering time is capacity and stays in operational units; salary multiplication does not turn it into savings. A durable change in control, resilience, or switching ability may be structural value, reported on its own terms. Potential quality, throughput, or revenue effects remain modeled upside until observed, recognised where relevant, and attributed. These categories belong beside one another. They do not form a defensible total.
What would count as proof?
Proof is a documented task portfolio, not a model ranking. Define the accepted outcome and baseline for each task class. State every cost component included in the comparison, then observe first-pass acceptance, retries, review effort, rework, exceptions, and recovery under the intended controls for a defined period. Record the decision and control owners, the permitted route, switching assumptions, and the next review date.
Report cash, capacity, structural value, and modeled upside separately. A cash claim needs an observed, attributable movement in money paid or received; an operating estimate or changed budget intention is not enough. If the policy cannot show why a route was chosen, what it costs to produce accepted work, and what would trigger reconsideration, it remains a preference rather than a P&L decision.
What remains unclaimed?
No model type is universally cheaper, safer, or more capable for every task. A benchmark position, vendor claim, or unit price does not establish a business outcome. Lower review effort does not establish cash, and switching freedom is not a realised saving. No revenue effect is claimed before it is recognised and attributed.
Choose the portfolio on the economics of accepted work, then keep the decision open to evidence.