A model improvement can invalidate an AI value case before it supports a new one. The old claim was measured under a particular task, review path, failure pattern and cost base. If those assumptions move, the number may remain an accurate historical record, but it is no longer a safe basis for today’s funding or scale decision.
The leadership response is to freeze the prior claim, redesign the workflow and test the new arrangement. A material change to the model or its harness, meaning the routing, tools, permissions and checks around it, should trigger a bounded assumption review with one owner, a review date and an expiry.
The claim belongs to the workflow
Consider contract review as an illustration, not a customer result. The original case might cover a defined class of contracts, a draft prepared for a specialist, human checking of every output and a unit cost for accepted work. Its value claim belongs to that whole arrangement, not to the model in isolation.
An improvement may change which clauses the workflow can address, the kinds or severity of failures, or the amount of exception work. That can justify testing a different task design. It does not, by itself, justify less human review. Any control change needs operating evidence from the relevant case mix, measured against an accepted-completion standard and the organisation’s control threshold.
End-to-end cost must also be reopened. Inputs, outputs, retries, tool calls, latency, monitoring, human checks and exception handling all shape the cost of accepted work. A changed component price is an input, not an economic outcome. A favourable technical benchmark can identify a question to test; it cannot establish reliability, control performance or cost in the redesigned workflow.
If the task changes, the comparison must say so. Evidence about a broader contract-review process may support a new baseline. It does not show that the original process simply became cheaper.
The resulting value claims belong in separate records. Measured time released is capacity; productive redeployment requires further evidence. Structural value requires evidence that a useful operating or control change has been adopted and persists. Realised cash is an observed, attributable net cash movement against a credible baseline. Revenue or another forecast effect remains modeled upside until recognised and attributed. Released hours do not become cash through a salary calculation, and the four records should never be summed.
The decision to make now
Require an assumption review when a model or harness change could materially affect task scope, accepted completion, reliability, review demand, cost, latency, tool use or controls. This need not become a full reapproval exercise. It is a focused decision about which assumptions still hold and what the current evidence can support.
Name one owner. Preserve the prior claim with its dates, task definition and operating conditions. Define the proposed workflow and control path, then set the evidence period, next review date and validity boundary for any revised claim. At expiry, a claim without current evidence returns to hypothesis status. Historical evidence can inform the next test, but it cannot silently extend the old conclusion.
What would count as proof?
- A dated record of the material change, the assumptions reopened and the accountable owner.
- Before-and-after definitions of the task, workflow, measured unit, accepted completion and control path.
- Current operating observations against the new baseline, including reliability, review demand, failures, exceptions, timing and tool use where relevant.
- Updated end-to-end cost inputs and separate evidence for capacity, structural value, realised cash and modeled upside.
- A stated evidence period, attribution boundary, review date, expiry rule and limits on what may be inferred.
Proof must describe the redesigned work as it operates, not merely the technical change that prompted the test.
What remains unclaimed?
A model improvement alone does not establish lower end-to-end cost, safe reduction in review, released capacity, structural value, realised cash or a business outcome. Each requires evidence from the current workflow and the appropriate value record. Modeled upside remains a forecast. A new task baseline does not validate the old task claim, and no cumulative value total should cross a material workflow change as if the underlying work had stayed constant.
Model change should reopen the business case
An improvement is an assumption change, not a value result. Date the claim, name its owner and make its expiry explicit.