A model's cost per call reveals little about the economics of a real process. The useful unit is an accepted, completed task, after adding retries, review and cases handed back to a person.
Define what completed means
An output does not count as a success simply because it was generated. It must meet the quality criteria and reach the next business step.
Add up the entire chain
Include model calls, retrieval, storage, orchestration, monitoring and tools. Small costs across multiple steps can exceed the main call.
Measure retries and exceptions
Record how many executions need another attempt and how many end in manual work. An average hides expensive, difficult categories.
Put a value on human time
Review, correction and incident handling are part of the cost. Use realistic times for each type of work.
Compare against the previous process
Savings should be assessed against a baseline of quality, speed and risk. A cheaper output that requires more remediation is not an improvement.
This framework is an original editorial methodology developed by DigitalNow.