Self-hosted AI agents: calculate cost per completed task
Build a practical cost worksheet for self-hosted agents that includes models, tools, retries, idle infrastructure and human review.
Owning a server does not tell you what an agent costs to operate. The application may call remote models, launch browsers, retain documents and require someone to correct incomplete work. This source-reviewed guide provides our own costing worksheet for a fictional document-processing service. It uses no vendor price assumptions and reports no measured savings.
Define the result you will count
For the sample service, a completed task is an accepted structured record extracted from a document, with a source reference and the required fields checked. A model response, an uploaded file and a started job are different events. Count attempts and accepted results separately so failed runs do not make the service appear cheaper.
Our suggested accounting unit is cost per accepted task: the allocated cost of operating the workflow over a period divided by its accepted tasks in that same period. Keep uncertain estimates visible. A period with no accepted tasks has no meaningful finite cost per accepted task; report the spend and missing outcome instead.
Map costs to the steps that create them
Separate costs that remain while the system is idle from costs that grow with activity. Also retain a view of human time: a cheap automated run followed by extensive correction may be a poor service. Use your actual invoices, usage records and time estimates rather than a generic price attached to a framework.
| Cost group | What to record |
|---|---|
| Infrastructure | Compute, storage, backups, network and idle capacity |
| Model use | Input, output, cache and other billed usage categories |
| External tools | OCR, search, browser sessions and paid integrations |
| Repeated work | Retries, failed attempts and duplicate processing |
| Human work | Review, correction, support and maintenance time |
Distinguish hosting the agent from serving its model
An application that coordinates remote model calls has different resource needs from a server that runs model inference. Do not assume hardware suitable for the first is suitable for the second. Ollama's official FAQ explains that available memory, parallel requests and context length affect its model-serving behavior and queueing.
For the document service, compare candidate setups using the same document set and acceptance checks. Record queue delay, total completion time, memory pressure and accepted outputs under the concurrency you expect. Include the idle period between batches. This is a proposed measurement procedure; the directory has not run it on your hardware.
Connect usage records to a job identifier
Give every job a stable identifier and attach its model calls, tool calls, retries and final outcome to it. Preserve provider request identifiers when available. Mark unknown usage as missing data instead of writing a zero. Reconcile a sample of your ledger against the provider's billing period and usage categories.
LiteLLM documents spend tracking and notes that pricing data and provider metadata affect its calculations. If you use a gateway, treat its reported spend as an accounting input that needs reconciliation, particularly for custom model names or negotiated rates. It cannot automatically account for your server bill or the person correcting a document.
Control expensive paths before increasing volume
Our suggested job policy sets limits on elapsed time, model calls, tool attempts and total work admitted into the queue. Choose those limits from your own pilot. When a limit is reached, return a visible pending or failed state with a reason, rather than marking a partial record as complete.
Investigate repeated work before buying more capacity. An extraction retried because of a downstream timeout may need better job-state handling, not a faster model. A document that always needs correction may belong in a manual queue. Show both the automated completion rate and human effort when comparing the next change.
Make the next spending decision from observed demand
Keep a small weekly worksheet: accepted tasks, incomplete tasks, observed spend, allocated infrastructure cost, review time and unresolved metering gaps. Calculate a range when assumptions remain uncertain. Compare the current setup with a proposed change using the same accounting boundary.
Expand capacity when measured queues, reliability needs or accepted demand justify it. Retain a simpler configuration if it completes the paid work within your service promise. The useful output is a traceable operating estimate that explains where money and time go, and which specific change would improve the cost of a result your customer can actually use.
Sources & further reading
Official sources support the specific product and protocol facts in this guide. The selection process and examples are editorial recommendations, not a claim that we tested the listed tools.
