Archive / Find out what your AI actually costs per finished task

Find out what your AI actually costs per finished task

Compare models on usable work, including retries and time spent cleaning up.

For: Solo founders, Small business owners

Topics: Accounting, Agent management

A model can be cheaper per call and cost more per accepted task. Pick one repeatable job and grade identical cases before looking at the bill.

1. Define done

Prompt

For [task] and five nonsensitive examples, write four observable pass criteria, including one disqualifier. Make a sheet for case, model, attempts, billed usage, review minutes and accepted yes/no. Do not run the experiment.

2. Score without the labels

Prompt

Grade [anonymized outputs] against [rubric]. Give pass/fail and exact evidence for each. Explain any criterion you cannot apply. Do not guess which model wrote it.

3. Count the whole cost

Prompt

From [results and hourly rate], divide billed usage plus human review and editing cost by accepted tasks. Include failed attempts in cost. Report counts and elapsed time separately. If none passed, show the ratio as undefined. Label any subscription allocation separately.

Try it on one example

Example: $1.20 in usage plus 12 minutes at $30 per hour is $7.20. Four accepted tasks would cost $1.80 each. One small sample is a reason to test again, not a reason to switch everything.

Inspired by this Hermes cost discussion: https://www.reddit.com/r/hermesagent/s/qS4uAcTteJ. Figures are illustrative.

Use it for: Compare models on usable work, including retries and time spent cleaning up.

Back to the archive