In the race to build better agent loops it’s easy to ignore token waste as a problem. Getting an idea into a working state is so cheap and easy that the pressure to spend and compete far outweighs concerns about waste.
Any project that gets past that initial implementation will need a reliable way of managing the way things get done, and that means knowing how much it costs. When you start counting your tokens against turns, tool calls, errors, failures and so on the waste can suddenly look pretty alarming.
While experimenting with using GitHub as an agent orchestration harness (1, 2, 3) I found that the blended cost per issue rate was reaching $3.43. Excluding the stalled issues, the rate is only $0.61 per issue. That seems much more reasonable, but is it realistic at a larger scale?
Digging deeper, I found that the cheaper features change 1 file within 10+ turns, things like documentation updates. Some were operational runs that handled some issue or PR management work. In most cases, they were unnecessary or low impact.
The work where an issue adds or updates 2-3 files within 40-50 turns costs about $0.99 on average. Those issues were the ones that introduced capabilities like rate limit handling, generating scripts, writing tests, etc.
The problematic issues that killed my cost per issue rate were things that introduced change-requests during review or that hit an error and looped through changes to address it or that inflated the context early in the process. 43% of my total spend over a week went to these bad issues, so I’ve been implementing max_turns, max_attempts, token_count_warning and things like that.
The magnitude of these cost per issue rates starts to become more meaningful when talking about more serious product development. Processing 10k issues at $3.50 per issue costing you $35,000 while knowing that you have wasted $25,000 of that pot on failures and errors would be very disappointing. Larger organizations running multiple projects with multiple teams would have to reconsider what they’re doing if those ratios remained that poor at scale.
I only know all this because I’m instrumenting event traces that are logging token counts, turns, and outcomes, and running some lightweight analysis over the data. Otherwise I would be looking at my service provider dashboards and thinking, “Hey, the costs are looking pretty manageable.” The truth is that those costs are masking a lot of inefficiency.
Provider dashboards show you spend, not waste, and those are very different things.
The context that makes cost data meaningful lives in the orchestration layer, not the API. If you’re not logging it there, you’re not managing your agent system. You’re just paying for it.
Leave a Reply