Organizations deploying AI agents may be in for a nasty surprise when it comes to the cost of tuning their performance. According to some surveys, nearly 80% of enterprises have deployed AI agents, but most don’t understand the cost of training them and evaluating their outputs, which can result in costs far exceeding expectations, experts say. Many organizations are still experimenting to find the best ways to catch agent problems before they cause chaos after deployment, says Lior Gavish, cofounder and CTO at AI observability vendor Monte Carlo. Because many organizations use a second large language model to vet the outputs of an LLM-powered agent, agent testing can be many times more expensive than testing traditional software, he says. Moreover, this method, called LLM as a judge, can be more expensive than running the agent itself, as the cost of running an LLM over an extended period can add up quickly.

Full feature : Organizations embracing agents often fail to estimate the costs of testing their output, with the non-deterministic nature of results often leading to complex and expensive evals.