Don't pay for AI support failures, says Gradient Labs CEO

Paying for successful problem resolution is a better business model, argues Dimitri Masin

interview Dimitri Masin, CEO of Gradient Labs, argues that companies using AI agents for customer support should only pay when the bot does its job.

"If you look at Salesforce, they price the automation per conversation," he told The Register in a phone interview. "So essentially, if you have a conversation with AI, no matter what it leads to, you pay $2."

According to Salesforce's own researchers, leading LLM-based agents tested on the CRMArena-Pro benchmark successfully complete single-turn (prompt and reply) tasks about 58 percent of the time and only about 35 percent of the time for multi-turn (back-and-forth conversation) requests.

That's both bad for customers and bad for the progress of AI agents overall. Paying regardless of results, said Masin, "doesn't create any incentive for Salesforce to actually make their agent better."

Salesforce did not respond to a request for comment. But the stats it touts in its marketing copy are, unsurprisingly, a lot better.

Between October 2024 and December 2024, Salesforce claims its Agentforce assistant on its Salesforce Help page "is now solving 83 percent of customer queries without a human, has halved the number of issues that require human intervention, and has nearly doubled its average number of weekly conversations."

Masin argues that billing per resolution is better for calculating return on investment (ROI).

Among enterprises exploring AI, ROI is a persistent worry. According to a survey [PDF] published by application security firm F5 last year, "when asked their biggest concerns about AI model development, training, and inference moving forward, nearly two-thirds of respondents (62 percent) in our annual survey told us they were worried about compute costs."

"If we don't resolve the issue and you still need to get your human team involved, then you don't need to pay us," Masin said. "And the reason why this is so powerful is because it creates a direct link to ROI, because every company knows what a contract costs them on average."

That allows a company to make an informed decision about whether paying for AI agent-based support is worth it – a calculation businesses have had trouble making. If the average cost for a human agent to handle a problem is $5 or $10, Masin explained, and a vendor presents the biz with the opportunity to accomplish the same result for significantly less, the ROI becomes easier to assess.

Unfortunately, it's not that simple. Masin explained that the resolution time for customer support engagements varies.

"In reality, you have a distribution of different conversations, some of them are super easy, where a human could resolve them in one minute," he explained. "And some of them are super hard, where humans on average spend like 30, 40 minutes actually resolving them. ​​And obviously, if you just charge a flat $1 per resolution, you don't account for that."

Gradient Labs has tried to capture that variation in its pricing model through tiered pricing. "So essentially, if you get to 50 percent resolution, then you offer one price," said Masin. "If you get to 70 percent resolution, you offer a higher price. Because the higher you get, the more complex queries you automate, the more value you give per resolution to the company."

Masin said his goal is to charge roughly 30 percent of what it would cost to have a human do the work, so the customer can save about 70 percent.

"So in theory, that is a no-brainer value proposition," he said. "But where it becomes tricky is if you automate only the easiest 50 percent [of support tickets] and they have saved only 20 percent of the time, then it's not quite the same equation. So they're maybe paying you 60 percent, but saving themselves 40 percent or paying you 70 percent and saving themselves 30 percent because of how it's skewed in terms of the easiest tickets versus the hardest tickets."

Masin claims that with no customization, Gradient Labs' agent resolves between 40 and 60 percent of queries, which typically translates to a cost savings of around 20 percent. "This is before you integrate any data points or like any actions or tools that AI agents could make on behalf of the customer," he said.

With more data integration, you can get to 80 or 90 percent, he said, noting that the process often takes three to five months to complete. A company case study with Sling Money claims a 78 percent resolution rate after optimization.

What's more, Masin claims that for his firm's last five financial technology customers, the AI agent deployments received a better customer satisfaction rate score than the internal customer support teams at those companies.

A key to high satisfaction scores, he said, is that the AI very quickly hands off to a human if it's stumped, instead of wasting people's time by asking questions and seeking clarification.

"Half of our customers disclose that it's an AI and half of them don't," said Masin. "Among those companies that don't disclose it, actually customers never realized that they talked to AI."

However, Masin admits AI agents aren't a one-size-fits-all solution. Pointing to sales and marketing agents, he said, "whenever I try some, they are really in most cases useless. Just produce something which is very, very high value and just solves a narrow problem." ®