Presented by Red Hat
During VentureBeat’s recent AI Impact event, the focus was on distinguishing enterprises successfully scaling agentic AI from those stuck in pilot phases. Brian Gracely, Red Hat’s senior director of portfolio strategy, shared insights on the real-world challenges companies face as AI agents move into production.
Gracely discussed the importance of cost management, the security vulnerabilities particular to autonomous systems, and the internal resistance that can hinder widespread agent adoption beyond initial supporters.
Enterprises are overestimating how far behind they are on AI agents
Many business leaders, especially those attentive to industry presentations and AI updates, fear they are lagging behind competitors in deploying agents at scale. However, Gracely suggested this fear is often based on a misunderstanding of how quickly organizations can learn once they start developing AI capabilities. Teams often advance more rapidly along the learning curve than anticipated.
This swift progress introduces a new issue—rising AI costs as agent usage grows, shifting cost management from a technical concern to a frequent topic in board meetings.
The use of agentic AI has increased significantly compared to the chatbot era, raising cost concerns for businesses. Additionally, companies are becoming more aware of their reliance on a limited number of model providers. Gracely noted that this situation prompts many businesses to seek alternatives for better cost and infrastructure control.
“The two or three top providers are already telling the market that they’re losing money, and they’re trying to go public to make up those gaps,” he explained. “At some point, the dependency on that means you’re either going to buy at a very high-cost level, or you’re going to figure out alternatives to control what you’re doing.”
Right-sizing AI models is the fastest lever for cutting agent costs
The primary cost issue arises when enterprises default to the most advanced model available, regardless of whether the task demands such complexity.
“If I’m simply trying to resolve an insurance claim, I don’t need to know about the history of Western civilization in my model, I don’t need to know World Cup soccer scores,” Gracely said.
Many companies utilize semantic routing to automatically determine the appropriate model size for each request, while infrastructure techniques like caching repetitive queries reduce the frequency of GPU compute usage. As Gracely explained, these strategies challenge the notion that efficiency and innovation are mutually exclusive.
“There’s a lot you can do at a GPU infrastructure level, and quite a bit you can do in terms of flexibility of models,” he explained. “Those give excellent choices in terms of the levers you’re trying to pull, whether you need efficiency or you need innovation. That shouldn’t be a binary choice.”
Gracely compared the financial discipline required for managing token spending to the FinOps practices developed over years to control cloud computing costs. These foundational frameworks will adapt even as terminology evolves, he noted, especially as organizations emphasize internal education on model selection to prevent teams from defaulting to the most visible option for simple tasks.
“The same way we first had to teach the financial people what an EC2 instance is and what an S3 bucket is, you’re going to have to start explaining tokens to them,” he said. “We don’t always need a Rolls-Royce. We don’t always need caviar, because we’re trying to do basic types of things.”
Patch speed is now critical as AI tools find vulnerabilities faster
The discovery of vulnerabilities through AI tools is prompting enterprises to reassess their patch management timelines. Traditional patch cycles may not suffice in a landscape where AI can quickly uncover—and attackers can exploit—new vulnerabilities.
“Most companies are probably going to have a window of somewhere between seven and 14 days to stay ahead,” he said. “There are groups, Red Hat included, that are going to build patches for these, but the embargo window is going to be short.”
AI is also redefining what security professionals should monitor. Instead of identifying isolated critical issues, AI security tools can detect combinations of minor vulnerabilities that only pose a threat when linked together. As software complexity and vulnerability detection accelerate, Gracely emphasized that the ability to swiftly manage and update software is evolving into a strategic capability rather than merely an operational one.
Subject matter experts and compliance teams decide whether agents scale
Ultimately, successful organizational adoption depends on the deep, ongoing involvement of subject matter experts whose expertise the agent seeks to replicate. Their engagement is essential from the start rather than an afterthought.
“You have to think about the incentives, what you do for people who participate in this work so they don’t feel threatened that it’s going to take away their job, and how you incentivize people in the long run to cooperate with that innovation,” he said.
Sponsored articles are content produced by a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. For more information, contact sales@venturebeat.com.
